This post was written by Steve Cohen, a Solutions Marketing Manager here at Dell.
Big data retention represents a fast-growing priority
Driven by industry-mandated compliance and business imperatives makes data retention a priority at the highest levels of management. Even as the cost of storage continues to fall, increasing data volumes combined with extended retention periods and on-demand access is placing pressure on Online Transaction Processing (OLTP) and Online Analytics Processing (OLAP) systems and repositories.
You have a Big Data retention problem when your retention needs require you to add resources and capacity at an alarming rate.
Since OLTP and OLAP systems are optimized for high-performance transactional and analytics, respectively, they have a corresponding high operational cost point to match. Continuing to hold historical, less frequently accessed data significantly degrades the performance of these production-critical systems, as well as magnifying the cost of specialized storage, servers and people skills that are required for ongoing operations.
Machine Generated data is a huge driver of Big Data and the need for efficient Big Data management.
Machine-generated data is being defined in different ways by different people. Some say it is strictly data that is generated without any direct human intervention, and others say it is data that includes the machine tracking of human activities as well. But whether the definition of machine-generated data is precise or a little more open, certain key characteristics nearly always apply:
- new records are generated with a high frequency
- the data itself is never changed
MGD is a leading driver of the big data explosion
Much of this data is generated from but not limited to:
- Image, audio, or video files
- emails, logs, call detail records
- documents, medical images, movies, gene sequences, data streams, tweets
- RFID tracking, smart sensors, mobile devices with geospatial information,
- scale-out cloud clusters to systems, PCs, mobile devices, and living rooms
- growing array of data collection devices
As this data ages it statistically is accessed less and plays a diminished role in critical business operations creating less value, and can therefore be targeted to be moved to a less expensive tier of infrastructure. Data can have long-term value if it is required to be retained for regulatory compliance or legal needs in which it has value associated with risk mitigation. Data can also find long-term value as an asset if it provides some kind of analytical value or resell value. All of these cases not only require retention, but also the ability to discover and analyze the data in a timely manner. The ultimate goal of IT is to drive down the cost to maintain data. IT is striving to strike the balance between data accessibility and analytics at the right cost.
New data challenges require new data solutions
As we are in the era of Big Data, the reward for investigating and understanding newer, innovative technologies can be equally big, both in terms of time-to-market and cost savings. When selecting a storage and data management partner to help you manage the “Big Data” challenge, you need a partner that addresses the entire spectrum of data assessment, data retention, and data use requirements of this new environment.
Check out this research by IDC which reveals how organizations can address the “Big Data” challenge and how Dell tackles this extremely important challenge.