Don’t Fear the Reaper, Active Archives are Here!
Storage Visions 2018, held in Santa Clara this past October, was a great opportunity to discuss the deluge of data that is being created and how to effectively manage and store that data for the long term. A common theme was the tsunami of data coming at us from 16 ZB today to 163 ZB by 2025, according to IDC. Santa Clara was also a fitting location as it is the heart of the artificial intelligence (AI) industry and as AI evolves, it will help us analyze and manage the data deluge more efficiently.
As a representative of the Active Archive Alliance, I had the opportunity to present on the topic of “active archive” and to participate on a speaker panel: “Don’t Fear the Reaper – Storing Data for The Long Term.” The premise of the panel was that the data deluge will result in increasing amounts of data that have long term value and that we need to make sure that this data survives well into the future.
Long term data protection and retention may be more of a challenge than we think when we look at the projections from IDC for all data created which factors in a 30% CAGR. But as one vendor pointed out on our panel, the rate of growth for unstructured data is actually much higher than structured data, more like 60% growth per year.
Then there is the simple classification of data between “hot” and “cold” with industry pundits saying that about 60% quickly becomes cold and therefore is best suited for lower cost archival storage tiers. But as unstructured data has a faster growth rate than structured data, some industries generate cold data at a much faster and higher rate than others. For example, in the healthcare industry 82% of data quickly becomes cold, and in the Media and Entertainment (M&E) industry 88% of content soon becomes cold.
The tricky thing about long term data retention is that the value of data can change over time, from hot to cold to hot again. Think of the M&E industry as a good example of this in the case of a celebrity experiencing a major life event. Suddenly a highlight reel of a life in Hollywood will need to be produced for the evening news.
One of my fellow panelists said it best, if not informally: “Companies don’t want to seal up data in a can and put it on a shelf.” Clearly in today’s fast paced 24/7 economy, everyone wants accesses to their data quickly. All of the data, by the way, and make that for an indefinite period of time.
But how do you satisfy that kind of a service level agreement in the face of exponential data growth and the ever present budget constraints faced by all IT departments? This is where active arching comes in offering modern strategies for solving data growth challenges while enabling efficient access to all your data throughout its life cycle.
In a typical active archive environment, data is managed from terabytes to exabytes across multiple storage tiers without adding complexity and is transparent to the end user. Intelligent data management software automatically aligns different types of data with appropriate storage tiers thereby reducing TCO and freeing up expensive primary storage. Data is automatically migrated as it ages from tier to tier and to newer generations of media type within a tier such as the latest in HDD technology or LTO tape to enable long term retention and accessibility.
While intelligent data management will continue to be enhanced with the evolving capabilities of artificial intelligence, look for active archive environments to evolve as well to provide solutions to the impending data deluge.