The Technology Push to Meet the Growing Demands of Archive Storage
The Active Archive Alliance recently published their 2021 Data Storage and Active Archive Predictions and Trends. That article included a prediction from Scott Hamilton of Western Digital indicating there will be new storage technologies and enhancements to support cost-effective active archives. The reason this is so compelling is, as Scott alludes to in the article, the highest growth segment of data comes from “secondary” storage use cases, which include a range of use cases from active archives to non-active cold storage.
As has been the case, and will be for the foreseeable future, the tech world will be battling to service this archive space in ways that are most appealing for both IT and the data owners. Simply put, retaining the data is one thing, but having the ability to more readily access and monetize the data is the ultimate objective. The corporations that store this data will be looking for the technology(ies) that provide the desired access capabilities at the lowest TCO. IT will be looking for solutions that are easy to deploy and maintain (and of course meet their users’ expectations). The users that own the data will be looking for technology that provides access to their data when and where they want it. The archive space is one of the most interesting, multi-dimensional parts of the market. There are several metrics that are key to this space – and this is what differentiates the myriad use cases out there.
Key considerations for archive storage solutions:
♦ Time to data (how long it takes to get data after the request)
♦ Frequency of access to data sets
♦ Cost per TB (Includes many components):
◊ Storage media cost
◊ Cost of power
◊ Cost of environmental control (temp and humidity)
◊ Server cost (all the hardware in addition to the storage media)
◊ Software cost – this likely varies for different solutions
◊ Cost of protection (if primary copy of data)
◊ Other (use case dependent)
♦ Management complexity
I will focus this blog entry just on the media – and perhaps in subsequent entries will elaborate on the other factors.
Virtually all storage media types have an opportunity to compete for this space, and the most visible media technologies include flash, optical, HDD, and tape.
While flash is running fast in declining $/TB, it is still multiple times more expensive than HDD and will be for the foreseeable future. And while tape has been the long-standing leader in low $/TB cost, the gap between tape and HDD continues to narrow.
I’ve been talking to customers with archive challenges for many years and watching (and even offering) various solutions, some based on tape and some based on HDD. What I have found over the years is most customers are not very compelled to switch from disk to tape unless the value proposition is more than an order of magnitude better, and for smaller customers (less than a small number of petabytes) the pain just isn’t great enough to motivate the switch to tape. The good news for these customers is the cost of HDD storage, like all other media, continues to decrease. Recent steps toward lowering $/TB and TCO include helium sealed drives, triple stage actuator, energy assist, and shingled magnetic recording (SMR), and there are many other technologies in development that will continue to push the HDD boundary.
Source: Western Digital. The areal density and $/TB push for HDD and Tape Technologies (source: https://www.insic.org/areal-density-chart/)
As for tape, the LTO consortium continues to push hard on tape’s evolution as evidenced by their published roadmap. Each generation of LTO has historically been released somewhere between two to three years after the previous generation, so that provides some temporal perspective on when these capacity points may be realized.
As indicated above, the difficulty to switch from disk to tape is not compelling in the single-digit PB range. However, we are now seeing organizations with exabyte scale, and for those environments, the cost benefits of tape as it stands today has been very beneficial. The question is, which technology, tape or HDD will deliver the best TCO for larger environments three years from now? Or, to what extent do they coexist to address different SLAs? I am excited to be in this race – the stakes are high, and the pace of innovation is staggering.