The AI Revolution Could Use Some Historical Perspective

The unprecedented progress of AI technology over the past couple of years has many in awe. They foresee an entirely new future where AI provides insight into everything and is woven into every fabric of existence. There may be some truth to that, but some of the predictions should be taken with a grain of salt. Historical perspective is needed on how technology changes things – and how the basics of computing continue to apply regardless of breakthroughs that appear to disrupt everything.

So said Philippe Nicolas, founder and analyst at Coldago Research, during a recent Active Archive Alliance webinar, WHY AI NEEDS ACTIVE ARCHIVE. He began by revealing a survey that asked people about their top three IT innovations since 2000 that positively impacted data storage. 82% of users chose media developments, with Flash being top of mind, as well as better interfaces such as NVMe. The advent of flash storage has opened the door to faster and more efficient storage as well as improved tiering.

In second place at 74% came means of access, particularly HTTP/S3. S3 (Simple Storage Service) brought about a surge in cloud storage. By storing data as separate objects along with metadata and unique identifiers, users could find files directly in the address space using the HTTP API. This made life a lot easier for archiving (S3 to tape), backup, disaster recovery, and analytics applications.

The third top answer to the question was operating system advances such as hypervisors, Linux system services, containers, Kubernetes, and the cloud operating model.

A review of these breakthroughs makes a couple of things clear:

*While some of these advances were major, their rollout in the enterprise was gradual. Things did not shift overnight.
*Plenty of problems and challenges still lay ahead for those attempting to implement these technologies in the early days. Cybersecurity, of course, was a major hurdle, as were the presence of legacy technology, the acquisition of the necessary skillsets, scalability, and the availability of tools that could harness new capabilities. The entire vendor ecosystem had to play catch up to ensure their hardware and software were compatible with the latest technologies.
*Any breakthrough, AI included, suffers the same teething troubles and takes years to mature.

“AI may be everywhere and is regarded as the new El Dorado, but it will take time, and the basics still very much apply,” said Nicolas. “AI needs access to large amounts of data and for that data to be stored for long periods of time.”

He also drew attention to the dangers AI presents. Anyone using public AI engines that submits confidential or corporate data poses a risk of that information escaping either into the public domain or into the hands of unscrupulous individuals and groups.

In some ways, AI might be regarded as the latest evolutionary step of a shift that has been ongoing for a quarter of a century. Without new media, better access methods, and the cloud operating model, AI would be hamstrung. Add to that list the latest wave of processors and accelerators such as GPUs. Each wave of innovation generates the next generation of tools and technologies.

Storage for AI, AI for Storage

Many of the breakthroughs over the last 25 years are coalescing into new forms of computing. The world of storage plays a critical role in embracing these innovations. Storage and archiving infrastructure are needed that are adequate for the task of supporting AI. That means low latency storage that provides high bandwidth and IOPS. For example, parallel file systems and full flash S3 are essential in training AI models. For inference, the full range of storage tiers and media can be utilized in the service of AI.

But there is another side to this. Just as storage will advance to serve AI, AI, in turn, will provide storage with new features and capabilities. AI embedded in storage applications enables learning of patterns, I/O needs, content, and more. It can enrich data policies and add autonomy and self-protection features, enabling far more granular tiering.

All of this adds up to a new era for AI-based active archiving. The vast majority of data will exist in a secondary storage archive. With S3 API integration, it is now easy to access data on tape.

“S3-to-tape is the new killer app for tape and optical storage of archived data,” said Nicolas. “S3 is the necessary bridge between AI and secondary storage, especially active archived data.”