admin's blog

Is Using Backup Software for Archiving a Good Idea?

Author: Valery Guilleaume, CEO, MT-C

A common problem in IT environments today is the fact that backup and archive are often conflated, resulting in too many organizations trying to use backup software to handle archive requirements. In fact, the last decade has shown that dedicated archiving products have struggled to gain widespread adoption. Archive is often perceived as difficult and disruptive, and something that is important for the long term that can be deferred for another time. Backup is seen as an immediate need that is about protecting today’s work. To illustrate the situation, let’s take the example of analog or ‘paper’ archiving, and digital archiving.

Companies have long-established processes for managing their paper archives. Therefore, transforming this data to a digital form would require time and add costs that organizations typically are not ready to invest in, and often require a complete change of their current workflows. The result is that backup is often used as a short-term solution to a long-term problem, despite the fact that backup solutions are poor substitutes for an effective digital archive strategy.

Today, we are facing a consumerization effect as enterprises, product administrators, and product end-users desire more easy-to-use archiving systems that can be self-service, or require minimal or no IT intervention. Consequently, “old school” archiving products are no longer cost-effective enough, and are too complex to implement and maintain, particularly in heterogeneous storage environments. In addition, digital archives require an understanding of multiple business and user data flows. A true policy-based enforcement for such traditional systems is required at the enterprise management layer, which often are monolithic, and inflexible. The result is that such systems were costly to implement and created disruption to users, which made it difficult for top management and justify. The result is that enterprises have to manage archive differently.

Historically, via their backup software, they managed to create an archive job in order to move data from expensive primary storage to lower cost media. However, enterprises now understand the drawbacks to this approach. First, if end users wish to retrieve data from their archive, they have to ask their IT service to move back the required information. However, the Service Level Agreement (SLA) to do such a job is not in alignment with the users’ needs and requires days to get any information back. Secondly, administrators are constrained to stick to an application/product even if they want to migrate to another one. To do so, they have to migrate all “backup” data, resulting in a lot of headaches and time. And lastly, due to compliance reasons, it is hard to verify which data has actually been archived. For instance, on which media does the data reside, in what state are the data and media, can we retrieve the data, and how we can make sure that the data will still be available for the duration required? 

These are complex questions and situations that more and more customers are facing. So, how can we reduce these risks today and resolve these issues? We have observed through our experience the following opportunities:

Reducing Vendor Lock-in
Avoid using software which is linked to a single vendor to avoid lock-in. Using a storage system that can manage data copies on LTFS tape is a good choice. LTFS is a way to write data on tapes in a file system, in an open format which allows access to anybody having a LTO drive (LTO5 and higher) to read any LTFS tape.

Online Data and Metadata
Catalogs Using systems that provide online and searchable catalogs of the contents stored and archived gives users the visibility they need. With this view, they can access their data as soon as they need it. Another advantage is that due to compliance requirements, it is necessary to verify that the data has been copied to dedicated archive tapes.

Hybrid Storage and Active Archival System
The ability to virtualize a file system built on hybrid storage including flash/disk/LTFS tapes provides an attractive option. Typically, 80 - 90% of your data is rarely accessed, whereas 10 - 20% is regularly used. By using a hybrid solution, this data would be transferred to the right place in order to reduce costs while being easily accessible at any time. Hybrid storage helps to reduce the TCO cost of the solution. Solutions exist to store massive data at a scale of hundreds of terabytes for a CAPEX cost of $400-$600 per TB, and an OPEX cost of $0.75 per month per TB! This leads to significantly low investment and operating costs. Now imagine the same calculation using only disk storage.

Open Solutions
This becomes even more compelling when automation techniques are used, which use metadata-driven control to link this type of storage and archiving solution with business applications.

It is important to understand the opportunities of a hybrid storage and active archive system. Companies should not be afraid of making changes to their archival system, as this approach would give them the chance to remove complexity and provide them with significant added value while successfully dealing with unprecedented data growth.

Active Archive Delivers Performance, Reduces Energy Consumption

Author: Rich Gadomski, VP of Marketing, Fujifilm Recording Media USA, Inc.

Today’s IT leaders are focused on every possible way to drive performance and extract as much value as possible from their systems and technologies while striving to conserve energy and reduce waste. One critical area that IT leaders are looking at continues to be data storage and the need for long term data archiving is at the forefront of delivering value to organizations of all sizes. The challenge is how to store more and more data cost-effectively without reducing performance or impacting user service level agreements.

An active archive implementation can be the perfect solution where users have unified online access to all of their data across multiple storage systems and media formats whether on flash, disk, tape or cloud. In an active archive, fixed content residing on primary storage is moved by policy to economy tiers of storage such as tape systems thus freeing up more expensive and energy intensive storage systems such as flash and disk.

Considering that the world’s data centers now consume almost as much energy as the country of Spain and consume just over 2 percent of the total U.S. electrical output, reducing energy consumption has increasingly become a key initiative for IT leaders. Two of the highest areas of IT energy consumption are related to servers and disk storage.

While cutting edge data centers rightly focus on including renewable sources of energy to help power their data centers and reduce their impact on the environment, best practices also dictate moving less frequently accessed data from the expensive, higher energy consuming storage tiers.

According to the Tape Storage Council’s 2017 State of Tape Industry report, energy costs for tape capacity are typically less than 5 percent of the equivalent amount of disk capacity. Since tape cartridges spend most of their life in a library slot or on a shelf, they consume no energy when not mounted in a tape drive. As archival capacity demands increase, tape capacity can be added without adding more drives.

The business case for active archive is compelling thanks to its ease of use and cost containment. Tape technology offers the most cost-effective, reliable, and energy efficient solutions available to the storage industry and is a key enabler for active archive implementations.

 

The Benefits of Active Archives for Media Content

By Laura Shepard

The continued growth of higher resolution formats and the evolution of the Internet and online distribution of media content, is changing the storage needs of content creators and their workflows. Higher resolution, increased frame rates and more channels of audio are making production storage burst at its seams. In addition, many assets that previously would have been discarded - or put into deep, inaccessible, long term, inexpensive archives - are now being demanded by our increasingly connected, news hungry, media-centric world.

At the same time, archive owners are increasingly being challenged to keep their archives viable, usable and manageable, as these higher resolutions increase project storage capacities. Archives are growing faster every day. Decisions need to be made about long term data preservation technologies that may be difficult to transition from, as technology evolves.

Another challenge for the media and entertainment industry is that many companies are becoming more geographically distributed and their workflows are encompassing users from around the city, country or even the world. These workflows, allow for a continuous 24 hours of production between locations. Collaborative workflows are difficult to support with standard storage and archiving solutions. Tracking and synchronizing content and versioning is difficult to implement, clunky to use, and often requires the users to wait for content. The truth is most archive owners would choose to keep every asset on a disk-based solution, if they could only afford to do so.

Highly responsive, active archive solutions are needed to relieve the demand on expensive tier-one storage and to bring these assets online, without breaking the bank. Active archiving facilitates collaboration and massive storage needs of media and entertainment companies.

An active archive based on object storage provides high access speeds, low latency location-awareness with optional replication and caching that make it an ideal solution for online content delivery.  It can scale to hundreds of billions of objects seamlessly, in a single name space. The integrated solution also delivers short-term storage with backup capacity and long-term archiving capabilities that supports the preservation of data with its associated user metadata.

Archives are undergoing a radical transformation, fueled by rapidly growing file sizes and new access requirements. An active archive enables users to archive content sooner reducing the cost of tier-one storage and creating new opportunities to monetize content. In addition, active archives increase data access and speed, at a substantially lower price point than traditional RAID disk-based solutions (and not much more than tape).

A Fresh Archive Solution for the Modern Data Center

By: Eric Polet

Modern enterprise data centers are some of the most technically sophisticated operations on earth. Ironically enough, they are also often fortresses of inefficiency, with some equipment being utilized less than 10 percent of the time, and servers being ineffective 30 percent of the time (using electricity but performing no useful information services). Storage administrators struggle to keep pace with rapid changes in computing equipment deployments and the ongoing costs of maintaining a responsive storage environment.

These problems have led organizations to focus more closely on improving their data center management. While almost every enterprise data center has taken steps to improve its operations, virtually all are less efficient, more expensive, and less flexible than they could be. Those shortcomings ultimately prevent data centers from delivering maximum business value to the organizations that own them. 

Enter active archive enterprise storage solutions. A data center built around the core principals of an active archive provides an organization access to all of its data, all the time. Through unified management, regardless of tier, users have the most responsive archive implementation available that frees up valuable space on their Tier 1 storage and maximizes the value of their data. With the ability to respond to future growth and evolve as data expands, an active archive solution is one that will grow with your organization, delivering a responsive archive, built to adapt to your changing storage landscape.

As organizations continue to face challenges of storing data in this age of exponential data growth, implementing an archive is  the most affordable way to store data for the long term. Active archive solutions provide scalable data storage, designed to meet the requirements of modern data centers. With integrated policy based management to intelligently manage how, and on what tier data is stored, organizations get the highest return on investment for their storage solution. Active archive solutions bring together leaders in the storage industry to construct a customized archive solution that meets the needs of today’s enterprise organizations.

 

Active Archive Alliance Welcomes New Member MT-C

By Peter Faulhaber

MT-C Logo

We are pleased to welcome our newest member, MT-C, to the Active Archive Alliance. MT-C simplifies access to content through its intelligent catalog and metadata, and brings a unique solution for an active archive that lowers storage total cost of ownership.

MT-C’s solutions are a great fit with the Alliance, which is focused on promoting active archives for simplified access to all of your data, all of the time.  Active archive technologies include file systems, active archive applications, cloud storage, high density tape and disk storage.  

Here’s a little background on MT-C and its core active archive product NODEUM:

  • Belgium-based MT-C is a fast growing software engineering company specializing in data storage and protection systems. The company provides solutions for companies that need guaranteed and flawless protection for their data, and serves a multitude of industries including biomedical and genetics research centers, cloud service providers, broadcasting and post-production, spatial centers, video surveillance, retail, banks and insurance.
  • MT-C’s key solution, NODEUM, is a highly scalable, hybrid software-defined storage platform that virtualizes attached storage nodes as flash, disks, high capacity tapes and cloud. Its embedded file management classification feature allows organizations to analyze their data content and to predict their future needs in terms of ILM (information lifecycle management). Its integrated content catalog, custom metadata tagging and search engine provides the classification to store, and retrieve massive volumes of structured and unstructured data. Storing and accessing data has never been easier. Futhermore, NODEUM lowers storage total cost of ownership with its high-density capacity and optimized active archival system and offers optimal, secure and long-term management of data.

With today’s fast-paced proliferation of long-term data and the growth of data sources and types, active archives are seeing increasingly strong interest and demand. We continue to expand memberships in the Active Archive Alliance with companies whose products support active archiving and that enable organizations to implement the best solution to fit their specific long-term storage and archiving needs.

If your company is interested in becoming a member of the Active Archive Alliance, please visit: http://www.activearchive.com/membership

What is so Special About Metadata?

A cognitive strategy to manage the data explosion

By Floyd Christofferson

When researching ways to contain rising storage costs and reduce the complexity of heterogeneous storage environments, it is natural to look at storage solutions for ways to solve these problems. Data lives on storage, so it seems reasonable to assume that the answers to managing the explosion of data would be found in various storage options.

The storage industry is naturally focused on storage-centric answers to the problem of data management. To a hammer, everything looks like a nail. But even the best storage products cannot do this alone. Storage-centric solutions simply do not have the intelligence about the data they store, nor were they designed to work across multi-vendor storage types, different file systems, or protocols.

The good news is all digital files contain multiple types of metadata that can drive a data-centric solution to the problem, and in the process bridge incompatible storage types and use cases. Metadata is literally data about the data. Think of it as a roadmap that gives you a bird’s eye view of everything about your data, and which can drive data management policies that transcend the storage layer.

As an example, data-aware management solutions can leverage the intelligence derived from multiple types of metadata to pro-actively plan for and implement storage optimization, data protection, workflow automation, business continuity, and other tasks.

There are many different types of metadata, starting with file system metadata, such as file name, create time, access time, modify time, etc. But most file types also include additional rich metadata in headers, such as geospatial coordinates or other information that can drive workflow policies. And then there is external metadata which organizations may have accumulated, which may live in other databases or records, such as project-related tags, or other information about their data that captures business value, retention policies and more.

As all of these metadata types are coalesced into an aggregated environment, a metadata-centric solution becomes data-aware, or intelligent enough to automate data and storage management without needing to alter the underlying storage infrastructure.  

This is also crucial for implementing an active archive strategy in a heterogeneous environment. Metadata-driven policies can move data anywhere, on any storage type or location. So the active archive can truly be universally accessible by any authorized user.

So rather than trying to physically normalize all the data at the infrastructure level to overcome silos, a data-centric strategy does this in metadata to enable global management across all storage types, file systems, and locations.  And as policies or use cases evolve over time, metadata-driven strategies ensure that the data lifecycle requirements can be implemented regardless of the storage types existing today.

In this way, metadata can drive data placement policies to virtualize any storage type including archive. So the active archive can be online and accessible for all users, whether it is on disk, tape, cloud, or a remote site. Such metadata-enabled systems also enable users to search on any metadata fragment, to find the data whether in active archive, or a disaster recovery site, or any other storage tier anywhere.

Data is the lifeblood of every organization; but protection and accessibility also need to be cost effective and flexible. By looking at data management solutions that can leverage the power of metadata, a whole new horizon of possibilities opens up to customers to manage any data, on any storage anywhere.

The Role of Cognitive Data Management in Active Archive Architectures

By David Cerf

Today’s storage environments are comprised of a complex mix of file and object storage, and various file  systems each with their unique behaviors. Due to the dependencies between clients and storage resources, users can find it difficult to deploy an active archive for migrating or tiering to cost effective storage. Complex file storage infrastructure can result in low utilization and limit data mobility. The continued growth in unstructured data (files & objects) is increasing the complexity, costs, and operational overhead for data management.


How metadata is changing active archives

Embedded in every file is a treasure that can be used to improve data management and power an active archive, it is metadata. Metadata may be just a simple description of a file but it has very relevant information about the data. When combined with other metadata, policy engines and artificial intelligence, metadata becomes an incredibly valuable source of hidden information that can be used to drive data lifecycle management, improve workflow and enhance applications. Most importantly, this metadata can drive an active archive architecture, using metadata as the trigger to enable data defined tiering and data protection.

Data management is now “cognitive”

Most of today’s file-based workflows have no way of discovering or making use of this information in a simple to use, automated manner. But new cognitive data management solutions with their policy engines, are designed to complement any existing storage environment and enable an automated active archive. This means any file system can now add tape and object storage easily, improving data preservation and retention, search and collaboration, resulting in evergreen storage strategies for simplified migrations and tiering.

2017 Data Storage and Active Archive Predictions

Over the past year, new storage technology innovations for active archives have enabled organizations to gain reliable access to all of their data, all of the time. As a result, these organizations have experienced increased cost savings, decreased energy consumption and improved storage administrator efficiency.

The key drivers that will impact the continued use of active archive in the future include the decreased cost of flash storage, greater automation, the rise of tiered storage workflows and the growth of tape in public and private cloud infrastructures. 

Members of the Active Archive Alliance recently shared their perspective on the outlook for data storage and active archive in 2017.  Here is a list of the top trends to watch:

         Automated Policies and Artificial Intelligence Come to Storage

Greater automation and the use of artificial intelligence (AI) will simplify storage management and the use of active archives. New technology will help resolve two of the greatest challenges facing data management - data classifications and storage classifications. New software tools will use metadata to power automation providing a simple solution for data management.

 

          Tiered Storage Workflows on the Rise

As the cost of flash storage decreases and disk continues to struggle to maintain its cost and capacity curve, customers will look to adopt tiered storage workflows. Flash’s useable cost could drop below $1.00 per gigabyte while tape in large systems will cross below $.06 per gigabyte near line and below $.03 per gigabyte for offline or cold storage.

 

          Energy Consumption Challenges Drive Active Archive Adoption

Active archiving will get a boost as organizations continue to seek ways to reduce energy consumption - a significant and growing component of operating expenses for today’s rapidly expanding data centers. With servers and HDDs consuming more than 30% of the energy required to run IT hardware, data center managers will look for ways to reduce utilization of power-intensive hard disk drive technology. Less frequently accessed data will be moved to an active archive where it will remain accessible but consume less power on more economical storage tiers such as automated tape libraries on premises or in the cloud.

 

         Onramps Drive Growth in Tape Usage

Tape usage will grow substantially as a key component of public and private cloud infrastructures for cold and active archive data. Tape’s inherent attributes of low-cost, reliability and even portability combined with the increasing availability of file and object-based onramps to tape will accelerate adoption of tape beyond its historical role as a target of backup and recovery software. Solutions that offer data management intelligence and that integrate well with storage targets will alleviate historical management burdens associated with tape automation deployment, further fueling tape’s penetration of cloud infrastructures where long term preservation and access to that data are required.

 

         Cloud and Object Storage Bring More Flexibility to Archived Data

Object storage software can transform an archive into an active archive that is positioned between high-performance storage and tape. A combined cloud and object storage infrastructure provides additional capabilities allowing users to collaborate at LAN speeds and access data from any device, anywhere in the world. As content continues to grow, an active archive can scale seamlessly to billions of objects in a single namespace. Flexible, user-defined data protection options will be key in making this a reality.

 

         Ethernet Continues to Gain Market Share

Ethernet is winning. There are still a number of ways to connect storage and active archives to compute - Fibre channel, Ethernet, SAS, SATA, and InfiniBand. Even though some of these connections are lighter weight from a protocol standpoint, Ethernet will continue to gain market share as the external host connection means.

 

The following Active Archive Alliance members contributed to this list: DDN Storage, Fujifilm Recording Media USA, Inc., Spectra Logic, Strongbox Data Solutions and Quantum.

Active Archive in the Summit Spotlight

By Rich Gadomski

I recently returned from the great city of Boston where I had a lot of fun with the locals who have a “wicked” sense of humor and a funny accent I can’t imitate being from New York. I was in town facilitating our 8th Annual Global IT Executive Summit. The theme this year was “Exploring the New World of Storage.” Boston was a fitting location given its history in shaping our country and the storage industry too. Speakers from the analyst, vendor and end user communities presented on the latest trends that are emerging in a new world of storage driven by so many innovations in flash, disk, tape, cloud and data management software.

One subject that kept popping up was the need to control runaway costs associated with unrelenting data growth that’s compounded by long-term retention requirements. A common solution that many speakers referenced was the need for a well-planned tiering strategy where data moves as it ages from expensive tiers of primary storage to secondary, tertiary and even the cloud as a fourth tier. Speakers presented on various solutions to manage data growth with long-term retention requirements stemming from compliance regulations, protection of business assets and content, disaster recovery and big data analytics.

A key concept that got a fair amount of the spotlight in the tiering conversations was active archiving solutions. In an active archive, long-term data remains cost effectively online and easily accessible by leveraging innovative and integrated solutions that intelligently manage data across flash, disk, tape and the cloud. This is really where cost savings can come into play by matching data types to the right tier of storage.

It was pointed out by more than one speaker that an active archive solution can be implemented with existing storage equipment and without changing workflows while being transparent to the end user. Since we can’t predict when archival data will be needed again and with that data being kept for longer and longer retention periods, what’s needed are solutions that are cost effective and automatically migrate data from generation to generation of low cost yet reliable media such as LTO tape.

It was also noted during the conference that with today’s advanced tape solutions, long-term affordable active archiving is a reality. Tape now offers: the lowest TCO for long-term storage thanks in part to its high capacity and very low energy consumption; the best reliability as measured in bit-error rate; transfer speeds greater than HDD at 360 MB/second; and a long archival life of 30+ years. Speaking of capacity, the potential for 220 TB on a single tape cartridge based on Barium Ferrite technology has already been proven. Barium Ferrite will enable achievement of the tape technology roadmap plans well into the future and will support active archiving for many decades to come.

Now if we can just get the Bostonians to pronounce the “r” in active archive, we’ll be all set!

 

 

New Active Archive Webinar Series

Join us for an exciting webinar series presented by leaders from the Active Archive Alliance. Our goal is to align the education and technologies needed to meet the rapidly evolving requirements for data archive.

Alliance members strive to extend solutions beyond the high-end supercomputing and broadcast markets to the greater general IT audience that is in need of online data archive options. The following three-part series will help to educate and promote active archive strategies for data storage.

Thursday, June 2 at 8am Pacific/11am Eastern

Object Storage May be the Cloud You are Looking For

  • Why use object storage?
  • Learn more through examples from organizations currently using object storage in an Active Archive solution.

    Hosted by: Quantum/HGST

Tuesday, June 7 at 8am Pacific/11am Eastern

Best Practices in Leveraging Active Archives to Solve Data Protection and Cloud Requirements

  • What is block, file, object?
  • How is this defining new approaches to active archive and object storage?
  • What considerations should customers take into account when implementing?

Hosted by: Spectra Logic/DDN Storage        

Tuesday, June 21 at 8am Pacific/11am Eastern

MLB Network Hits Home Run with Active Archive     

  • What is active archiving and why is it important today?
  • A quick lesson: backup vs. archive - what’s the difference?
  • Technologies used for archiving: tape, cloud, disk, flash and data movers.

Hosted by: Fujifilm/StrongBox Data Solutions       

Register now to learn more about how an active archive can give you access to all your data, all the time: http://bit.ly/1R6nUk2.