Archive and long-term data storage requirements are growing at a record pace. As a result, many businesses are wrestling with deciding whether to build their archive strategy on disk, tape, or a combination. According to IDC's June 2011 Digital Universe Study, the world's digital data is more than doubling every two years with approximately 1.8 zettabytes to be created and copied or replicated in 2011 and up to 7.9 zettabytes by 2015. In addition, individuals now generate up to 75 percent of all digital information. According to the study, the magnitude of 1.8 zettabytes of data is equivalent to:
- Every person in the U.S. tweeting three tweets per minute, non-stop, for 26,976 years
- Every person in the world having more than 215 million high-resolution Magnetic Resonance Imaging (MRI) scans daily
- More than 200 billion high-definition movies (each two hours in length), which would take one person 47 million years (24 hours a day, non-stop) to watch every movie
- The amount of information needed to fill 57.5 billion 32GB Apple iPads. With that many iPads we could:
- Create a wall of iPads 4,005 miles long and 61-feet high, extending from Anchorage, AK, to Miami, FL
- Build the Great iPad Wall of China—at twice the average height of the original
- Build a 20-foot high wall around South America
- Cover 86 percent of Mexico City
- Build a mountain 25 times higher than Mt. Fuji (more than 12,000 feet tall).
Fixed content, multi-media, broadcast, social networks, and compliance regulations define what’s commonly referred to as tier 3 data and are growing more than 60 percent annually. For many applications, the lifetime for data preservation has become infinite; the data is never deleted. This further increases the overall size requirements of the digital archive.
The growing list of government compliance and legal regulations worldwide describes the way data should be managed and protected, and how long it should be stored throughout its lifetime. For example, certain medical records, such as X-rays and MRI images, need to be kept for the lifetime of the patient, though they may only be infrequently referenced after they’re generated.
High-Capacity Applications Accelerate Tier 3 Demand
Much of the fixed content, compliance, and archive data that compose tier 3 are unstructured data in the form of file storage. Industry estimates indicate that more than 70 percent of all digital data is unstructured. Unstructured data typically includes bitmap images or objects, text, photos, video, and other data types that aren’t part of a database. The social networking wave has quickly become a major generator of low-activity, unstructured digital data that’s kept or archived indefinitely. Figure 1 lists some of the primary applications and drivers of tier 3 and archiving storage requirements.
Some traditional tape and disk applications have been trading places, further increasing tier 3 demand. While some backup applications are moving from tape to disk, given disk’s faster restore times, tier 3 archival applications are moving to more cost-effective tape storage. Many people confuse backup with archive, though they’re quite different functions. Backup is a copy function; archive is a move. Archive data can, and normally should be, backed up (see Figure 2).
Disk Drive Scenario: Challenges Are Mounting
Despite a difficult economic scenario globally, and an unprecedented decline in Hard Disk Drive (HDD) shipments for enterprise applications in 2009, HDD vendors continued to innovate new products to address enterprise storage demands. According to research from International Data Corp., HDD shipments for enterprise applications are expected to increase from 40.5 million units in 2009 to 52.6 million units by 2014. That will deliver more than 300,000 petabytes of storage capacity over the next five years to enterprise data centers and clouds. Industry consolidation has left only three HDD manufacturers compared to nearly 60 in 1980, greatly reducing the number of competitors and customer choices. The transition from 3.5-inch to 2.5-inch form-factor HDDs is expected to be complete by 2012.
Is disk a good solution for archive applications? Disks store most of the world’s mission-critical and high-performance data, making higher reliability and availability levels mandatory. However, disk drives are now facing some mounting challenges (see Figure 3):
- Disk drive rebuild times are becoming excessive as drive capacities steadily increase. In the early ’90s, it took about one minute to rebuild a disk drive. Today, a 2TB disk drive in a Redundant Array of Independent Disks (RAID) can take eight hours or more to rebuild. Within the next several years, expect to see 30TB disk drives which, if nothing changes, will take nearly a month or more to rebuild!
- Disk drive capacity is increasing at 30 to 40 percent annually but there are minimal performance gains left for disk drives, creating a growing access density problem.
- Disk has also fallen behind tape in reliability, which is a surprise to many, because the Bit Error Rates (BERs) for tape now exceed disk.
- The Total Cost of Ownership (TCO) for tape systems is now much lower than the TCO for disk systems.
The disk industry will have to address these mounting problems in the near future.