The storage industry is exciting. No, really; bear with me.
People are throwing millions of dollars at storage startups, which apparently seem more financially secure than investments like Facebook.
IBM CEO Virginia Rometty said that the problems are so large and data generation is so enormous that research centers now have to be located "in the middle of the problem."
The evolution of big-data decision-making will force every company, country, and government entity to actually become an authentic organization.
Thankfully, the way we store data constantly improves. Today, we’re in the middle of a switch from spinning disks to solid-state, flash storage. As NetApp's Paul Feresten writes:
Flash storage is a huge game-changer. Flash is incredibly fast, and it’s become much less expensive.
For the majority of cloud-based applications -- at least over the next few years -- flash will be combined with traditional hard disks, in a hybrid approach. But there’s starting to be an increasing need for storage systems that are 100% flash, particularly in transaction driven applications. The time will come when it’s realistic to do away with mechanical storage.
Storage history in brief
The earliest computers had no storage at all. EDSAC, Sir Maurice Wilkes' first fully electronically programmable computer, relied on an array of mercury tubes to store well under a thousand bytes.
Over time, the industry moved to magnetic tapes (which had high capacity, but were slow and difficult to search) and then to magnetic disks (via a brief flirtation with drums).
Hard disk drives were introduced in 1956 as data storage for an IBM real-time transaction processing computer. They were then developed for use with general purpose mainframes and minicomputers.
The first drive, the IBM 350 RAMAC, was approximately the size of two household refrigerators, storing 5 million 6-bit characters (the equivalent of 3.75MB) on a stack of 50 disks. Today, you could fit about 5 billion times the storage in the same space. The 350’s performance was glacial by today’s standards. At 8,800 characters per second, one of today's disk arrays could outperform a 350 about 100,000-fold.
In 1961, IBM introduced the Model 311 disk drive, which was about the size of a washing machine and stored 2 million characters on a removable disk pack. In 1973, IBM introduced the 3340 -- the first model to be known as a "Winchester," which set the basic design for all modern hard drives. By the late 1980s, such drives were so inexpensive that they were built into every PC, or used in external, add-on subsystems. For example, every Apple Macintosh manufactured between 1986 and 1999 had a small computer system interface (SCSI) port, into which you could plug an external hard drive.
Smaller disks -- diskettes -- had already been developed. The first "floppies" were 8 inches in diameter and could hold 80 kilobytes of data. IBM introduced the diskette commercially in 1971. Diskettes shrank in diameter and increased in capacity: first the 5¼-inch mini-floppy, then the 3½-inch disk.
When Apple introduced the Macintosh in 1984, it used single-sided, 3½-inch drives with an advertised capacity of 400KB. Both Apple and IBM went to double-sided diskettes of 720 to 800KB in 1986 and soon introduced a new 1.4MB high-density (HD) format. Compare that with today's removable medium of choice: An inexpensive USB flash key, storing 2,000 or more times that amount.
Over the past 20 years, the volume of assembled data and the need for rapid access has pushed the industry to solid-state drives (SSDs) employing flash memory. SSDs employ no moving mechanical parts, aren't subject to shock, are silent, and are rapid in both access time and raw speed. These are very important to businesses today.
Until recently, the drawback to SSDs was the cost. However, the inevitable application of Moore's law consigns that to the past. Storage is at the center of today's problems, and flash is a growing part of their solution.