Managing exponential growth of data

With a general perception that data storage is cheap, the chosen solution to deal with growth in data is often to simply throw more hardware at it. How often is this action taken only to find out 3 months later that the same problem has returned and more space is required? Increasing capacity is part of the solution and needs to be considered as part of long-term data storage strategy and retention policies.

The notion that data storage is cheap is very subjective and depends on many different factors beyond the price of disks. The acronym RAID, which originally meant ‘Redundant Array of Inexpensive Disks’, is somewhat misleading in that it conveys the message that storage is cheap without considering many other factors including:

  • The costs of other hardware requirements
  • The costs of physical space in data centres
  • Employment costs
  • Maintenance
  • Ongoing support

The overall cost of data storage is more importantly relative to the value of the data being stored. RAID now more commonly means ‘Redundant Array of Independent Disks’, which seems more appropriate.

Parkinson’s Law states that one’s work will expand to fill the time available to complete it. The same principle applies to space: a requirement for storage will increase until storage capacity has been reached. Buying a second filing cabinet has the long-term effect of doubling the quantity of documents stored. Notice at home that the same applies to cupboards, shelves and coat hooks, and how often a spare bedroom fills up over time. A corollary of Parkinson’s Law relating to growth of data is that stored electronic data will expand to fill whatever storage space is made available for systems to use.

The key areas to be fully explored before investing in new hardware are:

  • Housekeeping – cleanup of historical data storage where appropriate to reduce strain on systems
  • Ongoing policy – decisions on what data is stored and for how long
  • Capacity planning – projecting future storage requirements and proactively planning any storage expansion

Reducing the need for storage has an added benefit to the environment of reducing energy consumption.

We are committed to saving energy and resources. We offer our clients a challenge to use housekeeping, sound policy implementation and capacity planning to reduce storage requirements, and to contribute a portion of any financial savings from the storage budget to their favourite charity.