Managing exponential growth of data

Data storage is available at a low cost, and extending storage space is an easy solution to deal with data growth. However, how often do you take this action only to find out three months later that the same problem has returned and more space is required? Increasing capacity is part of the solution and needs consideration as part of a long-term data storage strategy and retention policies.

The notion that data storage is cheap is very subjective and depends on many different factors beyond the price of disks. The acronym RAID, which initially meant ‘Redundant Array of Inexpensive Disks’, is somewhat misleading in that it conveys the message that storage is cheap without considering many other factors including:

  • The costs of other hardware requirements
  • The costs of physical space in data centres
  • Employment costs
  • Maintenance
  • Ongoing support

The overall cost of data storage is more important relative to the value of the stored data. RAID more commonly means ‘Redundant Array of Independent Disks’, which is more appropriate.

Parkinson’s Law states that one’s work will expand to fill the time available to complete it. The same principle applies to space: a requirement for storage will increase until it reaches maximum capacity. Buying a second filing cabinet has the long-term effect of doubling the number of documents stored. Notice at home that the same applies to cupboards, shelves and coat hooks, and how often a spare bedroom fills up over time. A corollary of Parkinson’s Law relating to the growth of data is that stored electronic data will expand to fill whatever storage space is available for systems to use.

The key areas to be fully explored before investing in new hardware are:

  • Housekeeping – cleanup of historical data storage where appropriate to reduce strain on systems
  • Ongoing policy – decisions on what data is stored and for how long
  • Capacity planning – projecting future storage requirements and proactively planning any storage expansion

Reducing the need for storage has an added benefit to the environment of reducing energy consumption.

We are committed to saving energy and resources. We offer our clients a challenge to use housekeeping, policy implementation and capacity planning to reduce storage requirements and to contribute a portion of any financial savings from the storage budget to their favourite charity.

Streamlined IT Infrastruture

On numerous occasions with IT projects, the management decided to buy software which didn’t fit into the current technological environment. The consequences of this were costly.

Examples include:

  • The software required Linux, but the IT environment consisted of Microsoft servers. The business needed new servers as there was no opportunity to share with existing systems.
  • The solution required MySQL or Oracle while all the existing database servers were Microsoft SQL Server

Having a defined platform when it comes to deploying new business systems keeps costs to a minimum by taking advantage of existing hardware, software, services and business processes. The alternatives are more costly:

  • Skillsets – existing staff with Microsoft experience would need cross-training in new technology, or the business would need to hire new staff with the necessary skills to operate the system. Introducing a new business system on the same platform would take advantage of the existing skillsets. If additional team members are required, the skills will apply to the new system and all current systems deployed on the estate.
  • In-house software – if the environment has many different systems built in-house to support the business, does the new bespoke system use the same technology as existing systems? Introducing new programming languages may make the system unsupportable by existing teams and result in the need to hire new staff.
  • Security – the more complex and diverse the environment becomes, the higher the potential for breaches, and the greater the requirement for patch management and security testing

The vendor system under investigation may be one of the best in the marketplace. However, if it introduces too much diversity into your organisation, it will quickly become an unmanageable cost centre.

  • Accurately understand the requirements to make the system work
  • Understand your own IT environment
  • Evaluate how the new system will work in your environment and what the realistic cost expectation is factoring in technological diversity.