Data deluge

Data storage requirements and the associated costs are not a new pain point for IT organisations. What are new pain points are governance, risk and compliance imperatives. Data needs to be retrieved within a reasonable timeframe to meet legal discovery requirements.

In terms of South Africa's new Protection of Personal Information Bill, companies not protecting personal data to the degree the Bill requires will feel a lot of pain too. And then there are green initiatives, or more pragmatically put, the urgent need to reduce power consumption countrywide.

What all this means is that IT and business need to get their pretty little heads around what data the company has, where it is, what needs to be kept for legal, governance, compliance or regulatory purposes, what must be retained for operational purposes, and how to do all this without having a nervous breakdown.

Part of the storage problem is that throwing more disk at it has been regarded as a viable option for far too long. Says HP South Africa business development manager for storage Yesh Surjoodeen: "Pain points are fairly common among the customers we chat to - how to cope with the growth in data, how well they can forecast growth in terms of addressing their needs, are they geared for that change, has storage been provisioned correctly, how can they get away from spending on storage year after year with costs going through the roof?

"It boils down to cost," he says. "Hardware, management, administration. It's really about [understanding] your environment and how you can better use what you have. This perhaps is where the virtualisation argument is the strongest because it addresses concerns on different levels."

Sun Microsystems SA storage practice lead Manfred Gramlich says it's all about optimisation. "I would tell any organisation out there that they need to optimise their storage infrastructure. Most organisations, especially the larger ones, are still experiencing a lot of growth in storage. We see it every day, especially in the Web 2.0 space. Storage continues to grow, prices are coming down to a level where it's difficult to reduce costs further. Companies continue to buy - they have to - because of the amount of information they need to retain, store and manage on a daily basis. If you look at what Gartner says; utilisation is between 30% and 40%, which means between 60% and 70% of storage infrastructure is being wasted. If organisations were to properly manage this, they could regain wasted space and defer investments."

Key to managing storage requirements is getting a handle on your data. Tools and techniques like data classification and data de-duplication can help. In the first instance, data is classified using a selected methodology. For example, the degree of importance and the duration of importance. Once all data has been flagged as, for example, critically important for the next 10 years/of critical importance for the next two weeks and thereafter non-critical, companies can put systems in place to ensure critical information is stored in high-availability storage systems for the duration of that criticality, before being put out to pasture for the long-term in less available, and less expensive, storage.

Data de-duplication is a tool that ensures only new data is backed-up onto storage infrastructures. This assists with several things. Firstly, bandwidth is expensive and the more data you have to transport to offsite storage infrastructures, the more you have to spend on it. Secondly, backups are time-consuming and the sheer volumes being backed up today make daily backups something of a nightmare. Thirdly, it reduces duplication of data on disk space, which may cost an arm and a leg.

Says Symantec storage specialist Sheldon Hand: "The cost of storage is starting to spiral out of control. Demand is still growing at 60% per annum while the average utilisation rate is approximately 30%. This is the real challenge for customers. They have storage out there, they keep buying a lot, not utilising it correctly, and it's starting to consume more of the IT budget.

"The problem with storage costs is that a lot of it is not apparent upfront. If you think of how customers typically apportion storage costs, they combine different technologies under storage - backup and recovery, disaster recovery, business continuity, the bandwidth needed between sites. It's not just the application or the disk; it's the supporting infrastructure that adds to the cost. Something of late coming to the fray is obviously data centre space. Storage consumes space and power. Storage devices generate a tremendous amount of heat and the cost of cooling is a huge issue. This [cost] is why customers are starting to take it seriously."

The solution? Hand believes the first step is to take an inventory of the physical storage one has and gain a good understanding of how that storage is utilised. "Figure out how to take under-utilised storage and use it where it needs to be used. Once that's done, you can plan storage purchases or acquisitions a lot better," he says.

Then you need to look at data classification, says Gramlich. "It's all about mapping data and storage services to business needs. What value does data have to the business? If it has no value, don't keep it."

Then you get to information life cycle management (ILM). Says Hand: "You need to develop an ILM strategy. The value of data changes over its life at an organisation, and you need to move it through your storage architecture as that value changes."

Gramlich says supporting that is your tiered storage architecture. "A tiered infrastructure lets you place classified data on the right level of storage. This has an immediate benefit to business. Horison Information Strategies has found that 15% of data in large organisations is deemed critical, 25% is task critical and 65% of data in most organisations is old, stale data that has no real value.

"Once you have your infrastructure and classified data, you need to consider process maturity. Look at the maturity of the processes around storage management. Are they consistent, repeatable and so on? There are a lot of these best practices available as well as consulting services. All of the vendors have them - Sun, EMC, IBM, HP.

It boils down to cost.

Yesh Surjoodeen, business development manager for storage, HP South Africa

"The interesting thing," he notes, "is that we find customers very reluctant to start doing something about the problem. A lot of the tools are available for storage resource management, which you can deploy to get a look at what kind of data landscape you have. Where is your data, what system is it on, how old is it, when was it last accessed? All this information is available and companies have to start this journey."

Online lessons

It's all about optimisation.

Manfred Gramlich, storage practice lead, Sun Microsystems SA

So the question, as HP's Surjoodeen points out, is how do online providers like Google Mail and Snapfish provide unlimited data, for free, to consumers? And how do they manage it? If they can do it and offer it for free, where can corporates take a leaf out and make their lives easier?

"They need very scalable, available and cheap environments," he notes. "Companies need to start looking at affordable storage with extreme scalability. This is the next chapter for storage." Enough said, really.