Tackling big data storage

Ninety-percent of the world's data was created in the past two years.

"Can today's technology cope with this kind of growth in the next five or 10 years?" asked Tony Reid, EMEA pre-sales director at Hitachi Data Systems, during the Hitachi Information Forum in Johannesburg yesterday. "Probably not," he noted.

As such, tech companies, IT departments and businesses need to invest in offerings that will be able to cope as data stores continue to grow at exponential rates, he said.

According to Reid, one of the big challenges that has arisen as a result of this dramatic data growth is that much of it is unstructured, which means businesses struggle to gain value from the mounds and mounds of data they are storing.

What is interesting about the growth of data is that so much of this information is being generated during one of the biggest economic downturns the world has ever seen, said Hitachi Data Systems' CTO for EMEA, Bob Plumridge. "Despite this downturn, the demand for storage seems to know no end," he said, adding that the IDC and Gartner predict 100% growth for the storage market, year-on-year, for many years to come.

Legislation dictates that businesses need to keep certain types of data for specific periods of time, noted Plumridge, when discussing what is contributing to data growth. In addition, he outlined that the explosion of data is also a result of the fact that nobody deletes anything anymore. "It doesn't matter how trivial it is, it all gets stored in the same way," said Plumridge, adding that this causes problems when an organisation actually needs to analyse its data stores.

"Ultimately, what is the point of wasting money storing data that is of no use to the business?"

In order to guard against this data differentiation dilemma, Plumridge advised that companies strip down data when it is originally ingested by assessing the value of the information when it is first being stored. "You have to be looking at the data you are storing when you initially store it and decide what you should keep and what you should get rid of, not in five years' time."

Sitting down with ITWeb on the sidelines of the forum, Plumridge discussed where the storage market is headed. For him, the future is about consolidation. "Organisations will be looking at how they want their entire storage strategies constructed to meet the needs of their businesses," he said, noting that it is about using fewer devices, and in doing so, making storage more manageable.

"In the next few years, storage will be about trying to simplify and consolidate solutions to make it easier for customers to buy something that not only satisfies their needs today, but will also meet needs that may arise in the next two or three years."

According to Plumridge, developments in the storage market are being propelled by the explosion of the mobile device market, which is causing declines in the cost of solid state drives (SSD). "I wouldn't say the market for SSD is unlimited, but it certainly is huge. And as demand increases, price goes down and the reliability of this technology improves." SSD technology has also been so successful because people want to access their data anywhere, anytime, he said, noting that using SSD in big data analytics enables the real-time analysis and access people have come to expect today.

Ultimately, Plumridge believes that storage today, and in the next few years, needs to be fast, flexible and reliable. "The nuts and bolts behind the technology should no longer be the main focus," he concluded. "How this technology can be used to solve business problems, provide easy access to information and analyse data will be the focus of the future."