Machine learning is playing a vital role in storage, making it more efficient than ever before.
Data is growing fast. Businesses want their data on premises, in the cloud, or in a hybrid between the two. They want instant access to it, but at the same time want to be able to control who has access to what. They also want to be able to back up their data to protect themselves in the event of a cyber attack.
The upshot of these increasing demands on the data centre is that it is having to evolve at a rapid rate, becoming smarter, more efficient, able to store more data yet at the same time being able to protect that data. The downside is that companies are spending a lot of money to store what could be superfluous data. Businesses are finding themselves having to compromise on the amount of data they store in able to afford storage of the data they really need.
"While there's a perception that businesses are generating more data, the reality is that a lot of the data they're storing is duplicated data," says Rohan de Beer, COO of iSanity. "Backups are an essential component of every single business, regardless of size or industry, but some businesses are making copies of copies and placing an unnecessary burden on their storage."
You can't prevent businesses from making copies of their data for non-production purposes - these copies are often used to perform pre-integration tests or other vital tasks - but you do need to try to ensure that they don't overburden the data centre.
Then what is the solution to this conundrum? How can businesses ensure that they store the copies of the data that they need to without paying to store multiple copies of the same data - or even unnecessary data? There are several approaches that are currently in use to try to minimise data duplication in the data centre, some more successful than others. De Beer is proposing a smart storage system that's aware of the duplication process and has the tools to reduce both time and performance, as well as offering storage efficiency. He adds: "Your storage growth would dramatically slow down."
What makes a storage solution smart? Well it's all about artificial intelligence, analytics and Internet of Things applications coming together to ensure that the right data is stored and accessed as and when needed.
The principle for machine learning in any storage system is to be able to understand the data that's being stored, as well as the application or workload requirements being placed on the stored the data. The system then needs to be able to adapt in real time to the changing requirements of the workloads as they change during the day. This ability to adapt to the application's requirements is crucial as modern day workloads are extremely time-sensitive and data delivery cannot be delayed.
Advance artificial intelligence algorithms are used to be able to do exactly this. The storage system needs to constantly monitor the read data that's required by the applications. The data then needs to be moved to high speed disks or even system memory for near zero latency response times. Very few systems has this real-time capability as most traditional storage system rely on standard algorithms that monitor the past 24 hours and then move data to high speed disk on historical requirements. Modern machine learning systems are able to adapt in immediately to changing requirements and deliver the data to the applications with close to zero latency.
De Beer says: "What we're seeing is development of data centre architecture built around what's happening in the future, not today. It's the next generation software-defined data centre."