Be careful: The wrong data platform can lock you in

Johannesburg, 20 Oct 2021
Read time 4min 40sec
Marius Huysamen, Big Data Technical Team Lead, Obsidian.
Marius Huysamen, Big Data Technical Team Lead, Obsidian.

Information is crucial to your business, and thus selecting an appropriate data platform is very important. Yet companies often underestimate how crucial this decision is.

We can divide these concerns into two areas. The first relates to traditional enterprise data platforms that aren't cloud-native. At face value, these seem capable of meeting expectations around business information. 

"Big data platforms from the past decade or so address many of data's core needs," says Marius Huysamen, Obsidian's Big Data Technical Team Lead. "A data platform provides the ability to understand your data sources and to transfer data around from one source into a bigger pool. You can expose data to your community, catalogue that information and apply your governance controls. Data platforms also let you build machine learning models and do automated analytics."

But this is no longer enough. Big data platforms don't fit that well into hybrid cloud plans. Hybrid is fast becoming the primary model for business modernisation: Data collected by Forbes shows that hybrid cloud adoption has grown threefold in 2021. Data platforms that aren't cloud-native struggle to provide scale, containerisation and integration features.

The obvious solution is to use public cloud platforms' data tools. Yet, though public cloud data tools are powerful and have cloud-native features, they create some significant drawbacks.

"A lot of companies take a cloud-first approach, which in essence isn't wrong. But when they buy into a specific public cloud provider, they tend to use the data tools provided by that platform. Those tools aren't as well integrated as a prebuilt data platform. It also doesn't give you sufficient hybrid cloud security, or the ability to move your data and data processing where you need it. It's just putting your data in the cloud platform."

The risks of public cloud data platforms

Cloud-first can lead to data-second. The eagerness to use cloud services can under-appreciate how vital a data platform is for continued successful digitisation.

Data enablement is a headline priority. According to the report, Critical Success Factors to Achieve a Better Enterprise Data Strategy in Multi-Cloud Environment from Harvard Business Review Analytic Services and Cloudera, 69% of companies recognise a comprehensive data strategy as a requirement for meeting business objectives. Fifty-four percent plan to increase the amount of data they store in the cloud.

Hybrid cloud should create flexibility and choice, yet a hasty choice on data platforms and tools can have the opposite effect.

Huysamen explains: "A big problem with using proprietary data platform tools is that they work differently on different platforms. When you want to move or expand to another provider, perhaps to access different infrastructure, you may have to retrain a lot of skills and adapt processes. Your choice of a proprietary data platform can stop you from separating your data management and infrastructure choices."

Such data platforms are also not comprehensive in terms of services and integrations. If you continually introduce new products to close gaps in your data environment, "you might be chasing the wrong solution. An agnostic cloud-native data platform should cover all your data service needs. If you're on your third or fourth data product, you're adding a lot more than you should. Integrations also reveal gaps. Proprietary cloud data platforms will provide integration to a point, but you often end up having to create extra integrations on your side as you add services."

The choice of data platform thus impacts your ability to manage infrastructure, the effectiveness of available data skills and how much work needs to go into maintaining and expanding data services.

Keep data platforms agnostic

Prebuilt third-party agnostic data platforms solve these problems. They remain the same, regardless of which cloud provider you use, so skills remain stable. They are designed as a primary service and not a value-add for other cloud features. Thus security, governance and integration are prioritised. And since they won't lock you into a specific provider's environment, they separate infrastructure and data choices.

Beyond that, adds Huysamen, agnostic cloud-native data platforms "put a lot of emphasis on features such as data movement and access, analytics tools and dashboards. Many companies want those features and get excited when they see them as part of a larger public cloud offering. But let's be honest – a data platform is not the cloud provider's primary focus. You risk getting a diluted product. It's much better to invest in a data platform that can grow with you."

Ideally, choose such a data platform before starting your hybrid cloud journey. It will ensure you have the scale and flexibility to grow and maintain your hybrid plans. And if you've already started down the hybrid path, you can still add an agnostic data platform and its flexible cloud-DNA. Such platforms specialise in providing a seamless experience through open standards, backed by specialists whose business is to enable your company's information where and when you need it.

See also