Artificial intelligence (AI) adoption is soaring, but its success depends entirely on having access to the right data quickly − in an easy, secure and governed way.
However, in many enterprises, data resides in siloes across departments, branches and even countries. This means that data scientists working on a machine learning or AI model must spend a significant amount of time looking for the necessary data and pulling it all together.
In fact, they might spend 60% to 80% of their time just looking for data, extracting, transforming and loading it into a data warehouse, and only 20% to 40% of their time actually working on the AI models.
Additionally, AI initiatives face challenges around security and governance: organisations have valid concerns that AI might surface or leak sensitive data. For each challenge, there is a technology solution, but implementing multiple solutions only serves to complicate already complex environments.
The solution is to add a data virtualisation layer that connects to all the enterprise’s data where it resides, while also allowing businesses to secure the data, control access and implement rules.
With a virtualised data layer or data marketplace, AI deployments are simpler and faster.
Data virtualisation providing a universal semantic layer and built-in governance is an emerging model that is ideal for organisations with disparate data stores, such as banks, which see clear benefits from having a single layer that aggregates and exposes data at its source.
Data marketplaces for AI models
With data virtualisation, organisations gain comprehensive data marketplaces that expose a single view of data across multiple access channels, without having to move or copy it. This has the added advantage of minimising storage costs and ensuring information stays up to date.
Users can directly access data across the enterprise and immediately start using it for AI models. This allows valuable resources like data scientists to focus on what they are skilled at, instead of spending time on tedious manual tasks.
It is important that the virtualised data layer is an open platform that allows data scientists to use their preferred tools, and that it supports Model Context Protocol (MCP). MCP, an emerging standard that helps AI models and AI agents securely connect to enterprise systems and data platforms, helps AI applications understand what data is available, how to access it, the meaning and context of the data, and the governance and security controls around the data.
AI data assistants for easy access
Adding further value to the virtualised data layer, an AI bot can be integrated so that business users don’t need data science or business intelligence skills to get insights from enterprise data. With a natural language model in place, the user might ask: ‘Show me the sales figures in Gauteng for last week’, and the AI would retrieve the data and present the report.
Conversational AI can deliver a ChatGPT-like interaction with enterprise data, answering questions, explaining the steps being taken, surfacing assumptions, and even requesting clarification when questions are ambiguous. It might also offer to enrich the data or format it into a graph.
In future, the AI might also be tasked with updating the report weekly and sending it directly to the user or team members.
Simplifying governance
Because AI agents might be deployed to both read data and write actions, security and governance are critically important. A virtualised layer can enforce uniform, enterprise-wide security policies at the point of access, so AI agents operate within the parameters of the enterprise’s governance guidelines.
With an advanced virtualised data layer or data marketplace, the platform will support user and role-based authentication and authorisation mechanisms, and data-specific permissions, such as access to only specific rows or columns in a virtual view.
Every user that interacts with the virtualised data marketplace logs on with their own username and password to shop for data. They can request access or instruct their bots to do so, but access management rules will determine whether access is granted.
With a virtualised data layer or data marketplace, AI deployments are simpler and faster: it reduces time and effort for the data scientist, allows business users to interrogate the data in natural language, and enhances security to allow AI to interact with the data and take actions.
Share
