Shadow IT illuminates the gap between business needs and IT delivery
By Julian Field, Knowledge Integration Dynamics (KID)
As TDWI put it in its story: Meet the new BI-Enabled IT: "Industry luminary Jill Dyche champions a new, more responsive IT - enabled by business analytics," writes Julian Field of Knowledge Integration Dynamics.
Dyche was addressing attendees of TDWI's World Conference in Las Vegas and offered alternative roles for IT: as broker, order taker and custodian - vastly different from its traditional role. Lines of business, she said, specifically the chief marketing officer, would have more budget for IT than the CIO.
This budget creates a shadow IT, the IT behind the IT, and is a result of frustration. Marketers want an edge and they're prepared to do almost anything to get it. They set themselves up in opposition to IT when they develop their own IT systems, particularly where data is concerned, because IT managers want the systems and data in one place, where they can effectively manage it. Shadow IT develops distributed systems, multiple versions of the truth and by so doing injects an element of chaos into the system. For businesses like mine that's good news. We're here to fix the chaos. But I'm not sure this approach will necessarily help companies achieve their business objectives effectively.
Many vendors are using the frustration in lines of business like marketing to bypass the IT department to sell their kit. The result is that businesses buy technologies not "certified" by their IT departments, which presents challenges for IT and data architects.
Compounding the issue is that, in their haste to acquire the ability to manipulate their data, departments other than IT are not satisfied to wait while lengthy proofs of concept and other benchmarks are conducted. They need information and they need it now. They care little if a system is written in Fortran or C#. However, IT is responsible for systems that keep the business ticking over, which is why they are concerned with details. They need to ensure that everything works together harmoniously.
However, in their attempts to make it so, many CIOs become proponents of one vendor's technologies, believing that's the best way to ensure tight and seamless integration. They take it to near biblical proportions, believing that Microsoft, Oracle or Teradata are going to give them everything they need to get the job done. But the fact remains: you cannot use one brush to paint a Mona Lisa. And you cannot use one vendor's software to build an empire.
The rigid approach that IT likes so much coupled to an environment reliant on a single vendor more often than not exacerbates frustrations from lines of business. It becomes onerous to perform even the simplest analysis because there are rigid processes that consume all the competitive edges like a corporate Godzilla feasting on agility, flexibility, adaptability and speed. And that's when lines of business, such as marketing, bring in a product like QlikView or its equivalent to "save" the day.
That's why Dyche is correct in her statement that IT needs to change the way it operates. In the context of speed IT has always been playing catch up. It is cumbersome, slow, monolithic. It operates at the speed of dark. Additionally, the CIO is often told after the business strategy has been defined by the business CXOs that they must make it work, "or we'll find someone who can". In the mainframe's heyday a popular maxim went: "You can't get fired for buying IBM", and in many ways CIOs today are trying to find their IBM.
IT is typically scrambling to keep up with business then as it builds the systems to propel the strategy forward. But it takes time to build systems that co-exist with all the rest. IT wants the data in one place where they can manage it properly.
However, new technologies make it possible to retain the properly architected, tested and constructed approach while giving business people a modicum of control over their systems, the data at their disposal and the agility they crave. You can use technologies such as in-memory, high performance databases. Larger businesses are well positioned to take advantage of that because they have the skills in-house to effect the changes and still retain control of the overall data and systems. Their marketers are often also less agile, they are accustomed to the wheel turning a little slower, and are therefore a little less demanding of agility. Entrepreneurial types, however, are often highly agile. They are the first wave troopers who storm in and seize opportunities as they present themselves: their key strategy is based on surprise and speed. They need in-memory systems and high speed databases that give them the data they need to take informative decisions immediately and act quickly.
The type of business also impacts the preferable approach. Traders need data that will feed their decisions, decisions that could make or break companies. Bad trader data can bring a company down fast. Bad marketing data won't have the same impact. In many cases it can be expedient to shift data from operational systems down to the marketing department and let the marketers fiddle with it. Pragmatic and limited data volumes can be explored by marketers without heavily impacting operations or corrupting data integrity.
It is also important to remember that this is not a new challenge; it is as old as the shadow IT in finance departments story that is around a decade old now. Governance reigned in the extent of shadow IT in governance and protection and security of personal information (POPI) will also limit the powers of marketers to run their own systems - unless they are willing to formalise their approach as traditional IT has been doing for some time now. In which case it begs the question: why do it at all?