Tackling 'dark data' can be intimidating - even to the most accomplished of chief information officers.
So says Bryan Balfe, enterprise account manager at CommVault, who notes almost all organisations have volumes of dark data stored away in dusty vaults and off-site storage facilities, historically unaccounted for, unmanaged, and undervalued.
Market analyst firm IDC says by 2020, the data the world will create and copy annually will reach 44 zettabytes, or 44 trillion gigabytes.
Gartner defines dark data as the information assets organisations collect, process and store during regular business activities, but generally fail to use for other purposes, for example, analytics, business relationships and direct monetising.
Dark data can include legacy file shares, backup tapes, archives and former employee e-mail stores that are predominately unclassified and not visible or accessible. The challenge of dark data then becomes to balance its liability with the potential profit gains from using information more strategically, says Gartner.
A recent Gartner survey found 70% of organisations reported dark data will have a negative impact on storage and 53% said it would increase risk. The analyst firm believes how organisations handle the dark data challenge will become a key foundation for corporate competiveness, business intelligence and innovation, impacting every sector and every business function.
Balfe points out many organisations are discovering they lack both the policy and technology needed to efficiently manage data outside of the corporate data centre. Additionally, he notes, the growth of data - and big data especially - is causing enterprise to finally look at addressing the issue of dark data, if only to curb mounting storage costs.
"Organisations have very little awareness of the location, volume, composition, ownership, risk, and business value of their unstructured data," he says.
Based on the complexity associated with managing dark data, Gartner recommends organisations should review the scope of their unstructured data problems by using file analysis (FA) tools to understand where dark unstructured data resides and who has access to it.
"FA differs from traditional storage reporting tools because the technology doesn't just report on simple file attributes, but can also provide critical contextual information; with the ability to analyse, index, search, track, report on file mega data and even content," says Balfe.
He notes FA tools applied to dark data provide business value in a number of ways, one of which is by helping organisations reduce risk. By identifying which files reside where, and who has access to them, FA tools introduce an element of control, he adds.
Without FA tools, Balfe explains, it can be extremely labour-intensive to sift through the masses of irrelevant information contained within dark data.
"This 'run around' either consumes IT management's time and budget, leaving less bandwidth for immediate business needs, or requires a costly outsourced response."
Share