Think of the World Wide Web (WWW) as an application. Granted, it is a very, very large application, but it is an application nonetheless, which runs on the Internet. It is widely, and mostly agreed, that the Internet was created by the US Department of Defence's Advanced Research Project Agency to link funded research locations via a computer network. It was an experiment called APRANET.
Having expanded to a number of laboratories and academic institutions across the world, it eventually ended up at CERN, where Tim Berners-Lee invented the application now called WWW. His goal? To find a way to store and disseminate the massive volume of data the Hadron Collider was generating, in a functional and accessible manner.
No matter how or why, a massive volume of data became a game-changer.
Sunny skies
In May 2012, SA won the right to co-host the Square Kilometre Array, along with Australia. The clear skies and silence of the Karoo make it almost perfect for something like the SKA. This year alone, R6.47 million of the South African Science and Technology budget will be allocated to the MeerKAT project. [1]
That is big money as well as big business.
On 27March 2014, SA launched the first antenna that will make up the MeerKAT radio telescope. One of 64 antennae, this will be the precursor to the SKA outside of Carnavron in the Northern Cape.
With each antenna coming in at 19.5 metres tall and weighing about 42 tons, the full MeerKAT array will consist of 64 identical receptors. Basically, antennae with receivers, digitisers and other electronics installed. Once is it fully functional, the antennae will be connected to each other with 170 kilometres of underground fibre-optic cable, and will be working as a single astronomical tool which will be controlled and monitored from a remote location.
It is estimated that at full bore, the MeerKAT will generate enough data to fill about four-and-a-half-million standard (4.7 gigabyte) DVDs every day. [2]
Talk about big data
That got me thinking. The last time this generation experienced a data 'phenomenon' that came even close to what the people at the international SKA Organisation are talking about, the World Wide Web was the end result, and to a lesser degree, perhaps, even the Internet. In both cases, the need to disseminate lots of information quickly, effectively and accurately became some part of the precursor of the invention.
Today, most companies suffer from data overload in some form or another. More and more of it is being generated, along with more white noise. White noise is the uncontextualised, irrelevant data that surrounds the few nuggets of worthwhile data, the original intention of the creation, long surpassed by the sheer genius, inventiveness and ingenuity of a human mind.
It would seem to be human nature, along with invention, to err towards the destructive and the dark.
Standing on the edge of a revolution of 'always on' devices about to hit the market - I can already only partially imagine what lies ahead.
The capability to collect, store and effectively utilise the amount of data that will be collected by the SKA will push contemporary big data storage technologies like Hadoop past their extreme capabilities. To look at the birth of the universe, people will need to reinvent not only how they store and process, but also how they perceive such masses of really closely related data. Once that functionality becomes available for generic use, I cannot begin to imagine what that would mean for business. Perhaps more elegant, more ethical, more fluid data analysis and prediction? Perhaps the opposite. It would seem to be human nature, along with invention, to err towards the destructive and the dark.
Maybe this new incarnation of big data will finally remove the final vestiges of 'humanity' from data collation, analysis, insight.
Perhaps the purely logical will become the standard. To business, that would seem to be the ultimate win - a business plan that completely removes human emotion and operates solely on logic.
Whatever the future holds, one thing is guaranteed. When it comes to big data, the surface hasn't even been scratched yet.
[1] http://ewn.co.za/2014/07/23/Pandor-pushes-MeerKAT-project-in-parliament
[2] http://www.southafrica.info/about/science/meerkat-270314.htm#.U8_zxvmSy3M
Share