Big brother is getting smarter

The addition of analytics to traditional surveillance takes what was typically a reactive action and makes it more proactive, which presents a range of possibilities for businesses across all industries.
Read time 6min 30sec
Vanessa Tyne, senior KAM and team lead at Axis Communications.
Vanessa Tyne, senior KAM and team lead at Axis Communications.

In December 2018, a photograph taken by Jinkun Tech made headlines. The image was captured from the very top of Shanghai’s Oriental Pearl radio and TV tower. At 195 gigapixels, the photograph has such a high resolution that when zooming in, it was possible see the facial expressions of people walking in the street several kilometres away. Not only did the photograph entertain millions, this enormous image also showcased how digital is changing the surveillance game.

The storage challenge
Smart analytics can be used to solve the surveillance storage conundrum in two ways, says Laurence Smith, from Graphic Image Technologies. Firstly, it can be used to compress the footage, which basically reduces the amount of digital data in the video. Secondly, it can be used to enable conditional recording. Traditional recording is motion-based, says Smith. This technique has been used for many years, but it’s not efficient in outdoor environments because any movement in the background or changes in light might trigger the recording. With advanced analytics, this can be done intelligently, ignoring scenes that are triggered by background movements and preventing theses ‘incidents’ from being stored.

It’s predicted that cameras globally will produce nearly 2 500 petabytes of data per day in 2019, according to a 2016 report from HIS Markit

 Of those 2 500 petabytes, we’re only able to watch just 1% of the video, according to Vox’s senior product manager, Rudi Potgieter. With video analytics, we can unlock the remaining 99%, he adds.

Surveillance is typically a solution with a multitude of dumb devices creating massive amounts of useless video data that needs to be stored, but may never be looked at, says Jannie Erasmus, surveillance and analytics business lead at NEC XON. It was typically only useful when there was a need to review the data because something went wrong or a crime was committed, for example. With the addition of more advanced analytics using state-of-the-art algorithms and artificial intelligence, surveillance can now detect events automatically and predict behaviours in order to improve security, law enforcement and even streamline processes at large, private enterprises.

The addition of analytics to traditional surveillance takes what was a reactive action (looking at footage to get answers) and turns it into a more proactive platform, notes Glenn Noome, director at Smart Integration, a Ulwembu Business Services company. This is particularly true, and valuable, in instances where there are several cameras to monitor. Analytics also makes it possible to only record events that are critical for storing, leading to reduced storage space and lower bandwidth requirements. Only critical data, and not everything, is stored, he continues.

Video analytics in its ‘unevolved’ life form can be dangerous to use as your de facto way to compress video.

Rudi Potgieter, Vox

But businesses must be careful, cautions Potgieter. Video analytics in its ‘unevolved’ life form can be dangerous to use as your de facto way to compress video. If your rules are too strict, you’re potentially losing valuable information. On the other side of this discussion, if your analytics are poor, something like noise in the background could see you effectively recording the entire video clip. It’s all about using the right analytics, combined with a set of carefully created rules and computer vision, he notes.

This trend is powering the growth of Video Surveillance-as-a-Service (VSaaS), notes Erasmus. It’s also behind the increasing growth of surveillance analytics software, which Erasmus believes will soon be bigger than the hardware market, particularly as single video streams increasingly sustain the data needs of multiple applications.

On the edge

But one must remember that analytics require a lot of processing power, which puts a strain on the datacentre or the backend systems, adds Vanessa Tyne, senior KAM and team lead at Axis Communications. This is why edge-based analytics is such an attractive alternative. When analytics is performed on the device, we move away from an alert that is triggered by some sort of ‘tripwire’ and receive actual data that dictates how we should react. 

Could autonomous AI surveillance and IoT automate law enforcement?

Sure, all of this seems rather futuristic, but we certainly aren’t anywhere near some sort of Judge Dredd-like world where robots stop crime, says Vanessa Tyne, from Axis Communications. Some human intervention is still needed. There are solutions in play that use AI surveillance and IoT to aid law enforcement – tracking traffic infringements by reading licence plates or using footage to track serial offenders. But when it comes to more ‘serious’ crimes like murders and muggings, Tyne doubts that we want digital systems calling the shots.

NEC XON’s Jannie Erasmus agrees. “I don’t think you will ever be able to entirely automate law enforcement. Adding AI and analytics to surveillance will always be enormously useful, but AI will never replace human intuition. Robots can replace people in some other aspects of civil protection – such as fence patrols, detecting intruders – but these applications have limitations.”

This is not only a technology-related question, but also a philosophical one – do we want to allow machines to take the decision for us, or only to reach a certain conclusion and then enable a human to take the appropriate action, asks Laurence Smith, from Graphic Image Technologies. He believes that a machine can be used to detect the foe, track it and enable an action to eliminate this risk, but it should not perform that action autonomously and will require a human to step in and act in the most appropriate way.

Edge-powered video analytics helps to make our findings far more reliable, adds Noome. Furthermore, with edge-powered video analytics, it’s possible to search for incidents in less time and, depending on a business’ needs, you can set up proactive warnings relating to specific situations. “Edge equipment also enables integration with other systems directly from the hardware, without the need for processing on the software layer,” he says.

Edge analytics enable immediate outcomes, according to Forrester Research. Equipment enabled by the Internet of Things increasingly requires that the analysis of data — received from countless sensors — must return insights within milliseconds. This simply isn't possible if the data has to move many kilometres away in order to be analysed. With all of these connected devices, businesses have so many more touch points to engage with the customer and understand their needs and preferences, says Ian Jansen van Rensburg, lead technologist and senior systems engineering manager at VMware Sub-Saharan Africa. Deploying intelligent security surveillance solutions at the IoT edge can provide the information needed to make better calls around security, boost marketing efforts with more targeted initiatives, and can even offer the information needed to enable predictive maintenance of critical hardware.

All of this being said, video analytics doesn’t have to run entirely on the edge, cautions Laurence Smith, an executive at Graphic Image Technologies. Increasingly, we’re seeing distributed models where the computation is divided between the edge, local and remote servers, as well as the cloud.

Rise of the machines

We’re already seeing retail, health and industrial businesses using the intelligent edge, combined with surveillance cameras connected to advanced analytics, to improve shopping experiences, provide better healthcare services, improve cities and create safer, more controlled environments for all, says Erasmus.

The local landscape

In a country where security is of utmost importance, and is always top-of-mind, the local market has been slower in adoption, according to Smart Integration’s Glenn Noome. This is because local organisations are still building up trust in the technologies on offer. There is also a serious need for better infrastructure (fibre) at a more competitive price and education around how smart surveillance can provide more specific, useful notifications, he continues.

The South African market has not responded with the same enthusiasm as the rest of the world, partly because the analytics platforms and infrastructure used to enable this type of surveillance are expensive to run, adds Jannie Erasmus from NEC XON. 

In a country where unemployment is high, one must note that while this technology will reduce jobs on the one hand, it does open up an entirely new job market for labour resources to provide installation and offer related services and solutions, says Noome. To get South Africa up to speed, we must create demand and set up the training and education to meet that demand.

Smart and safe cities are using edge-based video analytics to plan future roads developments by monitoring citywide traffic flows and are optimising public transport by using these data-driven insights to create heat maps. These maps show how many people use which forms of transport at what times of day, he says. Similarly, retailers are also generating heat maps based on this technology to determine things like the gender and age categories of shoppers so that they can optimise store and mall layouts.

Facial recognition is being adopted on a massive scale globally and its use cases extend beyond security and access control, says Potgieter. From smart advertising and customer relationship management, to crowd and queue management, the possibilities are endless, and all of it is done on the edge. Facial recognition technology is also starting to play a crucial role in healthcare, Erasmus continues. Patients can be biometrically authenticated to help doctors and other healthcare personnel automatically prepare the correct patient files and ensure that they receive the right treatment.

A very interesting technology in the security space is the addition of gunshot detection solutions to smart cameras, says Tyne. With these acoustic sensor systems, the camera can detect noises at a specific decibel and then determine if these noises are just raised voices or if it’s actually a gunshot. In instances where it does pick up people shouting, the system can be taught to proactively send an alert because there is a high likelihood that an altercation will occur. In these instances, security can be dispatched before anything even happens, she says.

Only critical data, and not everything, is stored.

Glenn Noome, Smart Integration

Smith believes that combining the cloud with intelligent surveillance devices puts control in the hands of commercial, government and consumer markets and empowers them to reduce potential risks. The only ‘hurdle’ is us, he says. “We seem to be reaching social and privacy barriers much faster than technology barriers, which could mean that, in the end, the actual limitations will not be the technology itself.”

See also