Intel explores video-mining

By Leon Engelbrecht for Intel
Johannesburg, 24 Apr 2007

Intel is investing in technology that it says will revolutionise the way we watch television sport. It plans to have a product on the market by 2010 - just as SA gets ready to host the 2010 Fifa Soccer World Cup.

The technology, under development at Intel's research laboratory in Beijing, China, is part of the company's investment in tera-scale research.

Intel recently launched duo-core computers and is about to release quad-core processors that will be exponentially more powerful than the desktop that produced this report. In fact, Intel researchers have an 80-core processor operational at present.

Intel Beijing applications research laboratory manager Yimin Zhang is leading efforts in the personal video applications field. His team is using the world's biggest sport, soccer, as the touchstone.

"Tera-scale [computing] gives us power we never had before and we can do something wonderful with it," he enthuses.

The spread of digital media means that many consumers have hundreds of hours of digital video of children's parties and soccer games on tape. However, there is no effective way to mine it for clips featuring a particular grandmother, a specific player or a particular game play - be it a foul or a goal; and no one has the time to sift through the material.

Human and body detection

The technology also has enterprise and government applications, allowing both to speedily sift through hours of surveillance footage, for instance. For broadcasters, it will come close to automating the packaging of sport and news events.

In its current research incarnation, the soccer highlights detection software automatically finds highlights in the event being recorded - but also guards against needlessly recording the broadcaster's replay of highlights - thereby eliminating duplication. In 21 hours of recording, the algorithms used showed a precision of 99.84% and a recall ability of 99.37%.

The technology also allows human and body detection and tracking, meaning one can video-mine individual left-hand punches thrown by a boxer in the ring. The algorithms used further allow for "cast indexing", the technology that is needed to video-mine clips of a grumpy uncle or a happy granny, in that it provides for facial expression recognition in addition to the more obvious abilities to recognise and track faces in a crowd. By extrapolation, the technology can also detect and track clips featuring speeding cars, ships at sea or aircraft in flight.

This is where Zhang's software meets Intel's hardware. "Even if we perfect our video-mining algorithms, we still cannot do this in real-time - it is too slow and will take hundreds of times more time to process than the raw material is long," he says.

"Tera-scale technology allows us to parallelise the processing and dramatically reduce the time needed. Tera-scale technology allows us to easily find useful information."