What it is object detection and recognition

When it comes to video surveillance analytics, object detection and identification is a crucial component. This capability empowers operators to locate a specific object, such as a person, vehicle, or backpack, and then track it from frame to frame. The result? The ability to quickly identify the object through hours of footage, thereby giving security and police investigators critical forensic evidence; business owners much-needed insights; and more impactful applications for a diverse set of users.

To break it down, object detection is simply the ability to detect objects in video – however, this process is far from simple when executed by an advanced technology. The first step, “object extraction,” detects and tracks an object at any given time. Then, another technology called “background/foreground separation” is added, which identifies a static background of the scene.

The ability to extract objects from video and differentiate them from the backdrop against which they are detected, enables more sophisticated video analytic activity, such as forensic search, real-time alerting, or VIDEO SYNOPSIS^® technology – which displays all extracted video objects simultaneously, so that the full activity of the hours-long scene can be viewed all at once, rather than in linear time.

How object detection and recognition works

So, how is object detection – and subsequent object recognition – made possible? Through artificial intelligence. We encounter artificial intelligence and deep learning throughout our day, through tools such as voice recognition or automatic translation services; deep learning is an AI-backed discipline by which computers learn through exposure to data and execute task such as identifying objects or recognizing an object throughout a video. In order for deep learning to be successful, you need vast amounts of data, which must be processed and annotated. This data is then used to train the network until it is able to repeat what it has been trained to do.

In terms of object detection and recognition, the training data (which, in the case of video analytics, is video itself), will be broken down to individual frames to extract every item or object and train the system’s deep learning neural networks to associate specific objects with their classification and attributes. For example, if investigators are looking for a missing child wearing pink pants, they can leverage video analytics to search video evidence for objects that the system can recognize are people wearing pants that are pink. The user would filter his or her search based on these attributes, and the video analytics software – having been exposed to large data sets of people, pink objects, and pants – would be able to answer a query against actual footage: ‘identify all instances of people wearing pink pants.’

In order to detect matches, the deep learning algorithm will need to know what pink is and will need to have been fed thousands of data examples of this color. This is repeated ad infinitum for as many objects are required.

Object tracking

It is not enough that an object be detected, it must also be tracked. For any video search – whether as part of a law enforcement investigation or a way to gain deeper business intelligence – it is important to be able to accurately follow the object from the time it enters the scene until the time is leaves the scene, even if there is an obstruction or crowd.

Once the object is detected and tracked, descriptive information (or metadata) can be applied to it to create a vast structured database of classified objects. This step is what allows the technology to quickly examine hours of footage to identify the specific object – or, in our example, the pink pants.

When is it used?

Object detection and recognition can be used before, during and after an event occurs. Reviewing capabilities allow for the investigation of an event that has already occurred. But real-time alerting can also be enabled, which allows for the response to an event as it is happening in order to identify objects of interest. Object detection and recognition also has applications beyond law enforcement. For example, a real-time alert can be set for a specific camera at the entrance of a construction site to ensure that everyone who passes is wearing a hard hat. If someone were not wearing the designated object, in this case an orange hard hat, an alert would be sent to operators who could then respond in real-time.

Object detection and recognition are important underlying technological capabilities that drive sophisticated video analytics for diverse business and security use cases. Want to learn how video analysis can streamline your day-to-day decision making with actionable intelligence? Take the BriefCam Business Case Assessment.