Who rated this article

Object Classification FW 6.30

Object classification takes an object detected and tracked by either Essential Video Analytics or Intelligent Video Analytics, evaluates its geometric and speed properties, and decides whether the object is an upright person, a bike, a car or a truck.

This article provides an overview of object classification, describes how to configure it and what the limitations are.

 

Applications
- Traffic monitoring
- Gate monitoring
- … and wherever else an automatic distinction between upright person, bike, car and truck is needed or appreciated for easier setup of the video analytics.

Common product platform (CPP)
Bosch cameras can be clustered by their common product platform. As different platforms offer a different amount of processing power, this can make differences in the video content analysis (VCA) performance. For an overview of the different product platforms and the cameras belonging to them, see the tech note on VCA Capabilities per Device.

Limitations
- Available with Intelligent Video Analytics on Bosch cameras of CPP4, CPP6 &CPP7 with FW 6.10 or higher
- Available with Essential Video Analytics on Bosch cameras of CPP7 with FW 6.30 or higher
- Camera calibration needed
- All objects must move on a single, horizontal ground plane. Therefore object classification cannot be effectively used in uneven terrain (hills or valleys), or scenes that include staircases.
- Tracking mode “3D tracking” must be enabled
- Object groups cannot be separated and may be classified incorrectly
- Crawling / rolling persons will not be classified and cannot be distinguished from animals
- Persons and bikes seen from the front are easily confused. A bike from the front will only be classified as such if it is fast enough, otherwise it will be classified as a person.
- No differentiation between bicycle and motorbike
- Small objects with only a few pixel can be confused (for example, objects far away from the camera)
- Object class may change over time
- All objects start as an unknown object. They are only classified over the time if the object class can be determined sufficiently reliable

 

Step-by-step guide

 

Configuration
By default, this feature is only available if the camera has been calibrated and 3D tracking is active. If configuring an Intelligent Video Analytics or Essential Video Analytics task, the “Filter by Object Class” menu will be unavailable and you will receive the following message: “Classes are only available for calibrated cameras with 3D tracking active”.

3D tracking will also be unavailable in the “Tracking Mode” sub menu located under the VCA, Metadata Generation, Tracking menu in Configuration Manager.
Once you have properly calibrated your camera and saved the VCA settings, navigate to the VCA, Metadata Generation, Tracking menu in Configuration Manager:

• Select “3D tracking”
• Select Apply and save your settings
  Note: If you select “3D people tracking” in this menu, all objects will be classified as “Upright Person”
  
  During task configuration the “Filter by Object Class” menu will now be available:
• Select the Object Classification type that you want to trigger an alarm
• If no options are selected in this menu then all active objects in a scene will alarm
• Once an object class is selected, only that object class will alarm

 

From left to right: Object classification and 3D tracking are not available without camera calibration. Once the camera is calibrated, select 3D tracking, and then the object class filter can be used.

 

How does it work?
Camera calibration teaches the camera about the perspective in the scene. Due to perspective, in the rear of the video images the persons appear smaller, they cover less pixel in the image, though their real size is the same. Perspective is thus needed to determine the real size and speed of objects. However, this can only be done under the assumption that all objects move on a single, horizontal ground plane. Therefore object classification cannot be effectively used in uneven terrain (hills or valleys) or scenes that include staircases.

Object classification uses the object size, aspect ratio, speed and shape (outline) to differentiate between the four classes upright person, bike, car and truck for well-separated objects. This is similar to a manual configuration of size, speed and aspect ratio object filters but is done automatically and the performance goes beyond the single object filters.

Details for object class “Upright Person”

• An individual crawling or rolling on the ground, which is typical for perimeter intrusion scenarios, will not be classified as an “Upright Person”
  
• Small animals cannot be distinguished from a crawling/ rolling individual, and thus will not be classified
  
• Small children may not be classified correctly, or at all depending on calibration and FOV
• If objects are not visually separated they will be grouped together. A group of people may be classified as an “Upright Person”
• Groupings of larger objects such as trucks may not be classified
• “Object Classification” function replaced the “Head Detection” task in IVA 6.0. The basic logic behind this is that all “Upright Persons” have heads.

Details for object class “Bike”

• When classifying a “Bike” aspect ratio, size, and speed play major factors. The algorithm considers the following
• A bike from the side, based on calibration of the camera, is close to the size of a person. The exception is that it has wheels front and back, and in most cases faster than an “Upright Person”
• When viewing a bike from the front it resembles a person, so speed is the mitigating factor for classification.
• If in any FOV there is a question as to if an object is a bike or a person, the algorithm will default to an “Upright Person”. The logic behind the algorithm defaulting is that there is always “person” on the bike.

Details for object classes “Car” and “Truck”
When classifying cars and trucks, once again calibration plays a key factor. Vehicles are categorized based on size and speed.
- As shown below in a properly calibrated FOV, larger vehicles will be classified as trucks
- Cars pulling trailers may also be classified as trucks

Object Classification: Distance, Size, and FOV
When configuring video analytics for Object Classification it is important to understand the effect that distance has on objects in the FOV.

• Objects that approach from a distance may not be detected or classified until they are closer to the foreground of the camera’s FOV, and large enough in the image to reliably determine their properties.
• An approaching object’s classification may change as it draws closer and the video content analysis determines that it’s properties better fit a different object class

To eliminate false alarms caused by object classification changes due to distance, configure video analytics tasks in the foreground of the FOV.
Lastly, when an object is so large that it is still entering the video content analysis trigger area as it is leaving, the video content analysis will not have the time to isolate and classify the object. Size coupled with objects moving at a high speed through the FOV may cause objects to not be classified or trigger an alarm.

Solutions to the issues like shown below:

• Lens selection
• Changing the FOV