Object Classification FW 6.30
Object classification takes an object detected and tracked by either Essential Video Analytics or Intelligent Video Analytics, evaluates its geometric and speed properties, and decides whether the object is an upright person, a bike, a car or a truck.
This article provides an overview of object classification, describes how to configure it and what the limitations are.
Applications
- Traffic monitoring
- Gate monitoring
- … and wherever else an automatic distinction between upright person, bike, car and truck is needed or appreciated for easier setup of the video analytics.
Common product platform (CPP)
Bosch cameras can be clustered by their common product platform. As different platforms offer a different amount of processing power, this can make differences in the video content analysis (VCA) performance. For an overview of the different product platforms and the cameras belonging to them, see the tech note on VCA Capabilities per Device.
Limitations
- Available with Intelligent Video Analytics on Bosch cameras of CPP4, CPP6 &CPP7 with FW 6.10 or higher
- Available with Essential Video Analytics on Bosch cameras of CPP7 with FW 6.30 or higher
- Camera calibration needed
- All objects must move on a single, horizontal ground plane. Therefore object classification cannot be effectively used in uneven terrain (hills or valleys), or scenes that include staircases.
- Tracking mode “3D tracking” must be enabled
- Object groups cannot be separated and may be classified incorrectly
- Crawling / rolling persons will not be classified and cannot be distinguished from animals
- Persons and bikes seen from the front are easily confused. A bike from the front will only be classified as such if it is fast enough, otherwise it will be classified as a person.
- No differentiation between bicycle and motorbike
- Small objects with only a few pixel can be confused (for example, objects far away from the camera)
- Object class may change over time
- All objects start as an unknown object. They are only classified over the time if the object class can be determined sufficiently reliable
Configuration
By default, this feature is only available if the camera has been calibrated and 3D tracking is active. If configuring an Intelligent Video Analytics or Essential Video Analytics task, the “Filter by Object Class” menu will be unavailable and you will receive the following message: “Classes are only available for calibrated cameras with 3D tracking active”.
3D tracking will also be unavailable in the “Tracking Mode” sub menu located under the VCA, Metadata Generation, Tracking menu in Configuration Manager.
Once you have properly calibrated your camera and saved the VCA settings, navigate to the VCA, Metadata Generation, Tracking menu in Configuration Manager:
From left to right: Object classification and 3D tracking are not available without camera calibration. Once the camera is calibrated, select 3D tracking, and then the object class filter can be used.
How does it work?
Camera calibration teaches the camera about the perspective in the scene. Due to perspective, in the rear of the video images the persons appear smaller, they cover less pixel in the image, though their real size is the same. Perspective is thus needed to determine the real size and speed of objects. However, this can only be done under the assumption that all objects move on a single, horizontal ground plane. Therefore object classification cannot be effectively used in uneven terrain (hills or valleys) or scenes that include staircases.
Object classification uses the object size, aspect ratio, speed and shape (outline) to differentiate between the four classes upright person, bike, car and truck for well-separated objects. This is similar to a manual configuration of size, speed and aspect ratio object filters but is done automatically and the performance goes beyond the single object filters.
Details for object class “Upright Person”
Details for object class “Bike”
Details for object classes “Car” and “Truck”
When classifying cars and trucks, once again calibration plays a key factor. Vehicles are categorized based on size and speed.
- As shown below in a properly calibrated FOV, larger vehicles will be classified as trucks
- Cars pulling trailers may also be classified as trucks
Object Classification: Distance, Size, and FOV
When configuring video analytics for Object Classification it is important to understand the effect that distance has on objects in the FOV.
To eliminate false alarms caused by object classification changes due to distance, configure video analytics tasks in the foreground of the FOV.
Lastly, when an object is so large that it is still entering the video content analysis trigger area as it is leaving, the video content analysis will not have the time to isolate and classify the object. Size coupled with objects moving at a high speed through the FOV may cause objects to not be classified or trigger an alarm.
Solutions to the issues like shown below: