How to show tracked objects in Video Security Client maps?

[#1]

1 What is Geolocation? Geolocation is the identification of the real world geographic location of an object. For Bosch IP cameras, geolocation is the position of the Bosch IP camera as well as the objects detected and tracked by Intelligent Video Analytics, Essential Video Analytics or Intelligent Tracking in this camera either in the global positioning system (GPS) or in a local map coordinate system.

1.1 Applications

• Display of Bosch IP cameras and objects detected and tracked by IVA on an overview map. Provides location awareness and reduces the need to look at single camera views.
• Input and first step for multi-camera object tracking.

1.2 Limitations

• Only possible on planar ground planes
• Geolocation of the camera must be set accurately
• Calibration of the camera must be set accurately
• Video analytics tracking mode must be standard tracking, 3D tracking, 3D people tracking or ship tracking
• Geolocation for Intelligent Tracking only available on CPP7.3
• Accuracy of the position decreases with distance
• Due to lens distortion, the geolocation of objects at the borders of the video will be less accurate
• Geolocation of partly occluded objects may be displaced: If the lower part of the objects is occluded, the foot position on the ground plane cannot be determined accurately.
• Support by viewers / video client / video management systems needed. Currently only supported by Bosch Video Security Client.

CPP: Common product platform for Bosch IP cameras and encoders. FW 6.40 supports CPP4, CPP6, CPP7 and CPP7.3.

Step-by-step guide

2 Configuration via device web page or Bosch Configuration Manager

The configuration consists of 3 major steps: Teach the camera where it is located on the map (Geolocation), teach the camera about the perspective in its field of view (calibration) and set the video analytics to a tracking mode supporting geolocation.

2.1 Geolocation

Teaching the camera where it is located on either a global world map or a local map.
This can be done at

• Device WebPage: Configuration -> Camera -> Installer Menu -> Positioning
• Configuration Manager: General -> Positioning

First, a coordinate system needs to be chosen. There are two options: WSG 84, the most used GPS coordinate system. It describes the position of the camera on the earth in the spherical latitude and longitude coordinates. In addition, the ground level above sea level can be set. Cartesian, allowing a simple description of local 2D / 3D maps. The Cartesian coordinate system describes the local space with rectangular X, Y and Z coordinates, where X and Y describe the ground plane and Z the height.

For both coordinate systems, the direction into which the camera looks is described by the azimuth angle. The azimuth is defined as zero in the east or along the X-Axis and a positive azimuth angle means that the camera is turned counter clockwise when viewed from above, resulting in north at 90° or along the y-Axis, west at 180° and south at 270°.

2.2 Camera calibration

Teaching the camera about field of view and perspective.
This can be done at

• Device WebPage: Configuration -> Camera -> Installer Menu -> Positioning
• Configuration Manager: General -> Positioning

Calibration is determined by the following values:

• Tilt angle: The angle between the horizontal and the camera. A tilt angle of 0° means that the camera is mounted parallel to the ground; a tilt angle of 90° means that the camera is mounted top-down in a birds eye view perspective. The flatter the tilt angle is set, the less accurate the geolocation of the tracked objects will be. Estimates are no longer possible when you have reached 0°.
• Roll angle: The angle between the roll axis and the horizontal plane. The setting can deviate from the horizontal by up to 45°.
• Elevation: The vertical distance from the camera to the ground plane of the captured image.
• Focal length: The focal length is determined by the lens. The shorter the focal length, the wider the field of view. The longer the focal length, the narrower the field of view and the higher the magnification. For Bosch IP cameras with an inbuilt lens this value is determined internally by the camera and need not be entered.

 

How to calibrate:

• DINION / FLEXIDOME: Calibration is done once for the camera, independent of any VCA Profiles. Use “AutoSet” to automatically set all values known by the camera itself, or to update these values after a change in camera positioning. For all cameras with inbuilt lenses, the focal length will be thus set, and for all CPP7 and CPP7.3 cameras, the tilt and roll angles are automatically determined as well. Thus in most cases only the camera height above ground needs to be entered manually. Additionally, a calibration tool measuring vertical and ground lines as well as ground angles is available to sketch the perspective and let the camera deduce fitting values for unknown tilt, roll, height and focal length values on demand. Similar measuring elements are also available to verify the calibration. Note that vertical lines can also be used to measure height of standing persons. Best practice is to distribute the measuring elements well in the image, e.g. one to the front, one to the rear and one to the side.

• FLEXIDOME IP panoramic: The FLEXIDOME IP panoramic automatically sets the focal length information as the lens is inbuilt. However, is has no internal sensors for measuring angles, and due to the fisheye perspective, sketching of lines will not work either. For most scenarios, the FLEXIDOME IP camera is either ceiling mounted or a wall mounted, which can be directly selected in the calibration, and thus only the height of the camera above ground needs to be entered manually.

• AUTODOME / MIC: Calibration can be done once for the camera by choosing “ceiling” mounting position and entering the height of the camera above ground. For all pan-tilt-zoom positions of the camera, it will then automatically deduce the correct calibration and field of view itself. In case of prepositions with video analytics, the calibration can be fully adjusted to compensate for field of views on different ground planes, non-horizontal ground planes or inaccuracies for large zoom factors via the sketch tool. These calibrations are only active if a preposition is activated, but not when the camera just sweeps across the preposition while on a non-preposition tour or manual steering.

How to verify calibration:

• Alternative method 1: Set the video analytics tracking mode temporarily to 3D people tracking and have someone walk through the scene. If the artificial 3D person shape fits well around the person in all areas of the scene, then the calibration is good. If the artificial 3D person shape is too small or too large, then calibration needs to be adjusted.
• Alternative method 2: Check whether the location of objects in the scene fits to where they appear on the map.

Note that calibration handling has changed with FW 6.40, and the instructions above on how to calibrate are not fully valid for older FW versions.

2.3 Video analytics configuration

Set Intelligent / Essential Video Analytics to 3D tracking. As soon as the camera is calibrated, both video analytics will output the geolocation of objects in both standard tracking and ship tracking. However, performance increases if 3D tracking is selected, as the tracking is then done directly on the ground plane and thus position and speed are more accurate. If you only expect upright persons in the scene, then 3D people tracking can also be used. The only tracking mode that cannot not be used at all is the museum mode, as no geolocation of tracked objects is generated there.

3 Configuration via RCP+

Remote Control Protocol plus (RCP+) is a remote control protocol including the ability to send read and write commands to a device, and to subscribe for messages. Using RCP+, the following tags and configurations can be read and set:

• conf_camera_position: Geolocation of the camera, either none, WSG 84 or Cartesian.
• conf_camera_orientation: Azimuth, tilt and roll angles
• conf_camera_surroundings: None or elevation above flat ground plane.

Note that Intelligent / Essential Video Analytics tracking modes cannot be set directly via RCP+, though a full video analytics configuration can be uploaded.
For further details, please see the RCP+ documentation or contact the Integration Partner Program team (http://ipp.boschsecurity.com).

4 Output: The geolocation of tracked objects

All the information about objects detected and tracked by Bosch video analytics is available in the so-called metadata, and transmitted and stored together with the video. It can be accessed via a separate Remote Control Protocol plus (RCP+) stream or via real time streaming protocol (RTSP). Tools for understanding the proprietary metadata stream are available upon request through the Integration Partner Program (http://ipp.boschsecurity.com).

In the metadata, for every object its geolocation is added in the tag object_current_global_position whenever geolocation and calibration of the camera are available. It contains either the longitude, latitude and height above sea level for the WSG 84 coordinates, or x,y,z for Cartesian coordinates, depending on which coordinate system was provided as geolocation for the camera itself.