melodic noetic   Show EOL distros: 

Overview

This package wraps an automated pattern-based tracker based on ViSP library. The tracker estimates the object position with respect to the camera. It requires the tracked object 3d model as a VRML file and a configuration file.

The automated pattern-based tracker allows the user to detect a pattern in an image without user intervention using one of the following detectors:

• QR-code detection
• flashcode detection

Once the pattern is detected it will be tracked using an hybrid model-based tracker coming with ViSP that will use moving-edges and keypoint features. Finally, the automatic tracker is also able to detect loss of tracking and recover from it.

The package is composed of one node tracker. The node tries to track the object as fast as possible. The viewer coming with visp_tracker package can be used to monitor the tracking result.

The next video shows how to track a specific pattern textured with a QRcode. ViSP model-based tracker detects when it fails and recover the object position thanks to QRcode detection.

Calibration Requirements

Currently the visp_auto_tracker package requires calibration information from a camera_info topic. To this end visp_camera_calibration package can be used.

Features

The package purpose is to provide the 3D pose of an object in a sequence of images. The object has to be textured with a pattern on one face. The pattern has to be included into a white box, itself included in a black box.

This is an example of a valid QR-code pattern that can be downloaded here.

This is an example of a valid flash-code pattern that can be downloaded here.

Examples

You can run visp_auto_tracker on a pre-recorded bag file that comes with the package, or on a live video from a camera.

Pre-recorded example

To run visp_auto_tracker on a pre-recorded image sequence, just run:

roslaunch launch/tutorial.launch

The pattern used in this example can be downloaded here.

Live video examples

You have a ready-to-use roslaunch file in launch/tracklive_firewire.launch. This works with a firewire (1394) camera. If you have an usb camera (like a webcam) you can use launch/tracklive_usb.launch launch file.

You can launch with the following command line:

roslaunch launch/tracklive_firewire.launch

Config file

visp_auto_tracker centralises most of its parameters inside a configuration file following the boost::program_options default format.

The basic configuration file would look like this:

#set the detector type: "zbar" to detect QR code, "dmtx" to detect flashcode
detector-type= zbar
#enable recovery mode when the tracker fails
ad-hoc-recovery= 1

#point 1
flashcode-coordinates= -0.024
flashcode-coordinates= -0.024
flashcode-coordinates= 0.000
#point 2
flashcode-coordinates= 0.024
flashcode-coordinates= -0.024
flashcode-coordinates= 0.000
#point 3
flashcode-coordinates= 0.024
flashcode-coordinates= 0.024
flashcode-coordinates= 0.000
#point 4
flashcode-coordinates= -0.024
flashcode-coordinates= 0.024
flashcode-coordinates= 0.000

#point 1
inner-coordinates= -0.038
inner-coordinates= -0.038
inner-coordinates= 0.000
#point 2
inner-coordinates= 0.038
inner-coordinates= -0.038
inner-coordinates= 0.000
#point 3
inner-coordinates= 0.038
inner-coordinates= 0.038
inner-coordinates= 0.000
#point 4
inner-coordinates= -0.038
inner-coordinates= 0.038
inner-coordinates= 0.000

#point 1
outer-coordinates= -0.0765
outer-coordinates= -0.0765
outer-coordinates= 0.000
#point 2
outer-coordinates= 0.0765
outer-coordinates= -0.0765
outer-coordinates= 0.000
#point 3
outer-coordinates= 0.0765
outer-coordinates= 0.0765
outer-coordinates= 0.000
#point 4
outer-coordinates= -0.0765
outer-coordinates= 0.0765
outer-coordinates= 0.000

Common parameters

detector-type

The following detectors are supported

• detector-type= zbar: uses libzbar to detect QRcodes

• detector-type= dmtx: uses libdmtx to detect flashcodes

flashcode-coordinates

3D-coordinates in meters of the box delimiting the pattern (QRcode or flashcode).

inner-coordinates

3D-coordinates in meters of the white box containing the pattern.

outer-coordinates

3D-coordinates in meters of the black box containing the pattern.

ad-hoc-recovery

When set (tracker-type= 1) this parameter activates the tracking lost detection and recovery using flashcode-coordinates, inner-coordinates and outer-coordinates point coordinates.

Tracker states

The tracker is a state machine whose states vary during the tracking process. The process includes tracking, loss and recovery. These are the states used:

• Waiting For Input (id: 0) : Not detecting any pattern, just recieving images
• Detect Flashcode (id: 1) : Pattern detected.

• Detect Model (id: 2) : Model successfully initialized (from wrl & xml files).

• Track Model (id: 3) : Tracking model.
• Re Detect Flashcode (id: 4) : Detecting pattern in a small region around where the pattern was last seen.
• Detect Flash code (id: 5) : Detecting pattern in a the whole frame.

Viewer

When you track a model, you probably want a visual feedback. You can get one by connecting rviz to the outputed /object_position topic. visp_auto_tracker does not have a dedicated viewer. It can use the viewer provided with visp_tracker package, specifically visp_tracker/visp_tracker_viewer node.

Without connecting another node, you can also open a debug graphical output directly from the visp_auto_tracker node by setting the debug_display parameter.

The following figure shows the debug output (left) next to the external visp_tracker/viewer (right) in the case of the hybrid model-based tracker with QR-code initialisation:

Nodes

Report a bug

Use GitHub to report a bug or submit an enhancement.