Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Tracking your own model

If you are not familiar with ViSP model-based tracker, follow first the ViSP model-based tutorial.

To use the tracker with your own object, you need to create an object database.

The object database is located by default in the models directory were tree objects are provided: laas-box, floppy-box and shelf.

You can either add your own object in a new directory located in the models directory or choose to put your directory else where. In this case, you have to fill the ~model_path parameter accordingly when starting the client (or initializing the object using the 'init_tracker' service).

Required files

Consider that you want to track a new object called my-box.

Create first a folder called my-box, for example in the models folder besides existing laas-box. In this new my-box folder you have to introduce the following files:

• The CAD model of the object to consider, either in .cao or in .wrl format

  • my-box.cao: object CAD model in .cao format. This home-made format is described in here.

  • my-box.wrl: object CAD model in VRML format. This format is described here.

• my-box.init: initialization file used to estimate the initial pose by clicking on 2D points in the image. The content of this file is described here.

• tracker.yaml: best tracking parameters for this object (optional). The signification of the tracker settings is given here.

• my-box.ppm: this file is optional and corresponds to an image that shows where the user has to click to initialize the tracker.

• Additionally, my-box.0.pos contains the last initial pose and is created automatically by the client.

Once created, modify ~model_name parameter to my-box.

Modeling the object

Any 3D modeling software can be used as long as it can export its models to the VRML format. But in that case, be careful since all the triangles will be exported. In that case the model will contain all the triangle edges even those that are not real object contours. An example of such a model that doesn't work with moving-edges features is given here. That is why we suggest to use only a 3D modeling software to determine the 3D coordinates of some points and create the CAD model manually.

Keep in mind that this tracker tracks lines. Therefore, when loading an object, the lines are extracted from the 3D model and all the other information are discarded.

Warning: the surface normals are used to determine if a face is visible or not. They are taken into account and if they are not correctly oriented (normal vector toward outside the object), the tracker will fail.

It is also important to notice that the tracking frequency decreases as the number of tracked lines increases. Therefore, do not give to the tracker a full model of your object, trim all the lines which cannot be tracked precisely (bad gradient, hardly visible...).

Popular modeling software includes:

• Blender - open source, powerful modeling software

• AC3d - proprietary/non-free simple modeling software

• Modeling software list on Wikipedia

Writing the initialization file

The initialization file is used by the client to compute the object initial pose.

To compute the initial pose, the client does as follow:

• grab an image
• let the user click on a predetermined object location such as a corner
• repeat for all points in the initialization file
• compute the best pose such as the 3d pose projection matches the clicked points.

To do so, we need to associate 2D points with particular 3D points in the tracked object. This association is provided by the initialization file.

Here is the initialization file for the laas-box:

4
-0.175  0.175  0.156
-0.175 -0.175  0.156
 0.175 -0.175  0.156
-0.175 -0.175 -0.156

In this example, the user will click on four points. The first point coordinates are (X = -0.175, Y = 0.175, Z = 0.156).

Using your modeling software, determine the 3D coordinates of four particular points of your object (four corners for instance). They should not be on the same plane.

It is a good idea to provide an image showing in which order and where to click on the object. This file with extension .ppm is optional.

Providing the recommended tracking parameters

Optionally, you can provide the tracker.yaml file with specific tracking parameters:

Here is a good starting point:

{
mask_size: 3,
mu1: 0.5,
mu2: 0.5,
n_mask: 180,
ntotal_sample: 800,
range: 10,
sample_step: 3.,
threshold: 2000.0
}

This file is optional and not automatically read.