About | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange

Running the IAP segmentation and kinematic analyzer on prerecorded data

Description: The RBO interactive perception project (RBO-IAP) is concerned with the detection and interactive exploration of kinematic objects, i.e. objects with degrees of freedom. This tutorial shows you how to run the segmentation and the kinematic analyzer on data from a bag file.

Keywords: kinematic structure, image segmentation, articulation models, kinematic models

Tutorial Level: BEGINNER

The RBO-IAP package contains several demo data and tutorial launch files which you can use as a template for your own applications.

The first tutorial shows how to use the kinematic analyzer with prerecorded of artificial data bag files.

Getting the demo data

Go to https://github.com/tu-rbo/iap-demos and check out the launch files in the root folder as well as the folder artificial_data.

You can check out the whole repository by typing

• git clone https://github.com/tu-rbo/iap-demos.git

Attention: The folder real_data contains bag files of several gigabytes!

Artificial data

One revolute and one prismatic motion between a pair of boxes was simulated using our IAP simulator (release in preparation). For each of the two boxes, a random set of 25 features was generated which were directly published to /iap/feature_set_3d. There is no noise, so no features are lost, etc. and we can consider the data as being generated by a perfect feature tracker. When the motion is over, the simulator additionally publishes a perfect segmentation of the features into two clusters, each corresponding to one of the boxes. This segmentation is published to the topic /iap/segmentation. It also triggers the kinematic analysis.

The two bag files containing the data are iap_sim_revolute_motion.bag and iap_sim_prism_motion.bag.

Additionally, you find two video files, iap_sim_prism_motion.mpg and iap_sim_revolute_motion.mpg which give an impression of how the generated data looks like. It is important to mention, that the videos does were not used to generate the features - the features are directly generated in the simulator.

The corresponding launch files are

• artificial_data_revolute.launch
• artificial_data_prismatic.launch

The following nodes are called in the launch file:

• rviz: Visualization of the features, the found revolute / prismatic axis and the error cones

• rosbag: For playing the data

• kinematic_analyzer: Calculates the different kinematic relationships and publishes the most probable one

Take a look at the nodes and the parameters of the kinematic analyzer to learn more.

Real Data

In this tutorial we provide demo data and launch files to analyze it using the RBO-IAP. The provided rosbag files capture two scenarios: the robot actuates a prismatic joint (piece of furniture with drawers) and the robot actuates a revolute joint (piece of furniture with a door), and the scene is captured using a 3D sensor like Microsoft Kinect (RGBD). You can find the data in the folder:

• kinematic_analyzer/demos/real_data

The different bag files contain the input data in different processing steps:

• _all.bag files -> This is the raw data coming from a kinect-like sensor. It contains the necessary data to run the whole RBO-IAP pipeline, feature tracker, visual net and kinematic analyzer. You can play with this bag files, select manually the points to track, segment them manually or let the pipeline to run automatically.

• _f.bag files -> These files contain the data coming from the feature tracker. A set of reliable features was selected so you can focus on the segmentation (Visual Net) and the kinematic analysis.

• _f_s.bag files -> The data on this bags contains the results of the feature tracker and the segmentation by the Visual Net. Use this files if you want to deal only with the kinematic analysis.

Running the demos

In the folder you will find a suitable launch file to analyze each bag file. The launch files will load and define parameters and will launch the required nodes. You can change the parameters defined in these files, and also the parameters of each node to be executed (config files). The launch files also start rviz to visualize the final results. The vast amount of parameters makes imposible to provide launch files changing each one, just play around and find the most suitable combination for you!

The launch file don't play the rosbag files. The best way to proceed is to run the bag files manually, so you can stop the execution at certain time steps. Use the command:

• rosbag play --pause file.bag

The --pause option will make necessary that you press 's' to step the file.

If you run the _f.bag files, you should probably type first in terminal:

• rosparam set use_sim_time true

and the for launching the bag file:

• rosbag play --pause --clock file.bag

Basically, this uses the time of the simulation instead of the time contained in the bag file. This avoids the point features played from the bag file to be rejected because being older than the last segmentation.