Show EOL distros: 

(!) Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Starting the Manipulation Pipeline on the PR2 robot

Description: Covers bringing up the manipulation pipeline on the PR2 robot, to the point where it is ready to service pick and place requests.

Tutorial Level: BEGINNER

Next Tutorial: Writing a Simple Pick and Place Application

Starting the Manipulation Pipeline

We have provided a launch file that brings up the complete manipulation pipeline, using the most common options and arguments. In general, starting the pipeline can be as easy as:

export ROBOT=pr2

if you're on an actual PR2, or

export ROBOT=sim

if you're running a simulated PR2 in Gazebo.

Then launch:

roslaunch pr2_tabletop_manipulation_launch pr2_tabletop_manipulation.launch

The default is to launch using a head Kinect. To use the stereo camera instead, pass the additional argument:

stereo:=true

If you want to use slip detection, then add the additional launch argument

use_slip_controllers:=true

This will bring up the manipulation pipeline, along with all the other services and sensor processing that it needs, to the point where it is ready to service pick and place requests. After that, you should be able to run one of the pr2_pick_and_place_demos, or write your own applications which use pick and place functionality.

The Launch Process Explained

The manipulation pipeline is a fairly complex system; to facilitate debugging, or using the pipeline in a particular way to suit your application needs, it is helpful to understand all the sub-components that are getting launched by the one launch file above, as well as the available options.

Note that if you are using the debian package installation of ROS (which is the recommended way) you will not be able to modify this launch file in place. Instead, you can copy it into one of your own packages, edit it to suit your needs, and launch it from there.

Here are all the components of this launch file, explained.

Connection to the Model Database

<!-- client for object database running on remote server at Willow Garage -->
<include file="$(find household_objects_database)/launch/objects_database_remote_client.launch"/>

<!-- alternative option: database server running on a local machine -->
<!--
<param name="/household_objects_database/database_host" 
       value="wgs36"/>
<param name="/household_objects_database/database_port" 
       value="5432"/>
<param name="/household_objects_database/database_user" 
       value="willow"/>
<param name="/household_objects_database/database_pass" 
       value="willow"/>
<param name="/household_objects_database/database_name" 
       value="household_objects-0.2"/>
<node pkg="household_objects_database" name="objects_database_node" 
      type="objects_database_node" respawn="true" output="screen"/>    
-->

The manipulation pipeline is designed to work both on known objects, identified from a database, and novel objects. The details of the object database can be found in the package household_objects_database.

There are three options for using the database:

  • remote read-only access to a server hosted at Willow Garage. This is the default option in the launch file discussed here; in general, this option should work out-of-the-box and the manipulation pipeline should be able to use the database, as long as your machine has access to the internet. Details and troubleshooting can be found in the household_objects_database package.

  • download a database backup file and install it on a local machine. Details and tutorials for doing this can be found in the household_objects_database package. Once you have installed the server and restored the database, you can use the option that is included in the launch file discussed here, but it is commented out.

  • disable database use altogether. In this case, this part of the launch file can be absent. See details below in the Sensor Processing section for disabling database use throughout the pipeline.

Manipulation Prerequisites

<!-- manipulation prerequisites -->
<include file="$(find pr2_object_manipulation_launch)/launch/pr2_manipulation_prerequisites.launch"/>

This comprises all the services on the PR2 robot needed for manipulation, including things like IK, motion planning, collision environments, etc. You should rarely have any reason to modify this entry.

Manipulation Pipeline

<!-- manipulation -->
<include file="$(find pr2_object_manipulation_launch)/launch/pr2_manipulation.launch"/>
<param name="/object_manipulator_node/default_database_planner" 
       value="/objects_database_node/database_grasp_planning" />

This is the manipulation functionality itself. The launch file provided for this purpose is pr2_object_manipulation_launch/pr2_manipulation.launch. This file will:

  • load the various components of the manipulation pipeline, and remap all the needed services to match their correct names on the PR2 robot
  • load the parameters describing the combinations of arm and hands available for manipulation tasks. For a description of these parameters see the object_manipulator package, while an example file for the PR2 gripper is provided in pr2_object_manipulation_launch.

In addition, we are telling the manipulation pipeline what is the name of the service that can be used to retrieve grasps from the database.

Sensor processing for manipulation

  <!-- tabletop perception -->
  <include file="$(find tabletop_object_detector)/launch/tabletop_node.launch"/>
  <param name="/tabletop_node/use_database" value="true"/>
  <param name="/tabletop_node/model_set" value="REDUCED_MODEL_SET" />
  <param name="/tabletop_node/get_model_list_srv" 
         value="/objects_database_node/get_model_list" />       
  <param name="/tabletop_node/get_model_mesh_srv" 
         value="/objects_database_node/get_model_mesh" />       

  <node pkg="tabletop_collision_map_processing"
        name="tabletop_collision_map_processing"         
        type="tabletop_collision_map_processing_node" 
        respawn="false" output="screen"/>
  <param name="tabletop_collision_map_processing/get_model_mesh_srv" 
         value="/objects_database_node/get_model_mesh" />

This will load:

In addition, it will inform these nodes about the names of the service that can be used to retrieve information from the database of models.

If you want to disable the use of the database altogether, you can still rely on the capability of the manipulation pipeline to handle novel objects. To disable the use of the database:

  • change the parameter /tabletop_node/use_database to false. This will remove any source of "database recognized objects" in the system, so any components downstream should never have a reason to talk to the database

  • (optional) whenever your application calls the tabletop_object_detector and asks for recognition results using the TabletopDetection service, you should set the return_models field to false; otherwise, the detector will complain that it's being asked to recognize objects even though database use is disabled, but still work normally.

Writing Applications for the Manipulation Pipeline

Note that this process only brings up the manipulation pipeline. To write an application that uses the pipeline, see the next tutorial, Writing a Simple Pick and Place Application. If you just want to make the robot move around and do stuff by teleoperation, look at pr2_interactive_manipulation.

Starting the robot

On a real PR2, start the robot normally, i.e.:

robot start

For simulation, you can use the following:

roslaunch manipulation_worlds pr2_table_object.launch

Also do:

export ROBOT=pr2

if you're on an actual PR2, or

export ROBOT=sim

if you're running a simulated PR2 in Gazebo.

Starting the Manipulation Pipeline

We have provided a launch file that brings up the complete manipulation pipeline, using the most common options and arguments. In general, starting the pipeline can be as easy as:

roslaunch pr2_tabletop_manipulation_launch pr2_tabletop_manipulation.launch

The default is to launch using a head Kinect. To use the stereo camera instead, pass the additional argument:

stereo:=true

If you want to use slip detection, then add the additional launch argument

use_slip_controllers:=true

This will bring up the manipulation pipeline, along with all the other services and sensor processing that it needs, to the point where it is ready to service pick and place requests. After that, you should be able to run one of the pr2_pick_and_place_demos, or write your own applications which use pick and place functionality.

The Launch Process Explained

The manipulation pipeline is a fairly complex system; to facilitate debugging, or using the pipeline in a particular way to suit your application needs, it is helpful to understand all the sub-components that are getting launched by the one launch file above, as well as the available options.

Note that if you are using the debian package installation of ROS (which is the recommended way) you will not be able to modify this launch file in place. Instead, you can copy it into one of your own packages, edit it to suit your needs, and launch it from there.

Here are all the components of this launch file, explained.

Connection to the Model Database

  <!-- database server running on a local machine -->
  <rosparam command="load" file="$(find  household_objects_database)/config/wgs36.yaml"/>
  <node pkg="household_objects_database" name="objects_database_node" type="objects_database_node" 
        respawn="true" output="screen"/>    

The manipulation pipeline is designed to work both on known objects, identified from a database, and novel objects. The details of the object database can be found in the package household_objects_database.

This command will load on the parameter server the connection information for the database (server, port, username, password, etc.) then start a ROS node that provide services for accessing the database.

You can download a database backup file and install it on a local machine. Details and tutorials for doing this can be found in the household_objects_database package. Once you have installed the server and restored the database, you can write your appropriate config file for it and launch the same node as above.

Manipulation Prerequisites

<!-- manipulation prerequisites -->
<include file="$(find pr2_object_manipulation_launch)/launch/pr2_manipulation_prerequisites.launch"/>

This comprises all the services on the PR2 robot needed for manipulation, including things like IK, motion planning, collision environments, etc. You should rarely have any reason to modify this entry.

Manipulation Pipeline

  <!-- manipulation -->
  <include file="$(find pr2_object_manipulation_launch)/launch/pr2_manipulation.launch">
    <arg name="use_slip_controllers" value="$(arg use_slip_controllers)"/>
    <arg name="use_left_arm" value="$(arg use_left_arm)"/>
    <arg name="use_right_arm" value="$(arg use_right_arm)"/>
    <arg name="use_task_cartesian" value="$(arg use_task_cartesian)"/>
    <arg name="sim" value="$(arg sim)"/>
  </include>

This is the manipulation functionality itself. The launch file provided for this purpose is pr2_object_manipulation_launch/pr2_manipulation.launch. This file will:

  • load the various components of the manipulation pipeline, and remap all the needed services to match their correct names on the PR2 robot
  • load the parameters describing the combinations of arm and hands available for manipulation tasks. For a description of these parameters see the object_manipulator package, while an example file for the PR2 gripper is provided in pr2_object_manipulation_launch.

In addition, we are telling the manipulation pipeline what is the name of the service that can be used to retrieve grasps from the database.

Sensor processing for manipulation

  <!-- tabletop collision map processing -->
  <node pkg="tabletop_collision_map_processing" name="tabletop_collision_map_processing" 
        type="tabletop_collision_map_processing_node" respawn="false" output="screen"/>
  <param name="tabletop_collision_map_processing/get_model_mesh_srv" 
         value="/objects_database_node/get_model_mesh" />
  <param name="tabletop_collision_map_processing/static_map_cloud_name" value="full_cloud_filtered" />

  <!-- tabletop segmentation and object recognition -->
  <include file="$(find tabletop_object_detector)/launch/tabletop_complete.launch">
      <arg unless="$(arg stereo)" name="tabletop_segmentation_points_input" value="$(arg kinect_camera_name)/depth_registered/points"/>
      <arg if="$(arg stereo)" name="tabletop_segmentation_points_input" value="narrow_stereo_textured/points2"/>
      <arg name="flatten_table" value="$(arg flatten_table)"/>
  </include>

This will load:

If you want to disable the use of the database altogether and manipulation only novel objects based on their run-time point cloud: while this is generally possible, the launch file described here does not provide a simple parameter for that. You will have to dig a little bit through the code and figure out how to stop launching the things that hang waiting for the database, or tell them not to use it.

Writing Applications for the Manipulation Pipeline

Note that this process only brings up the manipulation pipeline. To write an application that uses the pipeline, see the next tutorial, Writing a Simple Pick and Place Application. If you just want to make the robot move around and do stuff by teleoperation, look at pr2_interactive_manipulation.

Starting the robot

On a real PR2, start the robot normally, i.e.:

robot start

For simulation, you can use the following:

roslaunch manipulation_worlds pr2_table_object.launch

Also do:

export ROBOT=pr2

if you're on an actual PR2, or

export ROBOT=sim

if you're running a simulated PR2 in Gazebo.

Starting the Manipulation Pipeline

We have provided a launch file that brings up the complete manipulation pipeline, using the most common options and arguments. In general, starting the pipeline can be as easy as:

roslaunch pr2_tabletop_manipulation_launch pr2_tabletop_manipulation.launch

The default is to launch using a head Kinect. To use the stereo camera instead, pass the additional argument:

stereo:=true

If you want to use slip detection, then add the additional launch argument

use_slip_controllers:=true

This will bring up the manipulation pipeline, along with all the other services and sensor processing that it needs, to the point where it is ready to service pick and place requests. After that, you should be able to run one of the pr2_pick_and_place_demos, or write your own applications which use pick and place functionality.

The Launch Process Explained

The manipulation pipeline is a fairly complex system; to facilitate debugging, or using the pipeline in a particular way to suit your application needs, it is helpful to understand all the sub-components that are getting launched by the one launch file above, as well as the available options.

Note that if you are using the debian package installation of ROS (which is the recommended way) you will not be able to modify this launch file in place. Instead, you can copy it into one of your own packages, edit it to suit your needs, and launch it from there.

Here are all the components of this launch file, explained.

Connection to the Model Database

  <!-- database server running on a local machine -->
  <rosparam command="load" file="$(find  household_objects_database)/config/wgs36.yaml"/>
  <node pkg="household_objects_database" name="objects_database_node" type="objects_database_node" 
        respawn="true" output="screen"/>    

The manipulation pipeline is designed to work both on known objects, identified from a database, and novel objects. The details of the object database can be found in the package household_objects_database.

This command will load on the parameter server the connection information for the database (server, port, username, password, etc.) then start a ROS node that provide services for accessing the database.

You can download a database backup file and install it on a local machine. Details and tutorials for doing this can be found in the household_objects_database package. Once you have installed the server and restored the database, you can write your appropriate config file for it and launch the same node as above.

Manipulation Prerequisites

<!-- manipulation prerequisites -->
<include file="$(find pr2_object_manipulation_launch)/launch/pr2_manipulation_prerequisites.launch"/>

This comprises all the services on the PR2 robot needed for manipulation, including things like IK, motion planning, collision environments, etc. You should rarely have any reason to modify this entry.

Manipulation Pipeline

  <!-- manipulation -->
  <include file="$(find pr2_object_manipulation_launch)/launch/pr2_manipulation.launch">
    <arg name="use_slip_controllers" value="$(arg use_slip_controllers)"/>
    <arg name="use_left_arm" value="$(arg use_left_arm)"/>
    <arg name="use_right_arm" value="$(arg use_right_arm)"/>
    <arg name="use_task_cartesian" value="$(arg use_task_cartesian)"/>
    <arg name="sim" value="$(arg sim)"/>
  </include>

This is the manipulation functionality itself. The launch file provided for this purpose is pr2_object_manipulation_launch/pr2_manipulation.launch. This file will:

  • load the various components of the manipulation pipeline, and remap all the needed services to match their correct names on the PR2 robot
  • load the parameters describing the combinations of arm and hands available for manipulation tasks. For a description of these parameters see the object_manipulator package, while an example file for the PR2 gripper is provided in pr2_object_manipulation_launch.

In addition, we are telling the manipulation pipeline what is the name of the service that can be used to retrieve grasps from the database.

Sensor processing for manipulation

  <!-- tabletop collision map processing -->
  <node pkg="tabletop_collision_map_processing" name="tabletop_collision_map_processing" 
        type="tabletop_collision_map_processing_node" respawn="false" output="screen"/>
  <param name="tabletop_collision_map_processing/get_model_mesh_srv" 
         value="/objects_database_node/get_model_mesh" />
  <param name="tabletop_collision_map_processing/static_map_cloud_name" value="full_cloud_filtered" />

  <!-- tabletop segmentation and object recognition -->
  <include file="$(find tabletop_object_detector)/launch/tabletop_complete.launch">
      <arg unless="$(arg stereo)" name="tabletop_segmentation_points_input" value="$(arg kinect_camera_name)/depth_registered/points"/>
      <arg if="$(arg stereo)" name="tabletop_segmentation_points_input" value="narrow_stereo_textured/points2"/>
      <arg name="flatten_table" value="$(arg flatten_table)"/>
  </include>

This will load:

If you want to disable the use of the database altogether and manipulation only novel objects based on their run-time point cloud: while this is generally possible, the launch file described here does not provide a simple parameter for that. You will have to dig a little bit through the code and figure out how to stop launching the things that hang waiting for the database, or tell them not to use it.

Writing Applications for the Manipulation Pipeline

Note that this process only brings up the manipulation pipeline. To write an application that uses the pipeline, see the next tutorial, Writing a Simple Pick and Place Application. If you just want to make the robot move around and do stuff by teleoperation, look at pr2_interactive_manipulation.

Wiki: pr2_tabletop_manipulation_apps/Tutorials/Starting the Manipulation Pipeline (last edited 2012-11-30 00:27:27 by MateiCiocarlie)