Person Tracking and Reconstruction from a Mobile Base with a 7 DOF Manipulator
Description: Person tracking and reconstruction from a mobile base with a 7 DOF manipulatorSubmitted By: Chris Burbridge & Lorenzo Riano ( University of Ulster Intelligent Systems Research Centre http://isrc.ulster.ac.uk )
Keywords: Kinect, Reconstruction, Schunk, Dynamic Window, Potential fields
The skeleton tracking library provided by PrimeSense provides a convenient way to extract and track people from a depth image. By combining this with the power of ROS, we have created a system to create a 3D model of a person using a mobile robot.
The mobile robot autonomously drives around the subject by using knowledge of the person's location, as acquired from the skeleton tracking system. In order to achieve efficient and safe navigation, we created a dynamic window controller node and combined it with a potential field. A Microsoft Kinect camera attached to the end of a 7 DOF Schunk Manipulator was autonomously controlled to maintain the user in view as the robot moved. In order to achieve this, we created a URDF model of the manipulator and adapted the PR2 Kinematics node to work with our robot.
As the robot drives, the points corresponding to the person are extracted from the RGB-D camera's point cloud. This is provided by an additional node named "person_grabber", which is a combination of the skeleton tracking example and the openni_camera nodes. The extracted points are then processed offline using the PCL library to register successive frames and render a reconstructed 3D model of the subject.
Video
How to Reproduce Your Entry
In order to reproduce our entry exactly, a MetraLabs Scitos G5 mobile robot equipped with a Schunk 7 D.O.F manipulator and a Microsoft Kinect camera is required. However, our entry can be partially reproduced using any robot equipped with an arm and a Kinect camera. As far as possible we have used standard message types, hopefully allowing our code to be easier ported if desired.
The code is split into several useful packages:
metralabs_ros
We created an interface for the MetraLabs robot. This uses the MetraLabs API to control the differential drive base and Schunk manipulator. The robot and manipulator is described in a URDF model file, which is loaded by the node and managed with a robot_state_publisher. JointState messages and "standard" topic names are used as far as possible to maintain compatibility with other robots.schunk_kinematics
This node provides forward and inverse kinematics. It is essentially the same as the PR2 kinematics, but with the addition of inverse velocity.schunk_gui
This node provides a gui for testing and controlling the manipulator.person_grabber
This node provides the skeleton transformations and person points from a single node. It is a hacking together of the openni_camera node and the openni_tracker node. Only points corresponding to the tracked user are published.arm_track
This node receives the skeleton position, and uses the kinematics node to calculate joint velocities that will keep the camera pointing at the person.dynamic_window
This node provides a dynamic window based obstacle avoidance. It subscribes to a laser topic and a commanded velocity, and publishes a safe velocity.potential_fields
This node provides the calculation of a potential field that will drive the robot around the person. It publishes command velocities to the dynamic window controller.velocity_safety
This node listens to manipulator joint velocities, and passes them on to the robot if they are not too unsafe.pcl_tests
These nodes save point clouds as they are published.
The point cloud is saved using the pointcloud_save node within the pcl_tests package. Once several clouds have been saved, the 3D model can be obtained using the ROS pcl package.
Code to Checkout
Install ros-unstable-base
Install the ni stack: see http://www.ros.org/wiki/ni.
Check out the uuisrc-ros-pkg repository:
git clone git://github.com/cburbridge/uuisrc-ros-pkg.git
All of the above packages are within the checked out repository. The packages can be compiled with:
rosmake <package-name>
If you have any queries, please feel free to contact us.
Dependencies
Software:
openni
Hardware:
MetraLabs Scitos G5 mobile robot
- Schunk 7 DOF manipulator
- Microsoft Kinect
