Documentation Status

openni_kinect: depth_image_proc | nite | openni | openni_camera | openni_launch | openni_tracker

Package Summary

Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include:

  • Microsoft Kinect
  • PrimeSense PSDK
  • ASUS Xtion Pro (no RGB)
The driver publishes raw depth, RGB, and IR image streams.

openni_camera

Package Summary

Released Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include: Microsoft Kinect, PrimeSense PSDK, ASUS Xtion Pro and Pro Live The driver publishes raw depth, RGB, and IR image streams.

Package Summary

Released Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include: Microsoft Kinect, PrimeSense PSDK, ASUS Xtion Pro and Pro Live The driver publishes raw depth, RGB, and IR image streams.

Package Summary

Released Continuous integration Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include: Microsoft Kinect, PrimeSense PSDK, ASUS Xtion Pro and Pro Live The driver publishes raw depth, RGB, and IR image streams.

  • Maintainer status: maintained
  • Maintainer: Isaac I.Y. Saito <130s AT 2000.jukuin.keio.ac DOT jp>
  • Author: Patrick Mihelich, Suat Gedikli, Radu Bogdan Rusu
  • License: BSD
  • Source: git https://github.com/ros-drivers/openni_camera.git (branch: indigo-devel)

Package Summary

Released Continuous integration Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include: Microsoft Kinect, PrimeSense PSDK, ASUS Xtion Pro and Pro Live The driver publishes raw depth, RGB, and IR image streams.

  • Maintainer status: maintained
  • Maintainer: Isaac I.Y. Saito <130s AT 2000.jukuin.keio.ac DOT jp>
  • Author: Patrick Mihelich, Suat Gedikli, Radu Bogdan Rusu
  • License: BSD
  • Source: git https://github.com/ros-drivers/openni_camera.git (branch: indigo-devel)

Package Summary

Released Continuous integration Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include: Microsoft Kinect, PrimeSense PSDK, ASUS Xtion Pro and Pro Live The driver publishes raw depth, RGB, and IR image streams.

  • Maintainer status: maintained
  • Maintainer: Isaac I.Y. Saito <130s AT 2000.jukuin.keio.ac DOT jp>
  • Author: Patrick Mihelich, Suat Gedikli, Radu Bogdan Rusu
  • License: BSD
  • Source: git https://github.com/ros-drivers/openni_camera.git (branch: indigo-devel)

Package Summary

Released Continuous integration Documented

A ROS driver for OpenNI depth (+ RGB) cameras. These include: Microsoft Kinect, PrimeSense PSDK, ASUS Xtion Pro and Pro Live The driver publishes raw depth, RGB, and IR image streams.

  • Maintainer status: maintained
  • Maintainer: Isaac I.Y. Saito <130s AT 2000.jukuin.keio.ac DOT jp>
  • Author: Patrick Mihelich, Suat Gedikli, Radu Bogdan Rusu
  • License: BSD
  • Source: git https://github.com/ros-drivers/openni_camera.git (branch: indigo-devel)

openni_camera is not available in C Turtle. Select a newer distro above.

The Diamondback OpenNI driver API is deprecated as of Fuerte. For migration help and API reference, see openni_camera_deprecated.

Kinect Accuracy and Precision

openni_kinect/kinect_accuracy

Kinect Calibration

openni_camera/calibration

OpenNI Demo

Don't forget to build the driver if you haven't already done so:

rosmake openni_camera

Launch the OpenNI driver:

roslaunch openni_camera openni_node.launch

To view in rviz, launch rviz:

rosrun rviz rviz

Set the Fixed Frame (top left of rviz window) to /openni_rgb_optical_frame.

Add a Point Cloud2 display, and set the topic to /camera/rgb/points.

Turning the background to light gray can help with viewing.

To view the color image in image_viewer:

rosrun image_view image_view image:=/camera/rgb/image_color

or following to view the grayscale image:

rosrun image_view image_view image:=/camera/rgb/image_mono

Changing driver settings

start dynamic_reconfigure

rosrun dynamic_reconfigure reconfigure_gui

select /openni_node1. You should see something like this:

dynamic_reconfigure.jpg

  1. image mode:
    • Kinect supports natively just SXGA@15Hz, which actually streams ~10Hz, and VGA@30Hz. All other modes at 30Hz are down sampled in the driver and modes at 25Hz are not supported by the Kinect.
    • Primesense device does not support SXGA yet, but all VGA and QVGA modes - only QQVGA is down sampled by the driver.
  2. debayering: Only valid for Kinect, since Primesense device streams uses YUV.
    • Bilinear: A fast debayering algorithm, but poor results on edges.
    • EdgeAware: uses linear interpolation only in direction with lowest gradient. (recommended)

    • EdgeAwareWeighted: uses bilinear interpolation weighted with the according gradient.

    • Note: for down sampled images, the debayering is done on a 2x2 block.
  3. depth mode: same as image mode, except no SXGA is supported.
  4. depth_registration: whether the depth image should be registered to the RGB image or not.
    • Note: if registration is turned on, the point clouds are in the '/openni_rgb_optical_frame', otherwise they are in '/openni_depth_optical_frame'.
    • Note: subscribing to /camera/rgb/points turns on registration automatically and can not turned if any nodes subscribe to that topic.
  5. depth_time_offset: Since the device does not return a time stamp which is synchronized with system clock, there is a delay between the capturing time and the time where the ROS message is created. This value can be used to add an offset time to the ROS message time stamp.
  6. image_time_offset: similar to depth_time_offset

Synchronization

Other than the Primesense devices, the Kinect does not support hardware synchronization. Thus the time points where the RGB image and the depth image are captured may be up to ~16ms away from each other. For all devices we use a Approximatetimesynchronizer to generate the point clouds /camera/rgb/points. For Primesense devices the hardware synchronization is turned on automatically if both streams are used (subscribing to image + depth).

Coordinate Frames

As per REP103 the following coordinate frames are layed out for the kinect/Primesense Reference Device

/openni_camera
|
|> /openni_rgb_frame
|  |
|  |> /openni_rgb_optical_frame  (Z forward)
|
|> /openni_depth_frame
   |
   |> /openni_depth_optical_frame (Z forward)

Overview

This package provides a ROS interface to depth sensors using the OpenNI standard. These currently include:

  • Microsoft Kinect
  • ASUS Xtion PRO (no RGB)
  • ASUS Xtion PRO Live (with RGB)
  • PrimeSense PSDK 5.0

openni_devices.jpg

openni_launch is the best place to begin using your Kinect or similar device. It provides processed outputs such as point clouds.

As of Electric, openni_camera has been trimmed down to just the driver node(let) publishing the raw depth, IR and (if applicable) RGB images. As of Fuerte, it has been trimmed of some heavyweight dependencies (OpenCV, PCL) and made a unary stack to ease installing and running openni_camera on resource-constrained systems.

(If you are using the old, monolithic driver API dating to Diamondback, see openni_camera_deprecated for migration instructions and API reference.)

Installation

Ubuntu installation

To install only openni_camera:

sudo apt-get install ros-electric-openni-camera

It's also recommended to install openni_launch:

sudo apt-get install ros-electric-openni-launch

ROS API

openni_node

OpenNI camera driver.

Published Topics

RGB camera
rgb/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata.
rgb/image_raw (sensor_msgs/Image)
  • Raw image from device. Format is Bayer GRBG for Kinect.
Depth camera
Published only when ~depth_registration is false (OpenNI registration disabled).
depth/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata.
depth/image_raw (sensor_msgs/Image)
  • Raw image from device. Contains uint16 depths in mm.
Registered depth camera (aligned with RGB camera)
Published directly by driver only when ~depth_registration is true (OpenNI registration enabled).
depth_registered/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata. Same as rgb/camera_info but time-synced to depth_registered/image_raw.
depth_registered/image_raw (sensor_msgs/Image)
  • Raw image from device. Contains uint16 depths in mm.
IR camera
ir/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata.
ir/image_raw (sensor_msgs/Image)
  • Raw uint16 IR image.
IR projector
projector/camera_info (sensor_msgs/CameraInfo)
  • Faked calibration for the IR projector. Identical to depth/camera_info, but with the baseline encoded in the P matrix. Stereo processing nodes can use the depth and projector infos as a left/right pair.

Services

rgb/set_camera_info (sensor_msgs/SetCameraInfo)
  • Set the RGB camera calibration.
ir/set_camera_info (sensor_msgs/SetCameraInfo)
  • Set the IR camera calibration.

Parameters

~device_id (string, default: #1)
  • Specifies which device to open. The following formats are recognized:
    • #1 Use first device found
      2@3 Use device on USB bus 2, address 3
      B00367707227042B Use device with given serial number
~rgb_frame_id (string, default: /openni_rgb_optical_frame)
  • The tf frame of the RGB camera.
~depth_frame_id (string, default: /openni_depth_optical_frame)
  • The tf frame of the IR/depth camera.
~rgb_camera_info_url (string, default: file://${ROS_HOME}/camera_info/${NAME}.yaml)
  • Calibration URL for the RGB camera. By default, looks in your ROS home directory for calibration identified by the device serial number, e.g. $HOME/.ros/camera_info/rgb_B00367707227042B. If no calibration is found, uses a default camera model with a typical focal length and distortion unmodeled.
~depth_camera_info_url (string, default: file://${ROS_HOME}/camera_info/${NAME}.yaml)
  • Calibration URL for the IR/depth camera. By default, looks in your ROS home directory for calibration identified by the device serial number, e.g. $HOME/.ros/camera_info/depth_B00367707227042B. If no calibration is found, uses a default camera model with the focal length reported by OpenNI and distortion unmodeled.
~time_out (double)
  • If set, checks every ~time_out seconds whether active cameras have streamed new data.
Dynamically Reconfigurable Parameters
See the dynamic_reconfigure package for details on dynamically reconfigurable parameters.
~image_mode (int, default: 2)
  • Image output mode for the color/grayscale image Possible values are: SXGA_15Hz (1): 1280x1024@15Hz, VGA_30Hz (2): 640x480@30Hz, VGA_25Hz (3): 640x480@25Hz, QVGA_25Hz (4): 320x240@25Hz, QVGA_30Hz (5): 320x240@30Hz, QVGA_60Hz (6): 320x240@60Hz, QQVGA_25Hz (7): 160x120@25Hz, QQVGA_30Hz (8): 160x120@30Hz, QQVGA_60Hz (9): 160x120@60Hz
~depth_mode (int, default: 2)
  • depth output mode Possible values are: SXGA_15Hz (1): 1280x1024@15Hz, VGA_30Hz (2): 640x480@30Hz, VGA_25Hz (3): 640x480@25Hz, QVGA_25Hz (4): 320x240@25Hz, QVGA_30Hz (5): 320x240@30Hz, QVGA_60Hz (6): 320x240@60Hz, QQVGA_25Hz (7): 160x120@25Hz, QQVGA_30Hz (8): 160x120@30Hz, QQVGA_60Hz (9): 160x120@60Hz
~depth_registration (bool, default: False)
  • Depth data registration
~data_skip (int, default: 0)
  • Skip N images for every image published (rgb/depth/depth_registered/ir) Range: 0 to 10
~depth_time_offset (double, default: 0.0)
  • depth image time offset in seconds Range: -1.0 to 1.0
~image_time_offset (double, default: 0.0)
  • image time offset in seconds Range: -1.0 to 1.0
~depth_ir_offset_x (double, default: 5.0)
  • X offset between IR and depth images Range: -10.0 to 10.0
~depth_ir_offset_y (double, default: 4.0)
  • Y offset between IR and depth images Range: -10.0 to 10.0
~z_offset_mm (int, default: 0)
  • Z offset in mm Range: -50 to 50

openni_camera/driver nodelet

Nodelet version of the OpenNI driver. Has the same ROS API as openni_node above.

openni_camera is now a unary stack. Previously it was a package in openni_kinect.

Overview

This package provides a ROS interface to depth sensors using the OpenNI standard. These currently include:

  • Microsoft Kinect
  • ASUS Xtion PRO (no RGB)
  • ASUS Xtion PRO Live (with RGB)
  • PrimeSense PSDK 5.0

openni_devices.jpg

openni_launch is the best place to begin using your Kinect or similar device. It provides processed outputs such as point clouds.

As of Electric, openni_camera has been trimmed down to just the driver node(let) publishing the raw depth, IR and (if applicable) RGB images. As of Fuerte, it has been trimmed of some heavyweight dependencies (OpenCV, PCL) and made a unary stack to ease installing and running openni_camera on resource-constrained systems.

(If you are using the old, monolithic driver API dating to Diamondback, see openni_camera_deprecated for migration instructions and API reference.)

openni_launch

It is highly recommended to use this package through openni_launch, which provides additional RGB-D processing capabilities for the data produced by openni_node

Installation

Ubuntu installation

To install only openni_camera:

sudo apt-get install ros-<rosdistro>-openni-camera

It's also recommended to install openni_launch:

sudo apt-get install ros-<rosdistro>-openni-launch

To use the Microsoft Kinect you also need a driver:

ROS API

openni_node

OpenNI camera driver.

Published Topics

RGB camera
rgb/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata.
rgb/image_raw (sensor_msgs/Image)
  • Raw image from device. Format is Bayer GRBG for Kinect.
Depth camera
Published only when ~depth_registration is false (OpenNI registration disabled).
depth/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata.
depth/image_raw (sensor_msgs/Image)
  • Raw image from device. Contains uint16 depths in mm.
Registered depth camera (aligned with RGB camera)
Published directly by driver only when ~depth_registration is true (OpenNI registration enabled).
depth_registered/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata. Same as rgb/camera_info but time-synced to depth_registered/image_raw.
depth_registered/image_raw (sensor_msgs/Image)
  • Raw image from device. Contains uint16 depths in mm.
IR camera
ir/camera_info (sensor_msgs/CameraInfo)
  • Camera calibration and metadata.
ir/image_raw (sensor_msgs/Image)
  • Raw uint16 IR image.
IR projector
projector/camera_info (sensor_msgs/CameraInfo)
  • Faked calibration for the IR projector. Identical to depth/camera_info, but with the baseline encoded in the P matrix. Stereo processing nodes can use the depth and projector infos as a left/right pair.

Services

rgb/set_camera_info (sensor_msgs/SetCameraInfo)
  • Set the RGB camera calibration.
ir/set_camera_info (sensor_msgs/SetCameraInfo)
  • Set the IR camera calibration.

Parameters

~device_id (string, default: #1)
  • Specifies which device to open. The following formats are recognized:
    • #1 Use first device found
      2@3 Use device on USB bus 2, address 3
      B00367707227042B Use device with given serial number
~rgb_frame_id (string, default: /openni_rgb_optical_frame)
  • The tf frame of the RGB camera.
~depth_frame_id (string, default: /openni_depth_optical_frame)
  • The tf frame of the IR/depth camera.
~rgb_camera_info_url (string, default: file://${ROS_HOME}/camera_info/${NAME}.yaml)
  • Calibration URL for the RGB camera. By default, looks in your ROS home directory for calibration identified by the device serial number, e.g. $HOME/.ros/camera_info/rgb_B00367707227042B. If no calibration is found, uses a default camera model with a typical focal length and distortion unmodeled.
~depth_camera_info_url (string, default: file://${ROS_HOME}/camera_info/${NAME}.yaml)
  • Calibration URL for the IR/depth camera. By default, looks in your ROS home directory for calibration identified by the device serial number, e.g. $HOME/.ros/camera_info/depth_B00367707227042B. If no calibration is found, uses a default camera model with the focal length reported by OpenNI and distortion unmodeled.
~time_out (double)
  • If set, checks every ~time_out seconds whether active cameras have streamed new data.
Dynamically Reconfigurable Parameters
See the dynamic_reconfigure package for Fuerte, and rqt_reconfigure package for ROS Groovy onwards on dynamically reconfigurable parameters.
~image_mode (int, default: 2)
  • Image output mode for the color/grayscale image Possible values are: SXGA_15Hz (1): 1280x1024@15Hz, VGA_30Hz (2): 640x480@30Hz, VGA_25Hz (3): 640x480@25Hz, QVGA_25Hz (4): 320x240@25Hz, QVGA_30Hz (5): 320x240@30Hz, QVGA_60Hz (6): 320x240@60Hz, QQVGA_25Hz (7): 160x120@25Hz, QQVGA_30Hz (8): 160x120@30Hz, QQVGA_60Hz (9): 160x120@60Hz
~depth_mode (int, default: 2)
  • depth output mode Possible values are: SXGA_15Hz (1): 1280x1024@15Hz, VGA_30Hz (2): 640x480@30Hz, VGA_25Hz (3): 640x480@25Hz, QVGA_25Hz (4): 320x240@25Hz, QVGA_30Hz (5): 320x240@30Hz, QVGA_60Hz (6): 320x240@60Hz, QQVGA_25Hz (7): 160x120@25Hz, QQVGA_30Hz (8): 160x120@30Hz, QQVGA_60Hz (9): 160x120@60Hz
~depth_registration (bool, default: False)
  • Depth data registration
~data_skip (int, default: 0)
  • Skip N images for every image published (rgb/depth/depth_registered/ir) Range: 0 to 10
~depth_time_offset (double, default: 0.0)
  • depth image time offset in seconds Range: -1.0 to 1.0
~image_time_offset (double, default: 0.0)
  • image time offset in seconds Range: -1.0 to 1.0
~depth_ir_offset_x (double, default: 5.0)
  • X offset between IR and depth images Range: -10.0 to 10.0
~depth_ir_offset_y (double, default: 4.0)
  • Y offset between IR and depth images Range: -10.0 to 10.0
~z_offset_mm (int, default: 0)
  • Z offset in mm Range: -50 to 50

openni_camera/driver nodelet

Nodelet version of the OpenNI driver. Has the same ROS API as openni_node above.

Report a Bug

Please report all bugs at https://github.com/ros-drivers/openni_camera/issues

See also

Use Kinect on Ubuntu with OpenNI

Note: This is the prerequisite for your usage of openni_camera / openni_launch for Kinect in ROS. You have to firstly install the Driver for Kinect on Ubuntu and make sure your kinect can work properly on ubuntu.

To get started, please Refer this blog and you may meet some incompatible contents, do not worry, the thought and path is correct.

The very important doc you should read carefully is that on OpenNI's GitHub site and that on SensorKinect GitHub site.

  • The Core thing for this is Installing Driver for Kinect on Ubuntu. The two important packages you should install are: The OpenNI UNSTABLE version

    The SensorKinect package

  • Here, I want to say that: after you have installed the OpenNI, and be ready to clone the git files for SensorKinect,DO NOT change directory!Otherwise you cannot generate the Redist directory for installation of SensorKinect.

  • The Cloning process could be very slow, you can checkout the files from here:

    http://yunpan.cn/cmTPmvr2D29Yj keycode: 32e4

  • There existed a Readme.txt file which is wriiten in Chinese. Don't worry, you can ignore it after you suffer this Installation for longer than 1 week cuz you will be the expert on this field at that time. ;-)

  • When you succeded in using Kinect_for_Windows on Ubuntu, the using of Kinect_for_Xbox is just 'unplug and plugin'.
  • It's really easier if you use Kinect_for_XBox with OpenKinect. unfortunately, OpenKinect does not support Kinect4Windows.

  • If you're using Virtual Machine launching Ubuntu on it, I don't recommend you continue. I failed too many times in trying to do it on VMs. And finally I succeded in real machine.
  • Read Carefully about This Blog: this blog. and follow it. Good luck!

Wiki: openni_camera (last edited 2016-04-27 08:44:08 by MartinGuenther)