Only released in EOL distros:  

Overview

This package introduces GUI Primitives for HRI that are designed to visualize and illustrate objects detected by the robot. These primitives also allow to interact with particular objects and give commands to the robot via them. The primitives consists of Interactive Markers and are visualized in RViz.

All primitives have a context menu which allows some interaction and setting of the visualization (ability to show description of the primitive, add/remove manipulators for rotation and translation, etc.).

Following predefined types of primitives and are specified in srs_env_model/PrimitiveType message.

Interaction Primitives

Billboard

billboard-chair.pngbillboard-person-head.png

  • The billboard can also illustrate movement of the represented object, e.g. walking person.

billboard-person.png

Bounding Box

  • Bounding Box illustrates dimensions of the object.
  • The bounding box is able visualize its dimensions as text labels.
  • The bounding box can be manually translated or rotated.

bbox.png

Object

  • Object represents a detected or real-world object which has its mesh in an object database.
  • Possible pre-grasp positions can be shown around the object. The visualization of pre-grasp positions aids operator to move gripper to a correct position for the grasping.
  • The object can show its bounding box (if specified).
  • The object can be manually rotated, translated and scaled in the scene.

pre-grasp2.png

object-with-bbox.png

Unknown Object

  • Unknown Object represents obstacles, unreachable objects or dangerous places in the scene so that the operator will be able to avoid collisions or crashes of the robot.
  • The unknown object can be manually rotated, translated and scaled in the scene.

unknown.png

Plane

  • Plane shows only a simple plane which can be tagged as table desk, wall, etc.

plane.png

Plane Polygon

  • It's a variation of the Plane which is defined as a planar polygon.
  • The plane polygon is shown as a transparent plane with the polygon inside it.

plane-polygon.png

Others

Clickable positions

Clickable positions can be added to the scene using /clickable_positions service or ClickablePositions action.

By clicking on the position are it's coordinates published on the specified topic.

Robot's pose prediction

For visualization of robot's predicted movement positions is provided service /robot_pose_prediction. Position are visualized using Markers.

Service takes 2 arguments:

  • geometry_msgs/Pose[] positions - predicted positions

  • ttl - markers lifetime

robot_pose_prediction.png

ROS API

Nodes

Node Name

Published Topics

Description

interaction_primitives_service_server

  • /interaction_primitives/primitive_name/update/pose_changed
  • /interaction_primitives/primitive_name/update/scale_changed
  • /interaction_primitives/primitive_name/update/menu_clicked
  • /interaction_primitives/primitive_name/update/movement_changed
  • /interaction_primitives/primitive_name/update/tag_changed

This node publishes interaction_primitives services.

Services

All services starts with prefix /interaction_primitives

Service Name

Input

Output

Description

/add_billboard

  • string frame_id
  • string name
  • string description
  • uint8 type
  • float64 velocity
  • geometry_msgs/Quaternion direction
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale

Adds billboard to the scene

/add_bounding_box

  • string frame_id
  • string name
  • string object_name
  • string description
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale
  • std_msgs/ColorRGBA color

Adds Bounding Box to the scene

/add_plane

  • string frame_id
  • string name
  • string description
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale
  • std_msgs/ColorRGBA color

Adds Plane to the scene

/add_plane_polygon

  • string frame_id
  • string name
  • string description
  • geometry_msgs/Polygon polygon
  • geometry_msgs/Vector3 normal
  • std_msgs/ColorRGBA color

Adds Plane Polygon to the scene

/add_object

  • string frame_id
  • string name
  • string description
  • geometry_msgs/Pose pose
  • geometry_msgs/Point bounding_box_lwh
  • std_msgs/ColorRGBA color
  • string resource
  • bool use_material
  • bool allow_pregrasp

Adds Object to the scene

/add_unknown_object

  • string frame_id
  • string name
  • string description
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale

Adds Unknown Object to the scene

/remove_primitive

  • string name

Removes primitive form the scene

/change_description

  • string name
  • string description

/change_pose

  • string name
  • geometry_msgs/Pose pose

Changes primitive's description

/change_scale

  • string name
  • geometry_msgs/Vector3 scale

Changes primitive's scale

/change_color

  • string name
  • std_msgs/ColorRGBA color

Changes primitive's color

/change_direction

  • string name
  • geometry_msgs/Quaternion direction

Changes billboard's movement direction

/change_velocity

  • string name
  • float velocity

Changes billboard's movement velocity

/set_pregrasp_position

  • string name
  • uint8 pos_id
  • geometry_msgs/Vector3 position

Sets pre-grasp position

/remove_pregrasp_position

  • string name
  • uint8 pos_id

Removes pre-grasp position

/get_update_topic

  • string name
  • uint8 type

Gets update topic for specified action (type)

/set_allow_object_interaction

  • string name
  • bool allow

Allows od denies interaction with specified Object

/get_billboard

  • string name

  • string frame_id
  • string name
  • string description
  • uint8 type
  • float64 velocity
  • geometry_msgs/Quaternion direction
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale

Gets billboard

/get_bounding_box

  • string name

  • string frame_id
  • string name
  • string object_name
  • string description
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale
  • std_msgs/ColorRGBA color

Ges Bounding Box

/get_plane

  • string name

  • string frame_id
  • string name
  • string description
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale
  • std_msgs/ColorRGBA color

Gets Plane

/get_object

  • string name

  • string frame_id
  • string name
  • string description
  • geometry_msgs/Pose pose
  • geometry_msgs/Point bounding_box_lwh
  • std_msgs/ColorRGBA color
  • string resource
  • bool use_material

Gets Object

/get_unknown_object

  • string name

  • string frame_id
  • string name
  • string description
  • uint8 pose_type
  • geometry_msgs/Pose pose
  • geometry_msgs/Vector3 scale

Gets Unknown Object

/clickable_positions

  • string frame_id
  • string topic_suffix
  • float64 radius
  • std_msgs/ColorRGBA color
  • geometry_msgs/Point[] positions

  • string topic

Shows clickable positions

/robot_pose_prediction

  • geometry_msgs/Pose[] positions
  • float64 ttl

Shows predicted positions of robot's movement

Messages

Msg Name

Content

Description

PrimitiveType

content

Types of Primitives

BillboardType

content

Types of Billboard

PoseType

content

Specifies if the coordinates are in the center or in the base of the primitive

MenuClicked

content

Message for update topic

MovementChanged

content

Message for update topic

PoseChanged

content

Message for update topic

ScaleChanged

content

Message for update topic

TagChanged

content

Message for update topic

PositionClicked

content

Message for position clicked topic

Published topics

All topics starts with prefix interaction_primitives.

Topic Name

Message

Description

/primitive_name/update/pose_changed

srs_interaction_primitives/PoseChanged

Publishes pose changes of the primitive

/primitive_name/update/scale_changed

srs_interaction_primitives/ScaleChanged

Publishes scale changes of the primitive

/primitive_name/update/menu_clicked

srs_interaction_primitives/MenuClicked

Publishes information about menu interaction of the primitive

/primitive_name/update/movement_changed

srs_interaction_primitives/MovementChanged

Publishes movement changes of the primitive (Billboard only)

/primitive_name/update/tag_changed

srs_interaction_primitives/TagChanged

Publishes tag changes of the primitive (Plane only)

/clickable_positions/topic_suffix

srs_interaction_primitives/PositionClicked

Publishes position which was clicked

Installation

Both components are in srs git in srs_interaction_primitives package and can be compiled with ROS standard tool rosmake

 rosmake srs_interaction_primitives

Usage

Adding primitives from C++

   1 // Create Interactive Marker Server 
   2 InteractiveMarkerServerPtr server;
   3 server.reset(new InteractiveMarkerServer("test_primitives", "", false));
   4 
   5 // Create billboard - /world is frame and my_billboard is primitives unique name
   6 Billboard *billboard = new Billboard(server, "/world", "my_billboard");
   7 billboard->setType(srs_interaction_primitives::BillboardType::PERSON); // type PERSON
   8 billboard->setPose(pose); // actual position of the billboard
   9 billboard->setPoseType(srs_interaction_primitives::PoseType::POSE_CENTER); // coordinates are int the center of the primitive
  10 billboard->setScale(scale); // scale of the billboard
  11 billboard->setDirection(direction); // actual movement direction
  12 billboard->setVelocity(velocity); // actual movement velocity
  13 billboard->setDescription("This is me!"); // description
  14 billboard->insert(); // creates interaction primitive a inserts it into IMS
  15 

Using predefined services

  • Run service server

rosrun srs_interaction_primitives interaction_primitives_service_server
  • Run RViz

rosrun rviz rviz
  • Add Interactive Marker Display and set Update Topic to /interaction_primitives/update.

Now you can call services and add or update Primitives.

Calling services from bash

Add Bounding Box

rosservice call /interaction_primitives/add_bounding_box '{frame_id: /world, name: bbox, object_name: obj, description: "", 0, pose: { position: { x: -1, y: 0, z: 0 }, orientation: { x: 0, y: 0, z: 0, w: 1 } }, scale: { x: 1, y: 1, z: 1 }, color: { r: 1, g: 0, b: 0 }}'

Add Billboard

rosservice call /interaction_primitives/add_billboard '{frame_id: /world, name: billboard, type: 2, description: "this is billboard", velocity: 5.6, direction: {x: 1, y: 1, z: 0, w: 1}, 0, pose: {position: { x: -1, y: 0, z: 0 }, orientation: { x: 0, y: 0, z: 0, w: 1 } }, scale: { x: 1, y: 1, z: 1 }}'

Add Plane

rosservice call /interaction_primitives/add_plane '{frame_id: /world, name: plane, description: "", 0, pose: { position: { x: -1, y: 0, z: 5 }, orientation: { x: 0, y: 1, z: 0, w: 1 } }, scale: { x: 1, y: 1, z: 1 }, color: { r: 1, g: 0.3, b: 0, a: 1.0 }}'

Add Object

rosservice call /interaction_primitives/add_object '{frame_id: /world, name: table_obj, description: "My table", 0, pose: { position: { x: -1, y: 0, z: 0 }, orientation: { x: 0, y: 0, z: 0, w: 1 } }, scale: { x: 1, y: 1, z: 1 }, resource: "package://gazebo_worlds/Media/models/table.dae", use_material: false, color: {r: 1, g: 0, b: 1, a: 1 }}'

Add Unknown Object

rosservice call /interaction_primitives/add_unknown_object '{frame_id: /world, name: uobj, description: "", 0, pose: { position: { x: -1, y: 0, z: 0 }, orientation: { x: 0, y: 0, z: 0, w: 1 } }, scale: { x: 1, y: 1, z: 1 } }' 

Add Object

rosservice call /interaction_primitives/add_object '{frame_id: /world, name: milk, description: "Detected milk", 0, pose: { position: { x: -1, y: 0, z: 0 }, orientation: { x: 0, y: 0, z: 0, w: 1 } }, bounding_box_lwh: {x: 1.0, y: 0.2, z: 0.1}, color: {r: 1, g: 1, b: 0}, resource: package://cob_gazebo_worlds/Media/models/milk_box.dae, use_material: True}'

Change pose

rosservice call /interaction_primitives/change_pose '{name: plane, pose: { position: { x: -1, y: 0, z: 0 }, orientation: { x: 0, y: 0, z: 0, w: 1 } }}'

Change scale

rosservice call /interaction_primitives/change_scale '{name: plane, scale: { x: 1.2, y: 2, z: 0.5 }}'

Change color

rosservice call /interaction_primitives/change_color '{name: plane, color: { r: 1, g: 0, b: 1, a: 0.5}}'

Change description

rosservice call /interaction_primitives/change_description '{name: plane, description: "something"}'

Set pre-grasp position

rosservice call /interaction_primitives/set_pregrasp_position '{name: object_name, pos_id: 1, position: {x: 0.2, y: 0.1, z: -0.5}}'

Remove pre-grasp position

rosservice call /interaction_primitives/remove_pregrasp_position '{name: object_name, pos_id: 1}'

Remove object

rosservice call /interaction_primitives/remove_object '{name: object_name}'

Calling services from C++

   1 #include <ros/ros.h>
   2 #include <srs_interaction_primitives/AddBoundingBox.h>
   3 
   4 int main(int argc, char **argv)
   5 {
   6   // ROS initialization (the last argument is the name of the node)
   7   ros::init(argc, argv, "interaction_primitives_client");
   8 
   9   // NodeHandle is the main access point to communications with the ROS system
  10   ros::NodeHandle n;
  11 
  12   // Create a client for the add_bounding_box service
  13   ros::ServiceClient bboxClient = n.serviceClient<srs_interaction_primitives::AddBoundingBox>("interaction_primitives/add_bounding_box");
  14 
  15   // Set parameters to new Bounding Box
  16   srs_interaction_primitives::AddBoundingBox bboxSrv;
  17   bboxSrv.request.name = "Bounding box";
  18   bboxSrv.request.object_name = "attached_object_name";
  19   bboxSrv.request.frame_id = "/world";
  20   bboxSrv.request.description = "This is Bounding Box";
  21   bboxSrv.request.pose_type = srs_interaction_primitives::PoseType::POSE_BASE;
  22   bboxSrv.request.pose.position.x = 1.0;
  23   bboxSrv.request.pose.position.y = 1.0;
  24   bboxSrv.request.pose.position.z = 1.0;
  25   bboxSrv.request.pose.orientation.x = 0.0;
  26   bboxSrv.request.pose.orientation.y = 0.0;
  27   bboxSrv.request.pose.orientation.z = 0.0;
  28   bboxSrv.request.pose.orientation.w = 1.0;
  29   bboxSrv.request.scale.x = 2.0;
  30   bboxSrv.request.scale.y = 3.0;
  31   bboxSrv.request.scale.z = 4.0;
  32   bboxSrv.request.color.r = 1.0;
  33   bboxSrv.request.color.g = 0.0;
  34   bboxSrv.request.color.b = 1.0;
  35   bboxSrv.request.color.a = 1.0;
  36 
  37   // Call service with specified parameters
  38   bboxClient.call(bboxSrv);
  39 
  40   ros::spinOnce(); // Call all the callbacks waiting to be called
  41 
  42   return 0;
  43 }

Wiki: srs_interaction_primitives (last edited 2012-09-25 11:29:32 by TomasLokaj)