Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

How to Write Pr2 Props

Android Client

Creating the Package

If you roscd to the existing android_pr2_props package you'll see a lot of different files. some of those files will get generated for us. We can follow some of the same instructions from the previous tutorial. Go to your ROS_DIR (where you installed your ROS android toolchain). Create a directory called android_pr2_props2 and navigate to it.

mkdir android_pr2_props2
cd android_pr2_props2

We can use the android_create tool to make some of the autogenerated files. The parameters are fairly complex. For a full description see the previous tutorial. Here is an example of what we'll put for our Props App.

rosrun appmanandroid android_create --create Pr2Props2 ros.android.pr2props2 Pr2Props2 icon "Pr2 Props2" pr2_props2 pr2_props2_app/pr2_props2

After running the script, you should see a bunch of new files.

To create our icon file, we can just borrow the ROS icon for now. Or you can use your own.

cp `rospack find android_gingerbread`/res/drawable-hdpi/icon.png res/drawable/icon.png

Then add the application to your tool chain install:

rosinstall ROS_DIR .
source ROS_DIR/setup.bash

If this produces any errors, just read the error message and it should indicate how to fix it. Remember to tailor the solution to your version of ROS.

Next build the application.

rosmake --threads=1

Filling in the Activity Class

Now we can actually start to make the application do something. Props is only a single Android activity. We can find the source for the activity if we roscd to the package and go to src/ros/android/pr2props/Pr2Props2.java.

roscd android_pr2_props2
cd src/ros/android/pr2props
vi Pr2Props2.java

Once you've opened up the activity file in the editor of your choice, you can see that it's mostly empty. Let's start to fill that in.

If we take a look at the onCreate() method, we'll notice that most initialization has been done for us. We can add a line to set the robot's height to 0.0 when it starts up. We'll make it a global variable and declare it at the beginning of our activity.

   1 public class Pr2Props extends RosAppActivity {
   2 
   3   private double spineHeight;
   4 
   5   /** Called when the activity is first created. */
   6   @Override
   7   public void onCreate(Bundle savedInstanceState) {
   8     setDefaultAppName("pr2_props_app/pr2_props");
   9     setDashboardResource(R.id.top_bar);
  10     setMainWindowResource(R.layout.main);
  11     spineHeight = 0.0;
  12     super.onCreate(savedInstanceState);
  13   }

Next we can look at onNodeCreate(), which has everything in it that happens when your node is created. Here you want to create your publisher to publish messages to the spine of the robot and move the torso up/down. If you are familiar with the Props App you probably know that there's more to it than just moving the torso of the robot. The other parts are controlled by services, so they'll get taken care of later. This also creates an interesting behavior where if for some reason the .launch file doesn't get launched on the robot, the torso can still move up and down because it's just relying on messages published to a topic.

Your onNodeCreate() might look something like this now:

   1   @Override
   2   protected void onNodeCreate(Node node) {
   3     super.onNodeCreate(node);
   4     spinePub = node.newPublisher("torso_controller/command", "trajectory_msgs/JointTrajectory");
   5     spineThread = new Thread(new Runnable() {
   6         @Override
   7         public void run() {
   8           JointTrajectory spineMessage = new JointTrajectory();
   9           spineMessage.points = new ArrayList<JointTrajectoryPoint>();
  10           spineMessage.joint_names = new ArrayList<String>();
  11           spineMessage.joint_names.add("torso_lift_joint");
  12           JointTrajectoryPoint p = new JointTrajectoryPoint();
  13           p.positions = new double[] { 0.0 };
  14           p.velocities = new double[] { 0.1 };
  15           p.time_from_start = new Duration(0.25);
  16           spineMessage.points.add(p);
  17           try {
  18             while (true) {
  19               spineMessage.points.get(0).positions[0] = spineHeight;
  20               spinePub.publish(spineMessage);
  21               Thread.sleep(200L);
  22             }
  23           } catch (InterruptedException e) {
  24           }
  25         }
  26       });
  27     spineThread.start();
  28   }

Basically this just creates a publisher to publish messages of the type trajectory_msgs/JointTrajectory to the torso_controller/command topic. As you can see, the actual construction and publishing of the messages is done in a separate thread.

Make sure to declare any undeclared global variables. Your global variables list should look like this:

   1   private double spineHeight;
   2   private Thread spineThread;
   3   private Publisher spinePub;

onNodeDestroy() is what happens when the node gets shutdown. You want to shutdown your spineThread and your publisher, spinePub:

   1  @Override
   2   protected void onNodeDestroy(Node node) {
   3     super.onNodeDestroy(node);
   4     final Thread thread = spineThread;
   5     if (thread != null) {
   6       spineThread.interrupt();
   7     }
   8     spineThread = null;
   9     final Publisher pub = spinePub;
  10     if (pub != null) {
  11       pub.shutdown();
  12     }
  13     spinePub = null;
  14   }

At the bottom you'll see code for handling the options menu. This can be left as is, or you can change it as necessary.

Now it's time to deal with the services. How will we make the robot give high fives? What we do is make a method called runService which does all the work of actually running the services. It takes a string for the service name and then creates a service client to send messages to the service node on the robot-side. In this case it actually just sends empty messages and gets empty responses. Our runService() method might look something like this:

   1   private void runService(String service) {
   2     Log.i("Pr2Props2", "Run: " + service);
   3     try {
   4       ServiceClient<Empty.Request, Empty.Response> appServiceClient =
   5         getNode().newServiceClient(service, "std_srvs/Empty");
   6       Empty.Request appRequest = new Empty.Request();
   7       appServiceClient.call(appRequest, new ServiceResponseListener<Empty.Response>() {
   8           @Override public void onSuccess(Empty.Response message) {
   9           }
  10 
  11           @Override public void onFailure(RemoteException e) {
  12             //TODO: SHOULD ERROR
  13             Log.e("Pr2Props2", e.toString());
  14           }
  15         });
  16     } catch (Exception e) {
  17       //TODO: should error
  18       Log.e("Pr2Props2", e.toString());
  19     }
  20   }

So that was how the client is going to make the requests, but where do the requests get made? In the Props App, the user hits a button to trigger each of the actions high five left, props right, raising the torso, etc. We'll briefly see what the buttons look like in the layout xml later but right now we can just make a bunch of callbacks to use runService() when buttons are clicked. For example:

   1   public void highFiveLeft(View view) {
   2     runService("/pr2_props/high_five_left");
   3   }
   4   public void highFiveRight(View view) {
   5     runService("/pr2_props/high_five_right");
   6   }
   7   public void highFiveDouble(View view) {
   8     runService("/pr2_props/high_five_double");
   9   }
  10   public void lowFiveLeft(View view) {
  11     runService("/pr2_props/low_five_left");
  12   }
  13   public void lowFiveRight(View view) {
  14     runService("/pr2_props/low_five_right");
  15   }
  16   public void poundLeft(View view) {
  17     runService("/pr2_props/pound_left");
  18   }
  19   public void poundRight(View view) {
  20     runService("/pr2_props/low_five_right");
  21   }
  22   public void poundDouble(View view) {
  23     runService("/pr2_props/pound_double");
  24   }
  25   public void hug(View view) {
  26     runService("/pr2_props/hug");
  27   }
  28   public void raiseSpine(View view) {
  29     spineHeight = 0.31;
  30   }
  31   public void lowerSpine(View view) {
  32     spineHeight = 0.0;
  33   }

The last two methods there aren't actually service calls. Those are just setting the spine height. The spine publisher thread will pick up on that and publish the corresponding messages.

Finally you should make sure you're importing all the right things. If you try to build it, you'll probably find out what's missing. Just in case though, these are the import statements from the original Props App and we can just steal those:

   1 import org.ros.exception.RemoteException;
   2 import ros.android.activity.AppManager;
   3 import ros.android.activity.RosAppActivity;
   4 import android.os.Bundle;
   5 import org.ros.node.Node;
   6 import android.view.Window;
   7 import android.view.WindowManager;
   8 import android.util.Log;
   9 import org.ros.node.service.ServiceClient;
  10 import org.ros.node.topic.Publisher;
  11 import org.ros.service.app_manager.StartApp;
  12 import org.ros.node.service.ServiceResponseListener;
  13 import android.widget.Toast;
  14 import android.view.Menu;
  15 import android.view.View;
  16 import android.view.MenuInflater;
  17 import android.view.MenuItem;
  18 import android.widget.LinearLayout;
  19 import org.ros.service.std_srvs.Empty;
  20 import org.ros.message.trajectory_msgs.JointTrajectory;
  21 import org.ros.message.trajectory_msgs.JointTrajectoryPoint;
  22 import java.util.ArrayList;
  23 import org.ros.message.Duration;

Congratulations! That's pretty much all the java code you'll have to write for this app! If you want, you can skip down to the part where we write the corresponding robot-side code and give it a quick read before finishing up the Android side. It might make more sense if done that way.

Layout XML

If you navigate to the root of your package and then to res/layout, there will be the layout xml in main.xml. If you're familiar with Android layouts then this should be easy and you can definitely skip this part of the tutorial.

There are a lot of different ways to accomplish the same thing with layouts, so the exact implementation can be somewhat arbitrary. Let's consider what we want to accomplish. We want to have a button for each action that we defined in our activity. It's a lot of buttons. We should probably group the similar buttons together under headings (all the high fives together, all the props together, changing torso height, etc). We should make the view scroll in case the app will be run on a device where not all the content fits on the screen at once.

To do that we can use a LinearLayout as the main layout. Then, we should have another LinearLayout inside that one, which will be the top_bar and have the dashboard components that you might have seen in the ROS Android apps. This shows basic status information about the robot (battery, run-stop status). The top_bar layout does not contain any other layouts. Our main layout should contain a ScrollView. The ScrollView's child can be another LinearLayout containing the buttons grouped under TextViews for headings. You can implement this yourself. The only trick with the buttons is that you have to make sure to define the names of the OnClick methods for your buttons in the xml since we didn't declare them in the activity. If you have any trouble you can take a look at the actual Props implementation at:

roscd android_pr2_props
vi res/layout/main.xml

If you write a couple of button descriptions and decide that it's too much typing for the moment you can actually just copy the main.xml from the original Props App. We don't need to change it.

cp `rospack find android_pr2_props`/res/layout/main.xml res/layout

Update Manifest

We need to make some minor changes to our manifest.xml as well. It can be found in the root of our package. Right now we depend only on the appmanandroid library. With the latest version of rosjava we should also explicitly depend on std_msgs and trajectory_msgs. Just add these two lines after the other package dependency:

   1   <depend package="std_msgs"/>
   2   <depend package="trajectory_msgs"/>

And that's it. Since we changed the manifest.xml we have to rosmake it again. If you make changes that do not change the manifest.xml you can use ant instead.

rosmake --threads=1

If you want to iteratively make changes to the app but you're not changing the manifest.xml then the best way to do it is to use ant. The following commands will build your project and clean it.

ant
ant clean

If you've made changes then it's important to clean the project before installing the app on a device because depending on which files have changed, ant might not recreate the class files appropriately. To install it on your device you can use:

ant debug install

If this fails then it may be because your computer does not recognize your Android device. That is outside the scope of this tutorial but you are encouraged to Google vigorously.

Robot-side Application

Writing the Python Script

The robot-side of the application must have a stack. In this case we're not going to release a whole new stack for our application. Since we're just testing we'll log (ssh) into the robot as the user 'applications' and put our stack under the ROS install directory. If this is a PR2 then it may be a directory called 'ros' in the applications user home directory:

cd ros
mkdir pr2_props2_app

The main part of the app here is a python script. Inside pr2_props2_app we can make a directory called 'scripts' and inside make a file called prop_runner with the editor of your choice.

cd pr2_props2_app
mkdir scripts
cd scripts
vi prop_runner

The start of our python script will look like this:

   1 #!/usr/bin/env python
   2 import roslib; roslib.load_manifest('pr2_props2_app')
   3 
   4 import rospy
   5 import os
   6 from std_srvs.srv import *

We're importing and using roslib only for bootstrapping reasons. It is appropriate to use rospy for most of your ROS Python needs. We then import rospy and os. We import os because we're actually going to use os.system to rosrun scripts to do the positioning for us. All we're going to do is queue a bunch of requests to run scripts and then send them to the system to run. We import everything from std_srvs.srv to allow us to respond to service requests. Also, if you don't usually write anything in Python, remember Python is whitespace delimited!

Let's start by making our QueueItem class. This is just an object to hold on to the command we're going to have the system execute and let us know when it's done. Let's also create an item and put it into a queue.

   1 class QueueItem:
   2     def __init__(self, txt):
   3         self.is_done = False
   4         self.txt = txt
   5 
   6 goal_queue = [QueueItem("pwd")]

Next as a way for things to get added to our queue, we'll make run_command:

   1 def run_command(cmd):
   2     item = QueueItem(cmd)
   3     goal_queue.append(item)
   4     while (item.is_done == False):
   5         rospy.sleep(0.01)

run_command will add create QueueItems out of the commands you want to run and add them to the queue. Now we have to actually figure out the commands that we want run.

If you navigate to /opt/ros/electric/stacks/pr2_props_stack/pr2_props/src you'll see a couple of .cpp files that basically just run a few different actions. That's what we want to run when the user presses a button in the app. We can use the rosrun command for that.

Let's make methods for each of the buttons in the app that can be pushed (except for the buttons to raise and lower the torso, those buttons have no robot-side code because they publish messages straight to topics that the spine subscribes to). Each method should queue a command to rosrun the appropriate script out of the ones we saw earlier:

   1 def high_five_double(msg):
   2     run_command("rosrun pr2_props high_five double")
   3     return EmptyResponse()
   4 
   5 def high_five_left(msg):
   6     run_command("rosrun pr2_props high_five left")
   7     return EmptyResponse()
   8 
   9 def high_five_right(msg):
  10     run_command("rosrun pr2_props high_five right")
  11     return EmptyResponse()
  12 
  13 def low_five_left(msg):
  14     run_command("rosrun pr2_props low_five left")
  15     return EmptyResponse()
  16 
  17 def low_five_right(msg):
  18     run_command("rosrun pr2_props low_five right")
  19     return EmptyResponse()
  20 
  21 def pound_double(msg):
  22     run_command("rosrun pr2_props pound double")
  23     return EmptyResponse()
  24 
  25 def pound_left(msg):
  26     run_command("rosrun pr2_props pound left")
  27     return EmptyResponse()
  28 
  29 def pound_right(msg):
  30     run_command("rosrun pr2_props pound right")
  31     return EmptyResponse()
  32 
  33 def hug(msg):
  34     run_command("rosrun pr2_props hug")
  35     return EmptyResponse()

You'll notice that they take in a msg. These methods are the handlers that rospy.Service() will make callbacks to in the main function. rospy.Service() recieves messages from the service client we created on the Android side.

Now it's time to use this stuff in the main function. We should create a node, which we can call 'pr2_props2_app' and then make the callbacks to our handlers that we just wrote.

   1 if __name__ == "__main__":
   2     rospy.init_node("pr2_props2_app")
   3     s1 = rospy.Service('pr2_props/high_five_double', Empty, high_five_double)
   4     s2 = rospy.Service('pr2_props/high_five_left', Empty, high_five_left)
   5     s3 = rospy.Service('pr2_props/high_five_right', Empty, high_five_right)
   6     s4 = rospy.Service('pr2_props/low_five_right', Empty, low_five_right)
   7     s5 = rospy.Service('pr2_props/low_five_left', Empty, low_five_left)
   8     s6 = rospy.Service('pr2_props/pound_double', Empty, pound_double)
   9     s7 = rospy.Service('pr2_props/pound_left', Empty, pound_left)
  10     s8 = rospy.Service('pr2_props/pound_right', Empty, pound_right)
  11     s9 = rospy.Service('pr2_props/hug', Empty, hug)

You'll notice that the on the Android side we sent empty messages and here we're returning empty responses. This is because it's not necessary to get any extra content from the message. The fact that it was sent/executed is sufficient. The last thing we need is to actually read from the queue we created and have the system run those 'rosrun pr2_props ...' commands. We can make a loop to go through the queue while the node is not shutdown, execute the goals, and remove them from the queue.

   1     while not rospy.is_shutdown():
   2         while (len(goal_queue) > 0):
   3             goal = goal_queue[0]
   4             goal_queue = goal_queue[1:]
   5             os.system(goal.txt)
   6             goal.is_done = True
   7         rospy.sleep(0.01)

This will create the behavior that when you press buttons more quickly than the actions can execute, your requests get queued. So if you were to press 'hug' ten times then the robot would sit there for about 3 minutes and give all ten hugs in sequence.

That's really it for the python code even though we cheated and used those .cpp scripts. For the next few sections we'll be basically following the steps from this tutorial: ApplicationsPlatform/CreatingAnApp

Launch File

Next we need to write a launch file for the application. We will place it in a directory called 'launch' and call it 'pr2_props_app.launch'.

roscd pr2_props2_app
mkdir launch
vi pr2_props_app.launch

The launch file will help launch the correct nodes when the application is started. The ROS Application Chooser (which you will want to download from the Android Market if you are going to be running any ROS Android applications on an Android device) will use these launch files when you start apps from insider the App Chooser. If you do not start your ROS app from inside the app chooser then you will have roslaunch the launch file yourself from your computer. For information on the format of the launch files see roslaunch/XML. Your launch file should look something like this:

   1 <launch>
   2     <include file="$(find pr2_props)/launch/pr2_props.launch" />
   3     <node pkg="pr2_props2_app" type="prop_runner" name="pr2_props2_app" />
   4     <node pkg="pr2_position_scripts" type="head_up.py" name="head_up" />
   5 </launch>

What we're doing is including another .launch file. It's actually in pr2_props_stack. You can roscd to pr2_props_stack and you'll see the package that we're searching for, pr2_props. Inside is the launch file we want to include. Then we also create a node for what's running in our Python script and also for the position scripts that make the robot face forward when we start up the app.

Interface File

The interface file is a file that is essentially blank for now. In the future it will be more important. It should look like this and be named 'pr2_props_app.interface' and it should also be in the root of the package:

published_topics: {}
subscribed_topics: {}

Icon

For now we're actually just going to steal the icon from the original Props app. You should definitely get your own eventually when you make real apps, but that's up to you. Go to the root to the package and type:

cp `rospack find pr2_props_app`/pr2props.jpg pr2props2.jpg

App File

The .app file is what the app manager uses to find out about your application. Ours will look like this:

display: Props2
description: Run PR2 Props
platform: pr2
launch: pr2_props2_app/pr2_props2_app.launch
interface: pr2_props2_app/pr2_props2_app.interface
icon: pr2_props2_app/pr2props2.jpg
clients:
 - type: android
   manager:
     api-level: 9
     intent-action: ros.android.pr2props2.Pr2Props2
   app:
     gravityMode: 0
     camera_topic: /wide_stereo/left/image_color/compressed_throttle

Most of that is pretty straightforward. One thing to be aware of is that the path names are all ROS path names so they just have the package_name/file_name no matter how many directories down in the package the file is.

Installing App

We have to add your package/unary stack to the .rosinstall file. Make sure you're in the ROS install directory. Then add the following line to the .rosinstall file:

- other: {local-name: pr2_props2_app}

After you save and close:

rosinstall .

Now add:

echo "Sourcing /u/applications/ros/setup.bash"
. /u/applications/ros/setup.bash

to the .bashrc in the home directory of the applications user.

Now we have to make a .installed file. Go to the local_apps directory (should be located under the home directory). We will name the file pr2_props2_app.installed and it will contain the following:

apps:
  - app: pr2_props2_app/pr2_props2_app
    display: Pr2 Props2 App

It's actually pointing to the .app file. This is hard to tell since we named everything 'pr2_props2_app'. But we don't include the .app extension because it gets automatically added.

Loose Ends: Makefile, stack.xml, manifest.xml, etc

There are a few more things to take care of before we can actually run the app. Because we just created this stack now, it's missing some important things that it should have.

We need a Makefile and CMakeLists.txt. If you want some background information on making those files you can look here rospy_tutorials/Tutorials/Makefile. We could have used roscreate-pkg to create our package at the start and that would have generated a template of these files for us. However since they're each only a few lines long we can make them ourselves this time.

First let's roscd to our package. Our Makefile only has to be one line:

include $(shell rospack find mk)/cmake_stack.mk

And our CMakeLists.txt looks like this inside:

cmake_minimum_required(VERSION 2.4.6)
include($ENV{ROS_ROOT}/core/rosbuild/rosbuild.cmake)
rosbuild_make_distribution(0.1.0)

Alternatively you can also just copy the same files from the original Props:

cp `rospack find pr2_props_app`/Makefile Makefile
cp `rospack find pr2_props_app`/CMakeLists.txt CMakeLists.txt

We also need to make a manifest.xml. It's pretty standard in terms of dependencies and should look like this:

   1 <package>
   2   <description brief="PR2 Props2 App">
   3     Application files for running PR2 props
   4   </description>
   5   <author>You</author>
   6   <license>BSD</license>
   7   <url>http://ros.org/wiki/pr2_props</url>
   8   <review status="na" notes="" />
   9   <depend package="roslib" />
  10   <depend package="rospy" />
  11   <depend package="pr2_props" />
  12   <depend package="pr2_position_scripts" />
  13   <depend package="std_srvs" />
  14   <platform os="ubuntu" version="9.04"/>
  15   <platform os="ubuntu" version="9.10"/>
  16   <platform os="ubuntu" version="10.04"/>
  17 </package>

We also need a stack description in the form of the stack.xml:

<stack>
  <description brief="pr2_props2_app">pr2_props_app</description>
  <author>Maintained by Applications Manager</author>
  <license>BSD</license>
  <review status="unreviewed" notes=""/>
  <url>http://ros.org/wiki/pr2_props_app</url>
  <depend stack="pr2_apps" /> <!-- pr2_position_scripts -->
  <depend stack="pr2_props_stack" /> <!-- pr2_props -->
  <depend stack="ros" /> <!-- roslib -->
  <depend stack="ros_comm" /> <!-- std_srvs, rospy -->

</stack>

Now we're done. Almost. We need to put a ROS_NOBUILD file in the root of the package/unary stack so that rosmake skips it. This file does not have any real content. We can just copy it from the original Props stack.

cp `rospack find pr2_props_app`/ROS_NOBUILD ROS_NOBUILD

Done! Deactivate and restart your robot (from the ROS Application Chooser you can push the "Deactivate" button). Once you reconnect, you should see your application listed in the Application Chooser.

If you see no applications listed, this means that your application's formatting is invalid, and it has caused errors. If you do not see your application listed at all, this means that you have skipped a step or failed to restart the app manager.

If there is an error, deactivate your robot, and find the latest log in the ~/.ros directory of the applications user. The *app_manager* files should tell you a bit about what happened.

If you see your application, click it to start it. You should see the application highlight and see your ROS nodes running, just as if you launched the roslaunch file manually.

You should run your applications through the Application Chooser because it will roslaunch the appropriate nodes for you. If you do not go through the App Chooser and instead just try to run the application by itself, you will have the manually roslaunch the .launch file for your application, probably from your computer.