Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Android Interactions

||<style="border: 1px solid black; background-color:#ccccff; text-align: center; vertical-align:top;" tablewidth="100%">In the example here we connect an android listener to an already running talker on the robot. ||

About

Android applications are so much easier for users (and even developers) to work with their robot than a pc and an ssh tunnel. Nonetheless, you quickly run into limitations, which we solve here via an interactions framework.

Unless you're the robot dev, you have to ask, or find some documentation somewhere, or just try and fail till you work it out. Here we provide one application, the android remocon¹ which will let you connect you to any interactions enabled ros robot and provide you with the answers to this question as well as a launchpad for downloading and starting other applications. A RosAppActivity (subclassed from RosActivity) class is provided with all the infrastructure required to work with the remocon.

An interactions enabled ros robot specifically means that the robot dev (the expert that you would have had to ask anyway) has prepared the robot with a node that stores the required information and receives requests from remocons for that information. This node has minimal runtime cost and is very easy to configure from yaml.

RosActivity applications get locked into specific topics in standard locations. They can't be remapped to easily work on another robot. While there is some effort via REP to standardise some locations, this applies to only a few select topics and in general, takes a huge amount of effort. There are always exceptions as well.

An android interaction configured in yaml for the interactions node on the robot passes remapping information to the remocon, who in turn passes it to a RosAppActivity which has a simple api to make use of this once the remocon has started the activity.

Roslaunched processes have a convenient way of reconfiguring the parameters a node uses for that particular roslaunch. RosActivity based applications do not have this capability. A RosAppActivity based application launched from the android remocon can. The information comes from the interaction specification provided by the robot in the same way as the remappings. The android remocon then passes this information to the activity upon launching.

Quick Demo

The validation step in the source installation shows how to run a quick demo to illustrate the interactions concept running a pair of talker-listener interactions between pc and smart phone.

Resources

Robot Side

• rocon_interactions - this package provides the robot side support and an overview.

• Interactions Tutorials - the pc-qt examples are a good intro to the interactions framework.

• Android Interactions Tutorial - specific tutorial example for android.

• Demo Yaml Configurations - example interaction specifications in yaml.

• Demo Interaction Launchers - these start the robot side interactions machinery and load configurations.

• Turtlebot Yaml Configurations - turtlebot interactions.

Android

• RosAppActivity - subclass this for a remocon launchable android application.

• Listener and Talker examples.

• Advanced examples - make a map, map_nav, teleop, etc.

Step by Step

Setting up an interaction requires some work on both sides - the robot and the android application.

Robot Side

• Install the following applications on the robot:

> sudo apt-get install ros-indigo-rocon-interactions ros-indigo-rocon-master-info ros-indigo-zeroconf-avahi-suite

• Create an interactions yaml file.
  • Specify activity name, display name, remappings, ... for each interaction.
• A launch file that starts the required interaction nodes and loads the interactions yaml.
• Roslaunch this file on the robot.

Plenty of examples for both yaml and roslaunch files in the resources section above.

Android Side

• Download rocon_remocon to your phone.

  • From google apps play store ²

  • OR Compile from source.

• Create an android application that subclasses RosAppActivity.

  • Set up any remapping handles inside the application.
• Download the application to the phone.
  • Alternatively upload it to the play store - the remocon can get it from there.
• Run the remocon
  • Many ways to add a master - ip, nfc, scan, qrcode (typically scanning the local network is easiest).

The above snapshots illustrate the steps through the remocon - selecting the master, viewing the list of permitted applications and finally the launched application. This is the only application that is required to start interacting with an interactions enabled robot.

As noted above, building the application simply requires subclassing the RosAppActivity class. This class follows very much the same usage as the RosActivity class with the addition of an api that handles remapping and a dashboard. Most of the functionality under the hood to get it to work with a remocon is hidden from the developer.

Plenty of examples of android applications above in the resources section above.

Example

This breaks down some of the files you would require for a very simple android talker-listener interaction. Much of this is exactly what is used in the Quick Demo.

Configuration

An interaction which would start the Listener reference android application and connect it to the robot on the /babbler topic.

- name: com.github.rosjava.android_remocons.listener.Listener
  role: Android
  compatibility: rocon:/*/*/indigo/jellybean|ice_cream_sandwich
  display_name: Listener
  description: Tunes into the babble (/babbler topic).
  max: -1
  remappings:
    - remap_from: chatter
      remap_to: /babbler

• role : this can be any text string and is just an informative way of grouping interactions for the user.

• rocon_uri : is a platform requirement specification, by setting this you can exclude the interaction from appearing in the remocon if the platform requirements are not met.

• display_name : you can actually list multiple listener interactions that remap to different topics, in this case the display_name is a unique differentiator.

• max : limit the number of connections, e.g. you wouldn't want more than one teleop interaction at a time.

• remappings : just like regular roslaunch style remappings.

Launcher

A launcher that fires up the interactions machinery, loads this configuration file (assuming it can be found in foo/android.interactions and also fires up a roscpp talker.

   1 <launch>
   2   <!-- ****************************** Arguments ****************************** -->
   3   <arg name="ros_master_name" default=" Android Interactions"/>
   4   <arg name="ros_master_description" default="An android interactions tutorial."/>
   5   <arg name="ros_master_icon" default="rocon_icons/cybernetic_pirate.png"/>
   6 
   7   <!-- ****************************** Zeroconf ******************************* -->
   8   <node ns="zeroconf" pkg="zeroconf_avahi" type="zeroconf" name="zeroconf">
   9     <rosparam param="services" subst_value="true">
  10         [ { name: "$(arg ros_master_name)", type: _ros-master._tcp, port: 11311, domain: local, description: "$(arg ros_master_description)" } ]
  11     </rosparam>
  12   </node>
  13   <!-- ***************************** Master Info ****************************** -->
  14   <param name="name" value="$(arg ros_master_name)"/>
  15   <param name="description" value="$(arg ros_master_description)"/>
  16   <param name="icon" value="$(arg ros_master_icon)"/>
  17   <node pkg="rocon_master_info" type="master.py" name="master"/>
  18 
  19   <!-- ************************* Interactions Manager ************************* -->
  20   <node pkg="rocon_interactions" type="interactions_manager.py" name="interactions">
  21     <rosparam param="interactions">[foo/android]</rosparam>
  22   </node>
  23 
  24   <!-- ******************************* Programs ******************************* -->
  25   <node pkg="roscpp_tutorials" type="talker" name="pc_talker">
  26     <remap from="chatter" to="babbler"/>
  27   </node>
  28 </launch>

• ros_master_name : just an informative string for the user when displaying the master in the remocon.

• ros_master_description : as above

• ros_master_icon : as above

• zeroconf : broadcasts the master so you can pick it up when you scan the local network from the remocon.

• foo/android : tells the manager to load android.interactions from somewhere inside package foo.

• talker : you wouldn't ordinarily have this in the interactions launcher...typically it would be already running.

Android App

This is a breakdown of the important parts of the official reference listener android interaction application:

Listener.java

public class Listener extends RosAppActivity
{
    private RosTextView<std_msgs.String> rosTextView;

Subclassing RosAppActivity for remocon/remapping functionality and utilising a RosTextView. RosTextView is an android_core class that hooks up to a std_msgs/String topic and prints incoming string messages to a text widget.

    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        setDefaultMasterName(getString(R.string.default_robot));
        setDashboardResource(R.id.top_bar);
        setMainWindowResource(R.layout.main);
        super.onCreate(savedInstanceState);
    }

This function implements the startup part of an ordinary android application lifecycle.

The top_bar is a linear layout embedded into the main layout we pass to the RosAppActivity for filling in with the default robot dashboard. At present the dashboard is very simple - just robot battery and charging status, both of which retrieve information from diagnostics coming from the robot. This feature may be dropped in the future.

    @Override
    protected void init(NodeMainExecutor nodeMainExecutor)
    {

This function is called once the remocon has launched the activity and it has successfully connected to the master. Typically you will do your topic creation inside here.

        String chatterTopic = remaps.get(getString(R.string.chatter_topic));

Catch remapping information.

        super.init(nodeMainExecutor);

Let the parent RosActivity do its thing.

        rosTextView = (RosTextView<std_msgs.String>) findViewById(R.id.text);
        rosTextView.setTopicName(getMasterNameSpace().resolve(chatterTopic).toString());
        rosTextView.setMessageType(std_msgs.String._TYPE);
        rosTextView.setMessageToStringCallable(new MessageCallable<String, std_msgs.String>() {
            @Override
            public java.lang.String call(std_msgs.String message) {
                Log.e("Listener", "received a message [" + message.getData() + "]");
                return message.getData();
            }
        });

Setting up the listener callback in the RosTextView.

        try {
            java.net.Socket socket = new java.net.Socket(getMasterUri().getHost(), getMasterUri().getPort());
            java.net.InetAddress local_network_address = socket.getLocalAddress();
            socket.close();
            NodeConfiguration nodeConfiguration =
                    NodeConfiguration.newPublic(local_network_address.getHostAddress(), getMasterUri());

The socket based test connection is there to ensure that the right network interface is used to connect with the ros master (quite often phones simultaneously run multiple network interfaces these days).

            nodeMainExecutor.execute(rosTextView, nodeConfiguration);

Hook up the subscriber.

    @Override
    public boolean onCreateOptionsMenu(Menu menu)
    {
        menu.add(0,0,0,R.string.stop_app);
        return super.onCreateOptionsMenu(menu);
    }

Note essential, but it is nice to have a custom options menu (the left button the bottom of a typical android phone).

    @Override
    public boolean onOptionsItemSelected(MenuItem item)
    {
        super.onOptionsItemSelected(item);
        switch (item.getItemId()){
            case 0:
                finish();
                break;
        }
        return true;
    }

Callback for the custom options menu. In this case it calls RosActivity's finish() which will handle any and all ros cleanup and close the activity.

AndroidManifest.xml

Other important activity configurations go in here. Pay particular attention to the intent-filters we use and the NodeMainExecutorService specification.

   1 <?xml version="1.0" encoding="utf-8"?>
   2 <manifest xmlns:android="http://schemas.android.com/apk/res/android"
   3       package="com.github.rosjava.android_remocons.listener"
   4       xmlns:tools="http://schemas.android.com/tools">
   5 
   6     <uses-permission android:name="android.permission.INTERNET" />
   7     <uses-permission android:name="android.permission.WAKE_LOCK" />
   8 
   9     <application android:label="@string/app_name"
  10                  android:icon="@mipmap/icon"
  11                  tools:replace="icon, label">
  12         <activity android:name="Listener"
  13                   android:screenOrientation="portrait"
  14                   android:label="@string/app_name">
  15             <intent-filter>
  16                 <action android:name="android.intent.action.MAIN" />
  17                 <category android:name="android.intent.category.LAUNCHER" />
  18             </intent-filter>
  19             <intent-filter>
  20                 <action android:name="com.github.rosjava.android_remocons.listener.Listener" />
  21                 <category android:name="android.intent.category.DEFAULT" />
  22             </intent-filter>
  23         </activity>
  24 
  25         <activity android:name="org.ros.android.MasterChooser" />
  26 
  27         <service android:name="org.ros.android.NodeMainExecutorService">
  28             <intent-filter>
  29                 <action android:name="org.ros.android.NodeMainExecutorService" />
  30             </intent-filter>
  31         </service>
  32     </application>
  33 </manifest> 

The Android Remocon, Web URLs and Web Apps

The android remocon can also be used to start web url's (useful for helping your user immediately find documentation) and web apps. See the rocon_interactions tutorials for more information and android_demo.launch, web.interactions and web_apps.interactions for specific examples.

Footnotes