(!) Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Operation on QNX

Description: Daily operation of Hironxo's Controller Box (that runs on QNX real-time OS)

Keywords: Hiro, NEXTAGE OPEN, QNX, controller

Tutorial Level: ADVANCED

Next Tutorial: rtmros_nextage/Tutorials/Changing Nextage Grippers in Robot Model

NOTE: All the tutorials available under the URL http://wiki.ros.org/rtmros_nextage/Tutorials are applicable to the multiple products of Kawada Industries; Hiro (only with the one that opensource software is installed) and NEXTAGE OPEN. To simplify the notation in the rest of the tutorials, we use HiroNXO to appoint the aforementioned robots.

Introduction

All of HiroNXO software is publicly available, which doesn't mean that you can just download and run it on your HiroNXO out of the box. You need to build/compile it on the robot's controller host ControllerBox that runs on QNX operating system. And to compile on QNX you need a developer license which you can purchase from the vendor of your region.

How to install necessary software on QNX is not disclosed either. Other than the developer's license issue, QNX is a commercial software that you're required/advised to be certified for operation, or at minimum should be familliar enough with it. Also due to the lack of robust packaging infrastructure (like ROS has) on QNX, installation work can involve a long list of manual tasks and thus is very error prone.

However, for the cases as below touching inside of QNX is required:

  • When you need to rollback to the previous closed-source GRX controller (to use the program you wrote for it).

  • See log files for debugging cases that aren't clear from logs available via API on Ubuntu software.

In these cases you can obtain the logon account info from either manufacturer or the software service provider.

In this page you'll find minimum things that you may find useful about setting on QNX on Hiro/NEXTAGE OPEN.

We assume that you can log on to QNX by:

YOURHOST$ ssh -l %QNX_YOUR_USER% %IPADDR_YOUR_QNX%

Daily Operation

NOTE: Need for a GUI tool to operate everyday task is noticed. Work is in progress.

Shutdown QNX

  • Using the tool officially provided by Kawada Robotics called NextageOpenSupervisor, you can do so from GUI. The tool's icon should be found on the Desktop when you're logged on to UIC with the nxouser account. No root password is needed.

  • Also if you prefer using commandline, do the following. Note that root password is required.
    QNX$ su -c '/opt/grx/bin/shutdown.sh'

Reboot QNX

The following command only works on NXO (ref).

QNX$ su -c '/bin/shutdown'

QNX Specific Troubleshooting

Before you move on to scroll down this page, you're advised to go through the troubleshooting for Hironx / NEXTAGE OPEN page to review general ways of shooting issues or getting support.

Although we hope you found solution/workaround there, sometimes you might need to look into logs deep in the robot. Here are some ways to troubleshoot issues on your Hiro/NEXTAGE OPEN that might be rooted in software on QNX.

Log Files

Type of Available Log Files

In /opt/jsk/var/log, you'll find a few types of log files:

  • Nameserver.log

    • Issues related to OpenRTM or CORBA are most likely written here.

  • Modelloader.log

  • rtcd.log

    • Issues concerning RT Components from hrpsys are recoreded.

Copy of each of above file is generated automatically in the same folder.

Operation with Logs

These log files above are generated automatically, and won't get removed automatically at the time of writing (3/18/2014). Also, currently there's no way to look into log files remotely other than logging in on to QNX (via ssh).

Need to retrieve logs for debug

Often while debugging you need controller's log that is only available in the log files on QNX. You can get them by either one of the following. Note that you need login info.

  • (RECOMMENDED) Run a script to get a zip ball of the log files (**NOTE** this feature is available from rtmros_hironx 1.1.25).

    # Simplest
    $ rosrun hironx_ros_bridge qnx_fetch_log.sh nextage qnx_nxo_user
    :
    $ ls
    opt_jsk_var_logs_20170602-020236.zip
    
    # Fetch only files generated after certain date. "1" can be anything except "archive"
    $ rosrun hironx_ros_bridge qnx_fetch_log.sh nextage qnx_nxo_user 1 2017-01-11
  • Alternatively, you can ssh into QNX and access the log file under the directory /opt/jsk/var/log.

Maintain logs regularly

Log files by themselves can easily take up gigabytes of the disk space. It is advised that you occasionally remove log files under /opt/jsk/var/log either manually or by custom script, since no mechanism to delete any log files. To do so, you can either use the same script qnx_fetch_log.sh or log on to QNX and remove manually.

  • # Remove all raw .log files to free disk space. Same .zip file will be kept in the log folder.
    $ rosrun hironx_ros_bridge qnx_fetch_log.sh nextage qnx_nxo_user archive

To Rollback to Closed-source Grx Controller

(This section will be filled per request basis. See Contribution section.)

Contribution

  • Hironx / NEXTAGE OPEN related QNX issues are currently handled in a private repository.

  • Post your thoughts on rtm-ros-robotics forum (this will be replaced by a designated issue tracker).

Hint to check server status

Since currently there's not much tools for you to remotely measure the status of processes inside of QNX on HiroNXO, you need to log in and check certain log files or processes. Below is the list of phenomenons each of which section describes what you need to check.

All 4 chest lamps on the robot are still blinking after powering it up

You want to see only white and green blink.

  • See /opt/jsk/var/log/rtcd.log. You want to see things like this:

    QNX$ more /opt/jsk/var/log/rtcd.log
    Logger::Logger: streambuf address = 0x805fc70
    hrpExecutionContext is registered
    pdgains.file_name: /opt/jsk/etc/HIRONX/hrprtc/PDgains.sav
    dof = 15
    open_iob - shmif instance at 0x80b3f58
    the number of gyros = 0
    the number of accelerometers = 0
    the number of force sensors = 0
    period = 5[ms], priority = 49
  • If you see something different from above, first check if /opt/jsk/var/log/Nameserver.log looks like:

    Sat Jan 24 10:55:33 2015:
    
    Starting omniNames for the first time.
    Wrote initial log file.
    Read log file successfully.
    Root context is IOR:010000002b00000049444c3a6f6d672e6f72672f436f734e616d696e672f4e616d696e67436f6e74
    6578744578743a312e300000010000000000000070000000010102000d0000003139322e3136382e312e313600009d3a0b00
    00004e616d6553657276696365000300000000000000080000000100000000545441010000001c0000000100000001000100
    0100000001000105090101000100000009010100035454410800000095fbc2540100001b
    Checkpointing Phase 1: Prepare.
    Checkpointing Phase 2: Commit.
    Checkpointing completed.
    
    Sat Jan 24 11:10:33 2015:
    
    Checkpointing Phase 1: Prepare.
    Checkpointing Phase 2: Commit.
    Checkpointing completed.
  • Then, check if /opt/jsk/var/log/ModelLoader.log says like:

    ready
    loading /opt/jsk/etc/HIRONX/model/main.wrl
    Humanoid node
    Joint nodeWAIST
      Segment node WAIST_Link
      Joint nodeCHEST_JOINT0
        Segment node CHEST_JOINT0_Link
        Joint nodeHEAD_JOINT0
          Segment node HEAD_JOINT0_Link
          Joint nodeHEAD_JOINT1
            Segment node HEAD_JOINT1_Link
            VisionSensorCAMERA_HEAD_R
            VisionSensorCAMERA_HEAD_L
        Joint nodeRARM_JOINT0
          Segment node RARM_JOINT0_Link
          Joint nodeRARM_JOINT1
            Segment node RARM_JOINT1_Link
            Joint nodeRARM_JOINT2
              Segment node RARM_JOINT2_Link
              Joint nodeRARM_JOINT3
                Segment node RARM_JOINT3_Link
                Joint nodeRARM_JOINT4
                  Segment node RARM_JOINT4_Link
                  Joint nodeRARM_JOINT5
                    Segment node RARM_JOINT5_Link
        Joint nodeLARM_JOINT0
          Segment node LARM_JOINT0_Link
          Joint nodeLARM_JOINT1
            Segment node LARM_JOINT1_Link
            Joint nodeLARM_JOINT2
              Segment node LARM_JOINT2_Link
              Joint nodeLARM_JOINT3
                Segment node LARM_JOINT3_Link
                Joint nodeLARM_JOINT4
                  Segment node LARM_JOINT4_Link
                  Joint nodeLARM_JOINT5
                    Segment node LARM_JOINT5_Link
    The model was successfully loaded !
  • If all of above looks ok, something complicated may be happening. ask for support.

Wiki: rtmros_nextage/Tutorials/Operation on QNX (last edited 2017-06-02 22:58:29 by IsaacSaito)