= Python unit testing =
<<TOC(3)>> <<Buildsystem()>>

== Disclaimer ==
(11/28/2013): Some parts of this page may be outdated, especially to the standard introduced after [[catkin]]. Until this page gets fully updated (so that this whole disclaimer section is removed by the author who completes the update), be careful to also read some other documents including:

 * http://docs.ros.org/hydro/api/catkin/html/howto/index.html

 * http://answers.ros.org/question/94526/any-good-examples-of-ros-unittests/

== Introduction ==
See also:

 * UnitTesting

 * [[gtest]]

Python unit tests use the built-in [[http://docs.python.org/library/unittest.html|unittest]] module with a small wrapper to enable XML output of the test results. In other words, Python `unittest`-based tests are compatible with ROS as long as you add an extra wrapper that creates the necessary XML results output.

There are two style of tests that are supported:

 * ROS-Node-Level Integration Tests: in general, these test against the external topic/service API of a Node. For these tests, it is assumed that your test script ''is itself a node''.

 * Code-Level Unit Tests: in general, these tests make direct calls into your code; i.e. these are typical unit tests. Your test script is ''not'' a node.

The syntax for running these tests is different, so it's important that you properly distinguish between the two. Node-level tests will incur additional ROS resources to test whereas code-level tests are more lightweight.

== ROS-Node-Level Integration Tests ==
Writing a unit test that acts as a ROS node is fairly simple in Python.  You just have to wrap your code using the Python `unittest` framework.

http://docs.python.org/library/unittest.html

=== unittest code ===
A bare-bones test results looks as follows:

{{{#!python
#!/usr/bin/env python
PKG = 'test_roslaunch'
import roslib; roslib.load_manifest(PKG)  # This line is not needed with Catkin.

import sys
import unittest

## A sample python unit test
class TestBareBones(unittest.TestCase):
    ## test 1 == 1
    def test_one_equals_one(self): # only functions with 'test_'-prefix will be run!
        self.assertEquals(1, 1, "1!=1")

if __name__ == '__main__':
    import rostest
    rostest.rosrun(PKG, 'test_bare_bones', TestBareBones)
}}}
NOTE: PKG should be the name of your package, and your package will need to depend on 'rostest' in the `package.xml` (`manifest.xml` with `rosbuild`).

Almost everything there should be familiar to ROS developers as well as unittest writers. Everything except for the first and last two lines are a standard unittest. The first two lines are the standard ROS-python boilerplate for setting up your python path.

'''IMPORTANT''': As this test is meant to be a ROS node launched via [[rostest]], we need to use the [[rostest]] wrapper in our main:

{{{#!python
    import rostest
    rostest.rosrun(PKG, 'test_bare_bones', TestBareBones)
}}}

The first line is also important:
{{{#!python
#!/usr/bin/env python
}}}

The parameters to `rostest.rosrun()` are:

{{{#!python
rostest.rosrun(package_name, test_name, test_case_class, sysargs=None)
}}}
 . `package_name`
  . Name of ROS package to record these results as.  Test results are aggregated by package name.
 `test_name`
  . Name to use for test.  This name will be used in the filename of the test results as well as in the XML results reporting.
 `sysargs`
  . Override `sys.argv`.
 `coverage_packages=['module1.foo', 'module2.bar']`
  . List of packages that should be included in coverage report.

The `rostest.rosrun` method assumes that your test is a `rospy` node and will perform extra operations to try and make sure that your node properly runs and is shut down.

`package_name` and `test_name` control where your XML test results are placed (i.e. `$ROS_ROOT/test/test_results/''package_name''/TEST-''test_name''.xml`). `rostest` also examines `sys.argv` for command-line arguments such as `--text` and `--gtest_output`.

=== Update your manifest ===
You will need to depend on the [[rostest]] package in order to setup your Python path correctly.

{{{{#!wiki buildsystem rosbuild
Edit your `manifest.xml` to add:
{{{
  <depend package="rostest" />
}}}
}}}}
{{{{#!wiki buildsystem catkin
Edit your `package.xml` to add:
{{{
  <build_depend>rostest</build_depend>
}}}
Do not declare a `<test_depend>`, because it will conflict with the required `<build_depend>`.
}}}}
=== Create a rostest file ===
You will need to run your node in a [[rostest]] file.  The rostest tool is based on [[roslaunch]], so this is similar to writing a roslaunch file for your test.  Please see the [[rostest]] documentation on how to integrate your node into an integration test.

== Code-level Python Unit Tests ==
You can also write normal Python unit tests with ROS based on the Python `unittest` framework.

http://docs.python.org/library/unittest.html

=== unittest code ===
<<Version(Diamondback)>>

A bare-bones test results looks as follows:

{{{#!python
#!/usr/bin/env python
PKG='test_foo'
import roslib; roslib.load_manifest(PKG)  # This line is not needed with Catkin.

import sys
import unittest

## A sample python unit test
class TestBareBones(unittest.TestCase):

    def test_one_equals_one(self):
        self.assertEquals(1, 1, "1!=1")

if __name__ == '__main__':
    import rosunit
    rosunit.unitrun(PKG, 'test_bare_bones', TestBareBones)
}}}
This is almost identical to a standard Python `unittest`.

{{{#!wiki buildsystem rosbuild
At the very top, you need to invoke `roslib.load_manifest` to setup your Python path.
}}}
{{{#!wiki buildsystem catkin
#Nothing needed.
}}}
The last two lines wrap your unit test with [[rosunit]] in order to produce XML test results:

{{{#!python
import rosunit
rosunit.unitrun('test_roslaunch', 'test_bare_bones', TestBareBones)
}}}
The parameters to `rosunit.unitrun()` are:

{{{#!python
rosunit.unitrun(package_name, test_name, test_case_class, sysargs=None, coverage_packages=None)
}}}
 . `package_name`
  . Name of ROS package to record these results as.  Test results are aggregated by package name.
 `test_name`
  . Name to use for test.  This name will be used in the filename of the test results as well as in the XML results reporting.
 `sysargs`
  . Override `sys.argv`.
 `coverage_packages=['module1.foo', 'module2.bar']`
  . List of packages that should be included in coverage report.

=== Running tests as part of 'make test'/CMakeLists.txt ===
You can have ROS automatically run these tests when you type {{{make test}}} by adding the following to your `CMakeLists.txt`:

{{{{#!wiki buildsystem catkin
{{{
catkin_add_nosetests(path/to/my_test.py)
}}}
}}}}
{{{{#!wiki buildsystem rosbuild
{{{
rosbuild_add_pyunit(path/to/my_test.py)
}}}
}}}}
=== Update your manifest ===
You will need to depend on the proper package in order to setup your Python path correctly.

{{{{#!wiki buildsystem rosbuild
Edit your `manifest.xml` to add [[rosunit]] as:
{{{
  <depend package="rosunit" />
}}}
}}}}
Edit your `package.xml` to add `rosunit` as:

{{{{#!wiki buildsystem catkin
{{{
  <test_depend>rosunit</test_depend>
}}}
}}}}
== Using Test Suites ==
<<Version(Indigo)>>

Test suites are useful when you want to write separate test cases for different bits of functionality. Test suites are composed of test cases, or even other test suites, which are then run in their entirety when the top-level test suite is run.

Here is a basic example of test suite construction:

`test/my_test_cases.py`
{{{#!python
import unittest

class CaseA(unittest.TestCase):

    def runTest(self):
        my_var = True

        # do some things to my_var which might change its value...

        self.assertTrue(my_var)

class CaseB(unittest.TestCase):

    def runTest(self):
        my_var = True

        # do some things to my_var which might change its value...

        self.assertTrue(my_var)


class MyTestSuite(unittest.TestSuite):

    def __init__(self):
        super(MyTestSuite, self).__init__()
        self.addTest(CaseA())
        self.addTest(CaseB())
}}}

Here, we have two test cases, `A` and `B`, which we add to `MyTestSuite`. We can then run the suite much like running specific test cases, except that we pass a string to rosunit or rostest rather than the test case itself:

`test/run_tests.py`
{{{#!python
# rosunit
rosunit.unitrun('test_package', 'test_name', 'test.my_test_cases.MyTestSuite')

# rostest
rostest.rosrun('test_package', 'test_name', 'test.my_test_cases.MyTestSuite')
}}}

This assumes a package structure like this:

{{{
.
├── CMakeLists.txt
├── package.xml
├── src
└── test
    ├── my_test_cases.py
    └── run_tests.py
}}}
More details about test suites can be found in the unittest documentation:

https://docs.python.org/2.7/library/unittest.html#unittest.TestSuite
== Loading Tests by Name ==
<<Version(Indigo)>>

You can also use the names of test cases or suites to run them with `rosunit.unitrun` or `rostest.rosrun`.

For example, if we have a test case like this:

`test/my_tests.py`
{{{#!python
class MyTestCase(unittest.TestCase):

    def test_a(self):
        self.assertTrue(True)
    
    def test_b(self):
        self.assertTrue(True)
}}}

We can run the two individual parts of the test case like so:

{{{#!python
rosunit.unitrun('test_package', 'test_name', 'test.my_tests.MyTestCase.test_a')
rosunit.unitrun('test_package', 'test_name', 'test.my_tests.MyTestCase.test_b')
}}}

The following lines are equivalent:

{{{#!python
from my_tests import MyTestCase

rosunit.unitrun('test_package', 'test_name', MyTestCase)
rosunit.unitrun('test_package', 'test_name', 'test.my_tests.MyTestCase')
}}}

More information about what the string specifier should contain can be found here:

https://docs.python.org/2/library/unittest.html#unittest.TestLoader.loadTestsFromName
== Important Tips ==
1. Make sure to mark your Python script as executable if it is a ROS node. You may need to use the SVN command

{{{
svn propset svn:executable ON yourscript
}}}
Instead, if the script is a unit test to be run with nosetest, make sure that it is '''not executable'''.

If no test is found, try to run nosetest inside the test directory

{{{
nosetest -vv
}}}
which will debug info on how the test scripts are searched

 1. PyUnit documentation can be found here http://pyunit.sourceforge.net/pyunit.html