SIG Coordinator: Tully Foote
Topics: ROS.org/wiki, answers.ros.org, Hudson, Non ROS packaged debian packages (ala opencv, openni, gazebo, pcl)
- Ken Conley
- Michael Carroll
- Tully Foote
- Ethan Rublee
- Melonee Wise
- Florian Weisshardt
- Ben Pitzer
- Troy Straszheim
Wish List of Use Cases
- Enable debbuild to run on storm servers
- Move Hudson/Jenkins job configuration to source control
- Moving server configuration to Puppet/Chef
- Enable external institutions to run build/test/doc/release infrastructure
- Install release scripts via easy_install/pip
- Enable non rosbuild build/test/doc/release
- User stories
- Make package review and support status much more visible
- Wish List of Use Cases
Wish List of Use Cases
- Smaller debs/strip debug symbols: it would be great to land this capability. It would greatly reduce the size of our debs, which has many, many wins
Enable debbuild to run on storm servers
Technically we can do this, but it has issues -- due to the deb caching, the test for deb_in_repo on external servers is not 100% correct and can make it impossible to get a full build in.
Move Hudson/Jenkins job configuration to source control
Right now we edit jobs directly on Hudson (with the exception of job_generation). It would be a better practice to maintain the job configuration scripts in source control and push them to the build server:
- Easier to share jobs configurations with external institutions
- Easier to revert changes
Releated to this it would be nice to make this pip installable instead of a ros package, based on rospkg.
Moving server configuration to Puppet/Chef
Right now server configuration is done in numerous places, from the Hudson jobs themselves to configuration within the run_chroot script. It's hard to know what configuration a build server will actually have and it also makes the chroot scripts less re-usable -- its frequently been forked in the past in order to do different configs and it also has a bunch of extra branch logic/options.
Puppet, for example, has intro tutorials on configuring SSH keys as well as hooks for apt/pip.
Enable external institutions to run build/test/doc/release infrastructure
Ben is already somewhat doing this with doc infrastructure, but it would be nice to make this reproducible.
Main thing to work on is externalizing configuration from scripts and some documentation.
Sub use cases:
- Run documentation infrastructure
- Run continuous integration for a particular platform (e.g. Fedora)
- Produce internal releases of code
Install release scripts via easy_install/pip
Right now the install scripts have to be installed via SVN checkout. They also rely on APIs within the ROS stack, which creates bad coupling. It would nice if the release scripts could be installed/upgraded via easy_install/pip. It would also be nice to migrate them to the new rospkg/rosdep/vcstools standalone tools, which have better APIs and are easy to keep migrating together.
Enable non rosbuild build/test/doc/release
Projects which attempt to be ROS agnostic which we develop or contribute to(ala opencv, openni, gazebo, pcl, ecto) are lacking reusable infrastructure for continuous integration.
- Setup post commit hooks for these projects, that trigger pull, build, test, documentation update.
- When a new deb is produced, need to trigger a rebuild of all ROS packages that depend on these. Its a manual process of version locking currently to rev a new debian of OpenCV, as we can never be sure of OpenCV's binary compatibility. PCL will benefit from this as well.
- Clear path for side by side installation of these libraries. Typically there can be only one on your system. Either the libraries can support side by side installs by mangling names and directories (soname is not enough), or the debians can specify that they conflict, in which case apt-get will prompt you to uninstall electric when you install fuerte.
I develop a library with a README. Users of my library usually build from source, install it, and then link against it somehow. I want ROS users to be able to use my library and release against it.
I develop a library, and i have unit tests and docs. I want automatic builds to occur on a regular basis, with tests and up to date docs, without having to write my own infrastructure.
I develop a ROS application that solves object recognition. However I use a few really great libraries that I have to build from source. I want to be able to release my ROS application, and these other dependencies are blocking me.
- register my library with ROS, writing what ever configuration is necessary for ROS to be able to build my library.
- commit some changes. It builds, tests, docs update.
- Run a release script, tags a release, kicks off a debian build, generates a rosdep, invalidates all ROS packages that use it. It gets installed to some FHS that i can point client code bases at easily.
If users are not using debs of the standalone library, most likely a sourcedep is being used? Possibly sourcedep should be auto generated for anything in the standalone index.
Make package review and support status much more visible
- Right now a development package vs a core package look the same on the wiki and in rosbrowse.
- Color coding for review status and support level would be great.
- Main needs: queue management (minor work)
- Need to rework email generation to enable migration forward to jenkins from our hudson instance.
roslocate, rosco, and rosws are recent upgrades to this toolchain.
job_generation: converts rosdistro files into Hudson jobs. Needs to be ported to standalone libraries (shouldn't be hard)
- run_chroot: runs test jobs in a chroot to keep machines clean. Also
brings machines into a known state. Pretty useful script, though I'd like to explore replacing the configuration management aspect with Puppet or Chef.
- storm/cloud: brings up external cloud servers for testing (currently private)
create.py: needs to be ported to standalone script (as well as ported to rospkg APIs)
- ROS debian building (needs a lot more work)
rosdeb: needs major cleanup, lots of duplicate code
- would like to make progress on migrating to pure-Cmake/rosbuild2 builds
- would like to strip debug symbols into separate package
- Non-ROS debian building (e.g. OpenCV. Needs more tooling to make
it easier to use/verify)