####################################
## For instruction on writing tutorials
## http://www.ros.org/wiki/WritingTutorials
####################################
## note =
## note.0 = [[catkin/Tutorials/using_a_workspace|Using Catkin Workspaces]]
## title = Overlaying with catkin workspaces
## description = Describes how to overlay a catkin workspace on to an existing system for development and testing.
## next = 
## next.0.link=
## level= IntermediateCategory
## keywords = catkin workspace
####################################

<<IncludeCSTemplate(TutorialCSHeaderTemplate)>>

<<Version()>>

<<TableOfContents(4)>>

Overlaying refers to building and using a ROS package from source on top of an existing version of that same package.  In this way your new or modified version of the package "overlays" the installed one.

== Prerequisites ==

For this tutorial we assume that you have [[ROS/Installation|installed the appropriate version of ROS]] ($ROS_DISTRO) and are using Ubuntu.

=== Prepare Environment ===

First start by installing the `ros-tutorials` ROS-package from apt-get:

 {{{
$ sudo apt-get install ros-$ROS_DISTRO-ros-tutorials
}}}

This should install several packages containing the examples from the ROS tutorials. You can confirm this by checking the path:

 {{{
$ roscd rospy_tutorials
$ pwd
}}}

{{{
/opt/ros/$ROS_DISTRO/share/rospy_tutorials
}}}

{{{{#!wiki blue/solid
'''Note:''' If you get '`roscd: command not found`' then you have probably neglected to source you environment, do that with this command:
 {{{
$ source /opt/ros/$ROS_DISTRO/setup.bash
}}}
}}}}

As you can see the roscd command takes you to the installation location or the package rospy_tutorials, where shared resources are installed. We will next overlay the ros-tutorials debian with the same packages but from from source in a workspace.

== Overlaying with a catkin Workspace ==

Next we will need to setup a workspace in order to build and source this overlaid package.

=== Create a catkin Workspace ===

Start by creating a folder to hold the new workspace:

 {{{
$ mkdir -p ~/overlay_ws/src
$ cd ~/overlay_ws
}}}

=== Adding Packages to Your catkin Workspace ===

Now you can add a few ROS packages from source to your `src` folder:

{{{
$ cd ~/overlay_ws/src
$ git clone https://github.com/ros/ros_tutorials.git
}}}

The second command clones the default branch of the "ros_tutorials" repository.

{{{{#!wiki blue/solid
'''Note:''' It is recommended to use [[rosinstall_generator]] and [[vcstool]] for managing workspaces to ensure that the right repositories and branches are used as well as to make it easier to e.g. update multiple working directories.
}}}}

=== Building Your catkin Workspace ===

Now that your workspace has the packages you wish to overlay you need to build it. Before building the workspace, however, you need to make sure that the environment is setup the way we want it. In this case we want to build the package(s) in this workspace "on top of" the things you have installed from debs, you can ensure this by first sourcing the deb's environment setup file:

 {{{
$ source /opt/ros/$ROS_DISTRO/setup.bash
}}}

Now you can build our workspace by invoking [[catkin/commands/catkin_make|catkin_make]] from the root of our catkin workspace:

 {{{
$ cd ~/overlay_ws/
$ catkin_make
}}}

The above command should have built your packages in this workspace into the devel space of this workspace (located at `~/overlay_ws/devel`). Now once you source that devel space you will have overlaid the packages in this workspace on top of the packages installed in `/opt/ros/$ROS_DISTRO`, lets do that now:

 {{{
$ source ~/overlay_ws/devel/setup.bash
}}}

=== Testing Out Your Overlaid Packages ===

Now that the example package, `rospy_tutorials`, has been overlaid lets try our test again:

 {{{
$ roscd rospy_tutorials
$ pwd
}}}

{{{
~/overlay_ws/src/ros_tutorials/rospy_tutorials
}}}

As you can see, roscd does not take you to `/opt/ros/$ROS_DISTRO/share/rospy_tutorials` any more, even though the `rospy_tutorials` package still exists in `/opt/ros/$ROS_DISTRO`, but instead takes you to the sources in your overlay workspace.

== Chaining catkin workspaces ==

It is possible to use several catkin workspaces at the same time, and thus to separate the build process of one group of catkin packages from another group of catkin packages. As an example you might want to install ROS core packages from source as one set of packages (an "underlay"), and packages specific to your lab as another set of packages.

{{{#!wiki blue/solid
Other than that, novice users should have little reason to use this feature, and work with just one workspace to avoid confusion. Having a chain of workspaces also forces the user to call ``make`` in workspaces in the right order, and can cause failures if overlaying packages contain packages on which packages in overlaid workspaces depend.

Still, for this tutorial we will chain a second workspace to our first one to demonstrate how it works.
}}}

So create a second workspace just like you created your first (called `overlay_ws_overlay` and `overlay_ws`, respectively):

{{{
$ mkdir -p ~/overlay_ws_overlay/src
$ cd ~/overlay_ws_overlay/src
$ catkin_init_workspace
$ cd ~/overlay_ws_overlay
## THIS IS THE CRUCIAL PART FOR OVERLAYING
$ source ~/overlay_ws/devel/setup.bash
$ catkin_make
}}}

Note how the output tells you the overlay chain:
{{{
...
-- This workspace overlays: /home/<user>/overlay_ws/devel;/opt/ros/$ROS_DISTRO
...
}}}

Also note the overlay requires that the workspace we want to depend on is active when we configure. So if you delete your build folder and configure again, you will have to remember to source the respective setup.sh again to create the chain of workspaces.

To test the overlay, open a new shell, then source the new setup.sh in `overlay_ws_overlay`:
{{{
# open new shell
$ source ~/overlay_ws_overlay/devel/setup.bash
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws/src/ros_tutorials/rospy_tutorials
}}}

We can also create another copy of rospy_tutorials in our second overlay workspace:

{{{
$ cd ~/overlay_ws_overlay/src
$ git clone https://github.com/ros/ros_tutorials.git
}}}

Now we can check with roscd whether the overlay works:
{{{
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws/src/ros_tutorials/rospy_tutorials
}}}

As you can see, it did '''NOT''' work. In order for ros tools to find a new package, we need to have configured it at least once. Also roscd works on a cache of locations which we need to update:
{{{
$ cd ~/overlay_ws_overlay
$ catkin_make
# this updates the cache for roscd
$ rospack profile
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws_overlay/src/ros_tutorials/rospy_tutorials
}}}
Now you should see roscd taking you to the uppermost overlay ``overlay_ws_overlay``.

Just as a recap, open a new shell(terminal window):
{{{
$ roscd rospy_tutorials
roscd: command not found

$ source /opt/ros/$ROS_DISTRO/setup.bash 
$ roscd rospy_tutorials
$ pwd
/opt/ros/$ROS_DISTRO/share/rospy_tutorials

$ source ~/overlay_ws/devel/setup.bash
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws/src/ros_tutorials/rospy_tutorials

$ source ~/overlay_ws_overlay/devel/setup.bash 
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws_overlay/src/ros_tutorials/rospy_tutorials$ 
}}}

== Limitations of overlaying ==

When you overlay packages, you need to be aware of certain limitations. Assume we have this workspace layout with 2 workspaces:
{{{
overlay_ws/
  src/
    package_a
    package_b  # depends on package_a
  devel/
    ...
overlay_ws_overlay/
  src/
    package_a
  devel/
    ...
}}}

Assume overlay_ws_overlay overlays overlay_ws. As you can see the package `package_a` exists in both workspaces, this one is overlaying the other `package_a`. 

=== Underlayed build artifacts shine through ===

With catkin, overlay only means that where '''similarly named''' build artifacts (e.g. c++ header files) exist in overlay_ws_overlay and overlay_ws, those in overlay_ws_overlay will be used. 
'''Differently named''' artifacts, or artifacts deleted in the overlaying packages, can still visible to dependent packages. This depends on whether the dependent package depends on one package in the underlay workspace that has not been overlayed. The same goes for the overlaying package itself, i.e. cpp files may compile using headers of the overlayed package under the circumstances described above.

So as a consequence, if overlay_ws/src/package_a defines a header file that overlay_ws_overlay/src/package_a does not define, this header file will still be usable by any package in overlay_ws_overlay. It is not hidden by the overlay. The only way to prevent this for sure is to not use overlaying. Else users must be careful not to remove build artifacts in overlaying packages.

=== Dependent packages in overlay are not rebuild ===

The other limitation is that `package_b`, which depends on `package_a`, will with this setup never be rebuild, meaning it will not use any build artifacts of overlay_ws_overlay/src/package_a. This can cause certain kinds of failures. 

To be on the safe side, users need to make sure that all such packages that depend on an overlaying package are also present as source in the overlaying workspace. This can also be achieved in the example above by creating a symbolic link from overlay_ws_overlay/src/ to overlay_ws/src/package_b.


Next: [[catkin/Tutorials/using_rosbuild_with_catkin|Using ROSBUILD with Catkin]]

## AUTOGENERATED DO NOT DELETE 
## TutorialCategory
## FILL IN THE STACK TUTORIAL CATEGORY HERE