Distributing a ROS system among multiple snaps

One of the key tenets of snaps is that they bundle their dependencies. The fact that they’re self-contained helps their transactional-ness: upgrading or rolling back is essentially just a matter of unmounting one snap and mounting the other. However, historically this was also one of their key downsides: every snap must be standalone. Fortunately, snapd v2.0.10 saw the addition of a content interface that could be used by a producer snap to make its content available for use by a consumer snap. However, that interface was very difficult to utilize when it came to ROS due to ROS’s use of workspaces for both building and running. At long last, support is landing in Snapcraft for building a ROS system that is distributed among multiple snaps, and I wanted to give you a preview of what that will look like.

Why would you want to do that?

Like I said, snaps bundling their dependencies is typically a good thing, and this applies to ROS-based snaps as well. Having an entire ROS system in a single snap that updates transactionally is awesome, useful for most deployment cases, and is generally what I recommend. However, there are some use-cases where this breaks down.

For example, say I’m manufacturing an unmanned aerial vehicle. I want to sell it in such a state that it’s only capable of being piloted via remote control. This is done with a ROS system, which in a simple world would be made up of:

One node to act as a driver for the RC radio
One node to drive the motors
Launch file to connect the two

You get the idea. In addition to that basic platform, I want my users to be able to buy add-on packs. For example, perhaps the vehicle includes a GPS sensor (as well as basic pose sensors). I’d like to sell an add-on pack that adds a very basic “fly here” autopilot, or perhaps a “follow me” mode. That’s another ROS system, perhaps something like:

One node to act as a driver for the GPS
One node (or perhaps a few) to act as a driver for the pose sensors.
One node to plan a path
One node to take the path and turn it into motor controls
A launch file to bring up this system

If we build both of these snaps to be standalone, we quickly run into issues:

Lots of duplication between them, as the autopilot snap will need to include most of the base behavior snap.
They both include (and will try to launch) their own roscore.
The duplicated snaps in each will try to access their respective hardware. This is a race condition: the first one up will win, the second will die. Or, depending on the hardware interface, they’ll both control it. That’s fun.

Using content sharing, we can actually make the autopilot snap depend upon and utilize the base behavior snap.

Alright, what does this look like?

Let’s simplify our previous example into two snaps: a “ros-base” snap that includes the typical stuff: roscore, roslaunch, etc., and a “ros-app” snap that includes packages that actually do something, specifically the classic talker/listener example. A quick reminder: this will only be possible in Snapcraft v2.28 or later. Also note that the example I’m about to walk through is a demo in Snapcraft, in case you want to look at the whole thing.

Create `ros-base`

(Note added after the fact: The “base” terminology here predates snap bases, a concept introduced for core18. If it was writing this post today, I’d use the word “foundation” instead of “base” to avoid the clash. That’s really the only core/base/foundational-type word not yet claimed by the snap ecosystem, heh. Anyway, don’t get confused here.)

To create the base snap, create a snap/snapcraft.yaml file with the following contents:

name: ros-base
version: '1.0'
grade: stable
confinement: strict
summary: ROS Base Snap
description: Contains roscore and basic ROS utilities.

slots:
 # This is how we make a part of this snap readable by other snaps.
 # Consumers will need to access the PYTHONPATH as well as various libs
 # contained in this snap, so share the entire $SNAP, not just the ROS
 # workspace.
 ros-base:
   content: ros-base-v1
   interface: content
   read: [/]

parts:
 ros-base:
   plugin: catkin
   rosdistro: kinetic
   include-roscore: true
   catkin-packages: []

That’s it. Run snapcraft on it, and after a little time you’ll have your base snap (the “provider” snap regarding content sharing). This particular example doesn’t do a whole lot by itself, so let’s move on to our ros-app snap (the “consumer” snap regarding content sharing).

Create `ros-app`

The starting point for ros-app is the current standalone ROS demo. We’ll use the exact same ROS workspace, but we’ll add a few more things and tweak the YAML a bit.

The recommended way to build a “consumer” snap (assuming it has a build-time dependency on the content shared from the “producer” snap, which ros-app does indeed have on ros-base) is to create a tarball of the producer’s staging area, and use it as a part to build the consumer.

Concretely, we can tar up the staging area of ros-base and use it to build ros-app, but then filter it out of the final ros-app snap (so as to not duplicate the contents of ros-base).

So let’s do that now. Get into the directory containing the now-built ros-base snap, tar up its staging area, then move it off into the ros-app area:

$ tar czf ros-base.tar.bz2 stage/
$ mv ros-base.tar.bz2 /path/to/ros-app

Now, in /path/to/ros-app alter the snap/snapcraft.yaml to look something like this:

name: ros-app
version: '1.0'
grade: stable
confinement: strict
summary: ROS App Snap
description: Contains talker/listener ROS packages and a .launch file.

plugs:
 # Mount the content shared from ros-base into $SNAP/ros-base
 ros-base:
   content: ros-base-v1
   interface: content
   target: /ros-base

apps:
 launch-project:
   command: run-system
   plugs: [network, network-bind, ros-base]

parts:
 # The `source` here is the tarred staging area of the ros-base snap.
 ros-base:
   plugin: dump
   source: ros-base.tar.bz2
   # This is only used for building-- filter it out of the final snap.
   prime: [-*]

 # This is mostly unchanged from the standalone ROS example. Notable
 # additions are:
 # - Using Kinetic now (other demo is Indigo)
 # - Specifically not including roscore
 # - Making sure we're building AFTER our underlay
 # - Specifying the build- and run-time paths of the underlay
 ros-app:
   plugin: catkin
   rosdistro: kinetic
   include-roscore: false
   underlay:
     # Build-time location of the underlay
     build-path: $SNAPCRAFT_STAGE/opt/ros/kinetic

     # Run-time location of the underlay
     run-path: $SNAP/ros-base/opt/ros/kinetic
   catkin-packages:
     - talker
     - listener
   after: [ros-base]

 # We can't just use roslaunch now, since that's contained in the
 # underlay. This part will tweak the environment a little to
 # utilize the underlay.
 run-system:
   plugin: dump
   stage: [bin/run-system]
   prime: [bin/run-system]

 # We need to create the $SNAP/ros-base mountpoint for the content
 # being shared.
 mountpoint:
   plugin: nil
   install: mkdir $SNAPCRAFT_PART_INSTALL/ros-base

Other than the ROS workspace in src/ (which remains unchanged from the other demo so we won’t discuss it here), we need to create a bin/run-system executable that looks something like this:

#!/bin/bash

# Would sure be nice if snapd gave us the triplet as well.
# It doesn't, so we'll just create it here. I'm only adding
# support for amd64 here, but one could add this logic for
# any arch they wanted (assuming ROS builds there, of
# course).
case $SNAP_ARCH in
amd64)
   export TRIPLET=x86_64-linux-gnu
   ;;
*)
   echo "Unsupported arch: $SNAP_ARCH"
   exit 1
   ;;
esac

export ROS_BASE=$SNAP/ros-base

# Add ros-base to the PYTHONPATH
export PYTHONPATH=$PYTHONPATH:$ROS_BASE/usr/lib/python2.7/dist-packages

# Add ros-base to LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/lib/$TRIPLET
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/usr/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/usr/lib/$TRIPLET

roslaunch listener talk_and_listen.launch

Why is this needed? Because the Catkin plugin can only do so much for you. The ros-base snap includes various python modules and libs outside of its ROS workspace that ros-app needs, so we extend the $PYTHONPATH and $LD_LIBRARY_PATH to utilize them.

From there, it’s as easy as running roslaunch (which by the way is contained in ros-base).

Run snapcraft on this, and after a few minutes (fairly quick since it’s re-using the base’s staging area to build) you’ll have a ros-app snap.

So now I have two ROS snaps. Now what?

You now have your ROS system split between multiple snaps. The first step is to install both snaps:

$ sudo snap install --dangerous ros-base_1.0_amd64.snap
ros-base 1.0 installed
$ sudo snap install --dangerous ros-app_1.0_amd64.snap
ros-app 1.0 installed

Now take a look at snap interfaces:

$ snap interfaces
Slot                     Plug
ros-base:ros-base        -
:alsa                    -
:avahi-observe           -
...
<snip>
...
-                        ros-app:ros-base

You’ll see that ros-base:ros-base is an available slot, and ros-app:ros-base is an available plug. This interface is currently not connected, so content sharing is not yet taking place. Let’s connect them:

$ sudo snap connect ros-app:ros-base ros-base:ros-base

Taking another look at snap interfaces you can see they’re now connected:

$ snap interfaces
Slot                     Plug
ros-base:ros-base        ros-app
:alsa                    -
:avahi-observe           -
...
<snip>

And now you can launch this ROS system you now have distributed between two snaps:

$ ros-app.launch-project
<snip>
NODES
 /
   listener (listener/listener_node)
   talker (talker/talker_node)
<snip>
process[talker-2]: started with pid [10649]
process[listener-3]: started with pid [10650]
[ INFO] [1487121136.757225517]: Hello world 0
[ INFO] [1487121136.860879281]: Hello world 1
[ INFO] [1487121136.960885723]: Hello world 2
[ INFO] [1487121137.057481265]: Hello world 3
[INFO] [1487121137.058298]: I heard Hello world 3
<snip>

Conclusion

Multiple ROS users have mentioned that the fact that a ROS snap must be completely self-contained is a problem. Typically it either interferes with their workflow or their business plan. We’ve heard you! We can’t pretend that the snap world of isolated blobs and the ROS world of workspaces merge perfectly, but the content interface takes a big step toward blending these two worlds, and the new features in Snapcraft’s Catkin plugin hopefully makes it as easy as possible to utilize.

I personally look forward to seeing what you do with this!

Why would you want to do that?#

Alright, what does this look like?#

Create ros-base#

Create ros-app#

So now I have two ROS snaps. Now what?#

Conclusion#