Develop your snaps faster

Perhaps you've noticed: building a snap is slow. Snaps bundle their dependencies, which take time to fetch and unpack, and they're typically made up of multiple parts, each of which need to be built and installed into the snap. Once you finish this process, you need to install the snap and test it out. Once you notice that your code is terrible and doesn't work the way you thought, you have to make a change and go back through that process again.

The typical snap-building experience

This is the snap development process I see too often:

  1. Run snapcraft, which fetches/builds dependencies and all parts, and creates a snap. This can take quite a chunk of time, depending on the snap.
  2. Install the snap to test it out.
  3. Realize it's broken.
  4. Tweak the code to fix.
  5. Run snapcraft again.
  6. One of two things happen, depending on the version of snapcraft being used: versions prior to 2.43 change nothing and happily create the exact same broken snap you just tested; later versions error out saying "hey, the source code changed." You might find that latter option equally annoying, but it's better, trust me. I'll explain why in a minute.
  7. As a result of (6), regardless of the outcome, the developer typically runs snapcraft clean and goes back to step (1).

Stop. There's a better way. Then there's actually an even better way.

The better way: careful cleaning

Let's say you just finished step 4. If you run snapcraft clean right now, you'll remove everything snapcraft has done: all the dependencies that have been unpacked for every part, all the source code fetched, everything. For the typical project, this shotgun approach is not necessary. Snapcraft supports cleaning individual parts, and even individual steps of individual parts if you want. In this example, you probably want to run:

$ snapcraft clean <part you changed>

As another example, say you just wanted to clean the stage step of "part1":

$ snapcraft clean part1 --step=stage

Using this approach, you only clean exactly what you need cleaned, and nothing else. All the other parts will remain as they were, and when you run snapcraft again it'll leave them alone and only update what you cleaned.

The even better way: a smarter snapcraft CLI

As of version 3.0 of the snapcraft CLI, it's gotten quite a bit smarter (2.43 also contains some of this, but it's better in 3.0). The problem is, implanting the extra brains it needed involved some very significant changes, so we decided to leave it mostly disabled by default for now. The only indication you get that something has changed is that it bails if you change the source of a part. Let me show you how to flip the switch to enable the new behavior (with a caveat: it's new, beware, and please log a bug if you see something weird).

Create a new file with the path ~/.config/snapcraft/cli.cfg, and put the following into it:

[Lifecycle]
outdated_step_action = clean

That's it. Now your workflow should be more intuitive, like this:

  1. Run snapcraft, which fetches/builds dependencies and all parts, and creates a snap. This can still take quite a chunk of time, depending on the snap.
  2. Install the snap to test it out.
  3. Realize it's broken.
  4. Tweak the code to fix.
  5. Run snapcraft again.
  6. Snapcraft should notice that your code changed, and will update that part to incorporate the changes. Return to step (2).

Much better, right? There's one more improvement that can be made to this workflow.

Stop installing your snap to test

Snaps are squashfs images, which by definition are read-only. It takes time to create the image, and also requires you to install the snap again every time it changes. Both of these are problems in the development loop: you can't tweak it easily, and needing to create a new one/install it again is time better spent coding. Allow me to introduce you to a feature that has existed for a while but about which few people seem to be aware: snap try.

When you run snapcraft with no arguments, you're asking it to go through its entire lifecycle of steps and then finally create a snap. Specifically, that's pull, build, stage, and finally prime. In the prime step, the snapcraft CLI creates a directory named prime/ and puts everything that makes up the final snap in there: the proper metadata that defines it as a valid snap, etc. The process of turning that into a snap is literally creating a squashfs image of the prime/ directory. If you then unsquashed that image, you'd end up with exactly the contents of the prime/ directory again. snap try is snapd's way of supporting taking a snap for a spin from a directory instead of from a squashfs image. It essentially gives you a read/write snap, which changes your workflow again. Now it looks like this:

  1. Run snapcraft prime, which fetches/builds dependencies and all parts, and creates all the metadata that turns it into a valid snap in the prime/ directory, but doesn't create a snap out of it.
  2. Run snap try prime/ to test out the snap defined in that directory.
  3. Realize it's broken.
  4. Tweak the code to fix.
  5. Run snapcraft prime again to incorporate your changes into the prime/ directory.
  6. Test the snap out again (your changes are already available for use since they're in the prime directory, no need to install or try it again). Return to step (3).

That's a pretty nice workflow. You could even skip step 4 and 5 if you really just want to hack in the prime/ directory; your changes will immediately be reflected in the snap being tried.

I hope this proves useful to you. As always, if you have any questions feel free to comment here, or ask in the Snapcraft Forum. Now go forth and develop snaps faster.

Comments

This is an awesome post. Is there a way to adopt this type of work flow within a container or the like? Just to keep the environment clean. I would love to see more posts regarding building snaps. The process of iterating over and figuring out some of the more complex snaps would be great. The process to solve the problems. That is the one thing that I find difficult. Is that each piece of software has so many caveats. I guess basically focusing more on the process versus the technical aspects.

By Bashfulrobot

Reply

> Is there a way to adopt this type of work flow within a container or the like?

I have a few things to say, here. First of all, the way I build snaps today is in containers. I have a LXD container dedicated to building snapcraft snaps, another dedicated to building nextcloud snaps, and so on. I create ephemeral ones to test stuff from people in the forum, etc. So in that sense, the same rules apply when building in a container.

That said, something I didn't mention in this post is the main feature of version 3.0 of the snapcraft CLI: build VMs. If you're using 3.0 and you specify a base in your YAML, the entire build will happen from within a multipass-generated build VM. That's still a story that's being fleshed out-- it's missing a few features, like the ability to `snap try` which is an important part of this workflow. However, the rest of this post applies there as well. In fact, you don't need to write that config file at all; that's the default behavior when using bases.

Does that answer your question?

By Kyle

Reply

That it does! Appreciated!

By Bashfulrobot

Reply