I’ve had a very particular itch for a long time, now. Something I love about open source is that it empowers developers to scratch their own itches, and a while back I decided to do exactly that. We’ve used it in production on the Nextcloud snap for a few months with relatively few issues, so I think it’s time to open it up for wider use… in case you share my itch. What is open source but a bunch of users scratching togeth…. okay this analogy is gross.
The problem
What is the problem that has annoyed me for years? It’s that reviewing pull requests for snaps sucks. It involves too much manual work. Sure, it’s relatively easy to automatically verify that the snap builds, as long as it’s a snap that actually builds within the Travis or CircleCI timeout. Then what? How do you verify that it actually works?
Testing that a snap actually works obviously depends on the snap in question. Nextcloud is a web application, so we actually have an entire suite of Capybara-powered tests to automatically verify the snap is sane. In some cases, that’s enough. For example, if a pull request is made that updates PHP, I can be relatively confident that none of those tests would pass if that PHP release was broken.
However, there are a whole host of things that can’t be automated. For Nextcloud, want to test that the Let’s Encrypt integration still works correctly? That’s all manual. What if it’s a GUI application? Those are a lot harder to fully test automatically. Besides, what if the PR changed the GUI itself? You probably want to take a look at it. Basically, you must install the snap.
Sadly, even if you build a snap as part of your CI, it’s usually lost after that ephemeral process. I mean, you can upload it to some sort of external service after it was built, but I’ve had mixed success with that, and it’s work. Besides, we already have one of those– the snap store! Why can’t we just upload there? Because it involves a very private token, which would need to be exposed on pull requests to the code random people submit, which means anyone can just steal your private token and start releasing snaps your behalf. I hope you agree that such a thing would be bad.
That turns the typical workflow for reviewing pull requests into this:
- Code review
- Pull the code
- Build a snap locally
- Test the snap locally
For every pull request. That’s actually a lot of work and a lot of time, and it meant that the snap moved quite a bit slower than I would have liked. I was always annoyed having to review pull requests (building the Nextcloud snap takes an hour), and I don’t want to be annoyed just because someone is trying to contribute! What a terrible attitude.
So I was on vacation a few months ago, and hacked together something that solved this problem, something that built and released snaps for every pull request without handing my token out to anyone who wanted it. I then reached deep into my imagination and came up with the name GitHub Snap Builder. This project is in beta and is far from finished (heck, it still has the default README). However, it’s in a usable state, and I wanted to write this blog post to get you started using it, if you’re interested. Let’s dive in, shall we?
The solution
GitHub Snap Builder (it just rolls off the tongue, right?) is a hosted solution. It runs on your own server, and you connect it to a GitHub App that you create in your organization (or under your user account). This allows it to run like any other CI system (e.g. Travis), report status on pull requests, etc. Let’s start by getting that GitHub App setup.
GitHub App
I’ll walk you through creating a GitHub App under your personal account, but doing the same for an organization is very similar. Let’s work off of the Creating a GitHub App page.
First, go to your profile settings, then to Developer settings in the left sidebar. From there click GitHub Apps in the left sidebar, and then hit the New GitHub App button.
Give it a name, and set the Homepage URL to whatever you want. The Webhook URL needs to be pointing at the GitHub Snap Builder, so use the URL of the server, e.g. http://domain-name.com. I know we haven’t talked about this yet, we’ll revisit it, just get something in there so you can create the app.
I know the Webhook secret says it’s optional, but if you’re using the GitHub Snap Builder (so fun to say) it’s not. Generate your own password, make it good, and put it in there. Keep track of it, you’ll need it in a minute.
Now for the permissions. GitHub Snap Builder requires the ability to read pull requests, and read/write commit statuses ( the place where your CI systems report success/failure). Under Permissions, scroll down until you see Pull requests and select Read-only in the drop-down. In that same section, find Commit statuses and select Read & write in the drop-down. Then continue scrolling down until you see the Subscribe to events section, and check the Pull request box (so GitHub Snap Builder is notified whenever a pull request is opened or updated).
Finally, hit the big green Create GitHub App button.
Now that your application is created, you need to do one more thing. Scroll down to the bottom of the page you’re on now (your new application) and you’ll see a big green Generate a private key button. Hit that. It’ll immediately download a private key that you’ll need in a minute. Keep the app page open, and move on to setting up your server.
Server setup
GitHub Snap Builder is written as a Ruby gem. As such, you can consume it in two ways. First of all, you can install the gem:
$ gem install github_snap_builder
Or you can install the snap:
$ sudo snap install github-snap-builder --beta --classic
We’re going to use the snap since it sets up the daemons and whatnot for us, but this post should still be useful for you even if you opt for the gem.
Once you install the snap, a daemon will be running. So how does this work? That daemon sits and waits for GitHub to tell it that a pull request event has happened (e.g. a pull request was created or updated). When that request comes in, GitHub Snap Builder will fire up an ephemeral container, build the snap, toast the container, create a new ephemeral container, push/release it to the proper channel, and toast the container again. Note the use of two different containers: it’s so we can’t possibly be running code from the pull request when we whip out the credentials necessary to upload the snap. GitHub Snap Builder was written so that the container tech is easily extended, but the only supported option today is Docker, so let’s install that:
$ sudo apt install docker.io
Why Docker, by the way? Because I have this thing deployed in a LXD container, and nested LXD containers have problems with snaps, but can run Docker just fine. Anyway, let’s configure GitHub Snap Builder. The following command will open up a YAML file in an editor:
$ sudo github-snap-builder.configure
You’ll see all sorts of options here with hopefully-helpful comments. Here’s where you need information from the GitHub App you just finished creating. Go to the App page, and grab its App ID. Set gitub_app_id in the config to that value. Set github_webhook_key equal to the webhook secret that you generated for the app. Grab the private key you downloaded, and open it up in a text editor. Copy the entire file (“BEGIN RSA PRIVATE KEY” and everything) and paste it into the github_app_private_key, formatting like the example given in the config.
Set build_type to “docker”. As I mentioned, nothing else is supported today.
You can set the bind address/port to whatever you want, but the defaults are given there (0.0.0.0:3000), and I left them alone.
Finally, let’s talk about that repos section. This is where you tell GitHub Snap Builder what repositories it’s authorized to build, what channel should be used for release, and provide the token that gives it permission to do so.
Each item under the repos section is a YAML object that is the slug of a given repo (<owner>/<repo name>). For the Nextcloud snap, that’s nextcloud/nextcloud-snap. Nested under that object are two options: channel and token.
When GitHub Snap Builder has built a snap, it will release it into a pull-request-specific branch in the store that only lives for 30 days, for example edge/pr-123. That branch is determined with the channel option, and is <channel>/pr-123.
The token option is something you’ll need to generate on your own machine. Run this command (replacing <snap-name> with the obvious thing and replacing <channel> with whatever you used for the channel option followed by a /*, e.g. edge/*:
$ snapcraft export-login\
--snaps=<snap-name>\
--channels=<channel>\
--acls=package_push,package_release -
Yes, note the - at the end, which will print a token to the screen. Copy that token, and paste it into the token option, thereby giving permission for the GitHub Snap Builder to upload snaps to the set of branches required. You’ll need to go through this process for each repo/snap you want GitHub Snap Builder to support building.
We could stop here. We could make sure the GitHub App was hooked straight up to the GitHub Snap Builder. It would be a simple matter of editing the GitHub App settings and setting the Webhook URL to http://my-server:3000. However, doing that would send all that data in the clear (including your app password), which is a bad idea.
I suggest installing nginx and using it as an SSL terminator and proxy for the snap (getting SSL certs is an exercise left to the reader, but I suggest Let’s Encrypt, it’s free). The configuration I used was pretty simple:
server {
listen *:443;
ssl on;
server_name <domain name>;
ssl_certificate /etc/letsencrypt/live/<domain name>/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/<domain name>/privkey.pem;
# teminate SSL and proxy to the actual internal web service.
location / {
proxy_pass http://localhost:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Ssl on; # Optional
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Host $host;
}
}
At this point, let’s revisit the GitHub App settings, and make sure the Webhook URL hits that nginx endpoint. With that config, it ends up being https://my-server and that’s it.
Activate GitHub App on GitHub repo
You already configured GitHub Snap Builder to build a specific repo when pull requests happen, but now you need to go install your GitHub app on that specific repo to complete the circle and actually start sending events. On the GitHub App page, hit the Install App item in the left sidebar, and then hit the green Install button next to your username (again, you could do the same for organizations). You’ll be given the option to active it for all repositories, or you can select which repositories you have in mind. As long as the repository you configured in the GitHub Snap Builder is among those you select here, it’ll ignore the rest and do what you want. Finally hit the Install button, and you’re done! Submit a pull request for that repo and watch a snap be built and released. The branch it ends up using will be provided in the status string.
Install from that branch and you’ll get whatever the most recent change in that pull request produced. As the pull request is updated, GitHub Snap Builder will release updates into the same branch.
As always, I hope this is useful to you. Feel free to log issues, and pull requests are always welcome!