Snapping Nextcloud: MySQL
In the previous post of this series, we discussed the process of and lessons learned from building PHP 7 for the Nextcloud snap. Nextcloud supports a number of databases, but they recommend MySQL, so for this post that's what we'll focus on getting into the snap.
MySQL is included in the Ubuntu archives, and honestly something like this should work fine:
But it doesn't.
Lesson 5: seccomp denials are not recoverable
I find that a lot of the people I talk to think of snaps as containers (they often compare them to docker). Yes, snaps are self-contained. Yes, snaps are isolated. No, snaps are not containers. Admittedly they utilize some of the same underlying technology, but they aren't in a chroot of any kind. They're simply very confined applications. It's not my goal to discuss the entire security story behind snaps (check the whitepaper out if you're interested), but you need to know a part of it for this lesson.
One of the confinement technologies used in snaps are seccomp filters, which allows one to provide a whitelist of system calls for a given application environment. This is one of the things handled by the concept of interfaces. When you specify interfaces for a given application, you expand the whitelist of syscalls it's allowed to make (among other things). When configuring the whitelist, one specifies what should happen if a disallowed syscall is made. There are several options:
- Kill the offending application immediately.
- Notify the offending application via a signal.
- Return an errno from the syscall.
- Notify a tracer of the denial.
Allow the syscall.
I'm not a member of the security team (so take my words with a grain of salt), but only two of these options were valid for the snap use-case: either kill the offending application, or return an errno from the syscall. The other options were I believe either racy or impossible to implement in a generic manner. The obvious winner there is to return an errno-- let the application handle its own denial, maybe it can recover. However, therein lies a seccomp filters shortcoming: if one uses the errno option, none of the denials get logged! So the security team chose the last remaining option: kill the offending application immediately.
It's terribly unfriendly to the application, but at least you get a warning about it in the syslog so you know what the issue is (and snappy-debug will make recommendations for you). It's worth noting here that the security team is working with upstreams to enable logging for errno, so we hope to use that instead once it's available. (Update: snapd uses errno and logging now that it's available upstream, so this is much less of an issue now).
Anyway, MySQL attempts to control the priority of its page cleaner threads with a setpriority() syscall, which is not contained within any of snapd's interfaces. MySQL makes this call, and rightly handles its failure, but snapd doesn't give it a chance to handle the failure: it kills it immediately. The only way around this was to fork MySQL, apply a patch to make that syscall optional, and build the MySQL part from source. I'm trying to move that patch upstream, but the MySQL folks seem reluctant to accept it since they don't understand why snapd doesn't use errno instead of kill. I can't really blame them, so I'll probably be maintaining this fork until snapd uses errno.
Beyond that issue, MySQL was pretty easy to get working. It runs based on config files which can contain environment variables, so I just ship custom versions in the snap.
The next post in the series will discuss the memcache.