For some reason (I can’t really put a finger on it exactly), I like to keep my servers up-to-date. I know! All those security patches, all those new features– they’re a killer!

In all seriousness, I host several websites on a Mirabox from right behind my TV. I hopped on the plug computing train way back when the SheevaPlug was just a dev kit, and I’ve been on it ever since. The reasons are pretty simple: reasonable power and capabilities while using less power than my desktop computer when it’s off (i.e. phantom power). I mean come on– the Mirabox in particular has two gigabit ethernet ports and two USB 3 ports. I have a fantastic RAID 1 setup running on that. Anyway, I digress.

The Mirabox comes with a custom flash of Debian Squeeze on it that worked just fine. A few weeks ago I decided to get it up-to-date.

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade # What was I thinking...

And reboot and… the Mirabox never came back. It just hung when it tried to bring the network interfaces back up. This is the end of the boot log:

<snip>
[ ok ] Setting kernel variables ...done.
[ ok ] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
[FAIL] startpar: service(s) returned failure: module-init-tools ... failed!
pool #0: pkt_size=1536, buf_size=1632 - 2048 of 2048 buffers added
eth0: started
ADDRCONF(NETDEV_UP): eth0: link is not ready
mlan: module license 'Marvell Proprietary' taints kernel.
Disabling lock debugging due to kernel taint WLAN FW is active
ADDRCONF(NETDEV_UP): uap0: link is not ready
Bluetooth: Core ver 2.15
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP ver 2.14
Bluetooth: L2CAP socket layer initialized
Bluetooth: RFCOMM socket layer initialized
Bluetooth: RFCOMM ver 1.11
WLAN FW already running! Skip FW download WLAN FW is active
ADDRCONF(NETDEV_UP): mlan0: link is not ready

This was, believe it or not, somewhat of a blessing in disguise, since I should NOT have been running the OS off of internal memory. Why, you ask? Because this just happened. There was no way to recover– I tried booting to different runlevels, just the shell, you name it.

I ended up rebuilding the server from scratch, but on an SD card this time so I could at the very least back it up (via dd) as I made progress.

And you know what the issue was? Check out the sources.list:

deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free
deb http://http.us.debian.org/debian stable main contrib non-free
deb http://security.debian.org squeeze/updates main contrib non-free
deb http://www.backports.org/debian squeeze-backports main contrib non-free

Does anything jump out at you? Perhaps the fact that the second line is actually the same repository URL, first of all. But that’s not the bad part: notice the mixture of squeeze and stable repositories. Stable is now Wheezy! Whichever idiot at Globalscale who set this thing up just potentially screwed all of you by mixing releases. That person has wasted nearly two weeks of my time.

The solution is simple: delete the second line. The URL was redundant anyway.

Now you’re just using Squeeze!

You might ask: “Well, why not just update to Wheezy?” Because the wheezy repos contain the package updates that cause the Mirabox to no longer boot. Which packages, you ask? One of these five (they’re all interdependent… I’m not sure which one breaks it):

  • initscripts
  • netbase
  • rsyslog
  • wpasupplicant
  • ifupdown

You’ll notice in the install logs (if you go ahead with that madness) that “netbase breaks ifupdown.” I suspect the root cause to be somewhere around there.

If you figure out how to successfully upgrade to Wheezy, let me know! For now, I’m happy with Squeeze.