Scott James Remnant Considered Harmful

I refer, of course, to the appearance of Bug#313400 in the archive and making unstable bootstrapping break in subtle ways on the very day I make my first debootstrap upload in a year and a half. Bastard.

I noticed the bug when converting the sid script to have the nifty detailed progress support Colin Watson implemented for Ubuntu. The relevant implementation detail is that various fds get munged around, and then held open by daemons like inetd and cron that get started by /usr/sbin/start-stop-daemon when they should’ve been skipped by the dummy /sbin/start-stop-daemon that debootstrap cunningly puts in place, which causes the pipe to stay open, and debootstrap to hang just before it finishes.

Ignoring that problem, the other hacking seemed to go fairly well.

Having debootstrap report progress while the install’s happening made me feel like putting all the confusing dpkg output somewhere else, so there’s now a debootstrap.log file in the /debootstrap directory in the root of the target, that gets moved to /var/log/bootstrap.log when (if…) the installation completes. That’ll probably make things awkward for installer types who might want to keep the verbose output showing up live on a tty, so something might have to be tweaked around there to have that come back. We’ll see.

The override munging I did yesterday seemed to work out, with Priority: required and Priority: important matching the packages that are meant to be part of the base system fairly well - at least on i386. So the sid script now also doesn’t have any hardcoded information about what’s in base, it pulls it from the Packages file instead. By the time it appears in the archive, there’ll also be Build-Essential: yes tags to help it work out which packages to install in the buildd variant. Unfortunately these are likely pretty wrong at the moment, but should improve soon enough. And won’t require a debootstrap upload in order to improve in future either! Awesome! (Also awesome: closing bugs that have been open for over four years!)

What else? Ah yes. I wrote a version of pkgdetails in perl (well, shell and perl), so that I could make the package just a simple matter of scripting. In theory that should let you just grab the deb and run debootstrap on Red Hat or Solaris or maybe even Windows. In practice, we’ll see – but hey, it gives me an excuse to close the bugs anyway. Unfortunately the perl version is noticeably slower than the C version, and unfortunately one of the particularly slow bits that parsed the entire Packages file was getting called once for each package and being almost as painful as reading this sentence. So the invoking code in download_release also got rewritten, and now, not only is the Packages file only parsed once while all the debs are being downloaded and validated, but the code’s in roughly the right form to make it plausible to implement support for pulling from multiple sources.

Finally, there’s the Release.gpg signature checking that Colin implemented for Ubuntu. Just install gpgv, point debootstrap at a trusted keyring with the --keyring argument, and you’re done. Sweet.

That’s about it, really. Most of the other things on the TODO list are probably right to be left for a little while yet, and there’s definitely a few bugs that’ll need to be worked out in the stuff already implemented. The CodeWiki seemed somewhat useful for organising development stuff – it’s nice to be able to see some progress, and have something that looks vaguely professional, while at the same time requiring no actual effort.

So, that’s 0.3.1. Surely something’s broken this time.

Opening Pandora's Archive

Huh, while the fair use inquiry’s open for submissions, the National Library of Australia’s crawling my website. Cool. (Hrm, apparently I have Anna to thank for the link.)

Cross-strapping Revisited

Quite some time ago now, I started hacking on cross-strapping support for debootstrap – that is, you start bootstrapping your install non-natively (eg, a Hurd install on a Linux system, a powerpc install on an i386, or a Debian install on a Solaris system), then boot into your half unpacked system, and reinvoke debootstrap to finish the job.

It got mostly implemented, then left to hang while I did other things, nominally waiting for an updated makedev. I went and had a look again last year when I started playing with darcs and forward ported bits of it, but not quite enough to have it work properly.

So third time lucky – I’ve been poking around at it again over the long weekend, and it’s getting a fair bit further this time. I decided to setup a CodeWiki to keep track of some of the things I’m hacking on, something like tridge’s junkcode but with a different focus.

The cross-strapping part’s working again, and a reasonable chunk of other stuff also, such as mostly syncing with Ubuntu, automatically resolving missing dependencies within the base system, merging the buildd variant into the main sid script to reduce duplicate code, and a first pass at the code support for determining required/base packages entirely automatically.

Of course, that latter one’s had some code scattered around my filesystem for ages too – it’s been waiting on ftpmaster getting the ability to specify tags in the Packages files on a per-arch basis. That’s still waiting on a fix to Bug#225947, which has had a patch in the BTS for over a year now, and which mvo has, in theory, cleaned up recently. In any case, I decided waiting was boring, and put my Debian ftpmaster hat on tonight and munged the required/important priorities to match required/base for i386 at least; we’ll see if that worked or not tomorrow.

Anyway, in the meantime:

Uploading via ftp debootstrap_0.3.0.dsc: done.
Uploading via ftp debootstrap_0.3.0.tar.gz: done.
Uploading via ftp debootstrap-udeb_0.3.0_i386.udeb: done.
Uploading via ftp debootstrap_0.3.0_i386.deb: done.
Uploading via ftp debootstrap_0.3.0_i386.changes: done.

Let’s see what breaks.

On Ubuntu

So, one of the current boring, argumentative threads on -devel is Is Ubuntu a debian derivative or is it a fork?. Which is a bit of a weird question, since really every derivative’s a fork, and vice-versa. The real question, of course, is whether Ubuntu is a “good” derivative or a “bad” one – whether it’s going to keep working with Debian and improve Debian, or whether it’s going to compete with Debian, stealing Debian’s users, developers, sponsors and mindshare, to the point where Debian’s a dead project.

If you ask, you naturally get a clear and confident answer: of course Ubuntu’s about working with Debian and giving back to open source hackers and whatever else. But of course, even if it wasn’t, that’s what you’d get told anyway, because that’s what people want to hear. So, like a lot of other folks, I’ve been keeping my ears pricked to see just how Ubuntu does fit into the free software world, and work out what I think of it. Having spent a couple of days at their Ubuntu Down Under conference post LCA, and generally had a chance to follow what they’ve been doing for a little while now, I thought I might post my current thoughts.

Of course, as always, this is going to be long, so here’s the executive summary for people who don’t want to read past the fold. My take is that while Ubuntu’s based on Debian, it’s an independent project, and is currently planning on becoming more so. While Canonical produces an excellent free software distribution, it’s not a free software project, and is entirely happy to build a large proprietary system, viz LaunchPad, and rely on it. While Canonical strongly supports the free software community and backs up that thought with deeds throughout the company from highest level to lowest, they’re a somewhat cut-throat startup that’s happy to put “getting to where we’re going” above a Google-like policy of “don’t be evil”. More importantly though, they produce a great distribution, that’s bringing unencumbered free software to a huge number of people, and making them happier in the process. So a mixed bag, but either way, Ubuntu’s doing a chunk of good, and definitely here to stay for a fair while yet.

And now that fold.

Read the rest ...