Junichi Uekawa asks about the support in Debian for LVM, and whether one can use LVM on the root filesystem.

There’s a question about the installer. I confess complete ignorance regarding debian-installer, but Anaconda for Debian has been doing LVM for a while now, on the root or just about any other partition. Once you get the system installed (either via native installer support or via some command-line manual setup), Debian’s infrastructure seems to have no difficulty. This is a configuration Progeny tests fairly regularly, so I’d expect everything to be OK.

Of course, one should not put /boot under anything besides a regular partition, which means that /boot should be mounted if the root is in a LVM. I’ve also heard that root LVMs under Linux are a pain to maintain. Nevertheless, there should be nothing Debian-specific about recommendations regarding the use of LVM.

UPDATE: Andrew Pollock reports that debian-installer supports LVM just fine, and complains about being forced to use LILO instead of GRUB as the boot loader. Actually, that’s one of the benefits of /boot being on a regular partition. I think it’s safer to keep all boot information outside systems like LVM as a general rule; getting GRUB back is a nice additional perk.

I’ve recently had need of a base64 codec for one of my projects. (For those who don’t know, a codec is a piece of software that encodes and decodes data using a certain encoding algorithm. If you’ve ever ripped to MP3 and/or played it back, you’ve used MP3 codecs. The base64 algorithm is one used in a lot of places to get binary data into places where binary data isn’t usually welcome.)

Anyway, the base64 algorithm is simple, standardized, and popular. Its use is mandated for certain tasks, and is a strong alternative in others. So, one would think, there should be plenty of good free implementations floating around that I could use, right?

Apparently not.

All the base64 code I could find in Google suffered from at least one of several deficiencies. Quite a few examples were in languages I wasn’t interested in, such as Java. Some looked good, but had licensing difficulties, including a few which had no discernable license grants at all from the original author. (Why even put such things out on the net?) Some had serious dependencies on data structures or other features of the larger projects they were a part of. Codecs that assumed stream-based operation were very common. And, sprinkled among the various code references, were glimpses into the frustration other projects have felt when performing the same search I was.

Clearly, there is a niche here left to be filled.

Well, at least there was. Those searching for such a thing can now use my base64 codec, implemented in base64.c and base64.h. The license is XFree86-like, and is 100% original code. It should be suitable for in-memory or streaming en/decoding, and does not attempt to allocate memory on its own. Being context-based, it can support multiple simultaneous transformations. I believe it to be fairly frugal regarding system resources.

Being a new implementation, I’m sure it has bugs. It’s not thread-safe, both because of a lack of locking mechanisms on contexts and because of the use of a single global variable internally. Additionally, I’m fairly sure the decode operation could be a lot smarter, and thus could perform much better. I’d like to think I’ve been careful regarding buffer management, though I’m not so stupid as to be confident of it. And it seems to be faithful to the spec, at least according to my testing. If I get a chance, I might improve on it later; alternatively, patches and other feedback are welcome!

Benjamin Mako Hill comments on, among other things, my recent post on the advent of Ubuntu.

For the record, he gets my intent exactly right. I do not consider “fork” or “outside-ness” to be perjorative in any way, or any kind of excuse to avoid cooperating. So the sense in which he disagrees about Ubuntu being a fork–the sense which sees Ubuntu as dissociating from Debian–is certainly not the sense of “fork” I intended.

Mako also brings up a good point. What does it mean to “be” Debian? And how can we avoid making existential questions like this into excuses not to cooperate?

At a fundamental level, Debian is about an engineering philosophy: the process that creates such great individual pieces of free software as Linux and Apache can also create an integrated system built entirely out of those pieces. This was the motivation behind Debian’s founding, at least. Since then, it seems to me that Debian has also encompassed the importance of community, as well as a more decentralized cooperative model (as opposed to the more command-hierarchy-based model most companies use) which tends to emphasizes consensus and best practices instead of expediency.

I would submit that “outside” projects and organizations can participate in this as well as anyone else, as long as they agree to the ground rules. This would, perhaps, distinguish between companies like Progeny and companies like Corel used to be: do they really believe in the essence of what makes Debian great, or do they merely see Debian’s product as a nice basis for their own efforts? Further, I see a bigger divide between these two groups than I see between “inside” groups (like CDD, Debian-Jr, etc.) and “outside” groups (like Progeny and Ubuntu) who all embrace the former philosophy.

And if that’s so, I think it’s entirely appropriate. There’s nothing wrong with using Debian to meet your own goals, but it is a little odd to respect Debian the product without respecting the process that created it. And to the extent that projects participate in the process, they become more a part of the community. That, I think, is what should determine the “Debian-ness” of a particular project: to what extent that project acts in the Debian spirit, which includes cooperating without regard to affiliations that might be divisive.

(Alas, Mako also reminds me of promises unfulfilled. I really need to get my lazy butt over to CDD and participate.)

When I was at DebConf in Brazil recently, I learned that Mark Shuttleworth (the founder of Thawte who also became the second space tourist on the International Space Station) was getting involved in Debian development by funding a bunch of Debian people. At the time, it wasn’t clear how things would shake out, but they just announced their future direction: a new Linux distribution, based on Debian and focused on the user. Scott James Remnant, one of the Canonical people, posted a little blurb on his blog about the future of Ubuntu and Canonical’s game plan.

Many of their future plans involve things Progeny has been wrestling with for some time now: relationships with Debian and the wider community. So, here follows some advice for the Canonical people, for what it’s worth. (Nothing?)

• Recognize your differences. It’s a fact that Ubuntu is a fork. It’s also a fact that Canonical will have different priorities and opinions than the Debian project. If these weren’t true, Canonical could just use straight Debian. It’s important to not expect otherwise, both for Canonical and for Debian; otherwise, frustration tends to come up when things don’t mesh like you want them to. Being a fork isn’t good or bad; it’s how you handle the fork that makes the difference.
• Be aware of your limitations. Debian has a lot of needs, and many of these will align strategically with Canonical’s needs. That doesn’t make those needs the right ones for Canonical to solve, unless Canonical truly has the resources to take them all on at once. It’s too easy to bite off more than you can chew. Don’t be afraid to make resource decisions that might differ from those the Project might make.
• Be careful when expecting things from the community. Often, things happen at their own pace, and there isn’t a whole lot a company can do to force things that doesn’t also make the company look like a bully. It’s best to expect that you’ll have to do all the work, and let yourself be pleasantly surprised. The pleasant surprises do happen, and frequently, but it’s not wise to take them for granted.
• Don’t rule out cooperation with “competitors”. So far, most of the Linux world has taken the attitude that cooperation is a good thing, even among competitors. I have experience in this through working with Red Hat on Anaconda, who have been very helpful with our efforts despite being competitors with us in several ways.

I suppose one more Debian company makes for competition for Progeny, but I do hope we all can find niches to prosper in. So, good luck to Ubuntu and Canonical. Progeny looks forward to taking advantage of your good work (heh heh).

(ObDisclaimer: I am speaking for myself here, not Progeny.)

Based on a casual reference from Edd Dumbill, I thought I’d check out the Zero Install system he mentions.

It starts out as an intriguing concept. There are obvious security implications, though, and they seemed to have addressed them here. Unfortunately, that page makes the following mistake:

Only the Zero Install software itself is a potential risk to system security. With traditional (non-zero-install) systems, every application, library and documentation package is a potential root compromise.

Of course, this system is all about downloading software off the Internet and running it on your local machine, every piece of which is a potential risk to system security. Thus, this particular “fact” is snake oil, and these people now have a very high burden of proof to overcome before I consider their system trustworthy.

It’s tempting for people to think they’ve solved a particular security problem just because they handle it better than other people. There may be benefits to the Zero Install approach, and they may even be a theoretical improvement over other systems. But “almost right” doesn’t cut it in security, and if they’re amateur enough to make claims like the above, why should I believe that their execution will be any more competent?

Signifying Nothing: Print it

Very interesting. We’re hoping to do something like this as part of discover, covering user-space hardware configuration for things like SANE. When it comes to setting up printers, it only makes sense for us to hook into this instead of doing our own.

Planet is a nifty little aggregator; it takes the posts to various Web sites and puts them all on a single site, with links to the original stories.

One of the Planet developers also runs an aggregator for Debian stuff. So, the other day, I inquired about joining. As they say, the rest is history, and I am now feeding to Planet Debian.

I have been kind enough to only feed Debian-relevant stuff. From the political posts I’ve seen so far, I’m not convinced that the rest of my stuff wouldn’t generate more heat than light, though everyone is invited to prove me wrong. I will highlight one of my more recent posts that may be of interest to Debian folk: Why Discover, a defense of discover’s relevance.

(This post was originally an E-mail, sent to explain the reason why discover deserved to be supported in some software. By popular demand [Ian], it is now posted here, slightly edited.)

Discover has three basic design criteria:

• Arbitrary data. Strictly speaking, discover has no concept of a “Linux kernel module” or “XFree86 driver”. It simply associates data with hardware, and can return that data when the hardware is found. It’s up to add-on utilities to make sense of the data.
• Versioned data. Version specifications can be attached to data, and can be used to choose between alternatives.
• Cascading data. That is, newer data can be loaded and override old data.

Thus, it’s easy for us to do things like the following:

• Pull down data updates from a central source at runtime.
• Support userspace hardware configuration for utilities such as SANE, CUPS, UPS daemons, CD burn utilities, etc. Adding support for a new utility does not require code changes to discover; often, this can be done with a new data file and a shell script.
• Support multiple versions and multiple software types. For example, right now we have XFree86 3.x and 4.x driver information in our database, and could easily add X.org and freedesktop.org X server driver info when needed without losing the earlier data. (Or someone else could provide that information and hook it in themselves.)
• Cross-platform compatibility. We are developing sysdeps that make discover work with FreeBSD, for example. While Linux kernel information wouldn’t likely be useful, userspace configuration would likely be the same everywhere.
• Driver alternatives. We already do ALSA sound drivers with 2.6 and OSS with 2.4, but we have had several requests to allow ALSA drivers with 2.4 as well. We will support this in our next data release by providing override data that can easily be added to discover’s configuration. You could handle, say, David Hind’s PCMCIA drivers vs. kernel PCMCIA drivers or open-source vs. proprietary NVidia XFree86 drivers the same way.
• Alternative purpose drivers. We could, for example, specify framebuffer kernel modules and DRI kernel modules for video hardware separately, and load either or both as we see fit.
• Fine-grained versioning. If a new driver is introduced in Linux 2.6.6, our data can reflect that, so 2.6.5 users aren’t confronted with module load failures.
• Data annotation. We mark data with dates and IDs currently to show when data was last reviewed (or not, in their absence). We will be adding a “important to debian-installer” tag so that d-i can strip out unnecessary data to save space. We’re also thinking about a rating system that would allow users to vote on, say, whether e100 or eepro100 works better for their particular Intel network card.
• Packaging. We have already written a spec for associating package data with hardware. So, for example, a utility could offer to install SANE for you if you plug in a scanner, or the installer could notice that you’re installing on a Toshiba laptop and add toshutils. Long-term, we’d like to write a utility that will be able to download kernel module source and build it for the running kernel automatically; this, along with a data update, could allow for field updates of hardware support without requiring new kernels.

Other hardware utilities, such as kudzu or hotplug, suffer from the same deficiency: they are inflexible and difficult to update. As an example, I was recently made aware that the version of kudzu in Debian is too old to support Linux 2.6 kernels; the maintainer is concerned about updating it because of other potential problems he had observed. By contrast, we have made code enhancements to discover recently, some of which make it work better with 2.6, but none of them were required for us to provide good 2.6 support.

None of this is intended to be harsh or confrontational. I don’t think other hardware utilities suck. It may be that discover has fatal defects in its design, and kudzu or hotplug becomes “the way forward” in Linux hardware detection. But we certainly think our ideas about hardware configuration deserve attention, and we’d really like to make them available for others to use.