FHS Refresh

I’ve been busy tonight spamming mailing lists and otherwise getting the word out: the LSB workgroup is preparing to update the FHS.  This update has been a long time in coming; FHS 2.3 (the current version) was released back in 2004.  Since then, a lot has happened, and it’s starting to look like the FHS is holding things back due to the lack of updates.

For the longest time, the FHS was cared for by its original editors: Dan Quinlan, Rusty Russell, and Chris Yeoh.  We should all be grateful that they created a useful and well-written standard–one that has been resilient enough to remain useful for six years without changes.  Even though it’s time to move on, we should not forget that we are building on a strong foundation they laid for us.

So, you may be asking: how can I help?  Glad you asked!

  • First of all, get the word out!  If you know people who might be interested (developers for Linux distributions, standards people, etc.), point them to this post or to the LSB announcement linked earlier.
  • We have set up the usual open-source project infrastructure: a bug tracker, version control (using Bazaar), a mailing list, and a wiki (of sorts; it’s actually a page on the LSB wiki).  Come and join in!  Subscribe to the mailing list, post comments on the wiki, check out the source and submit patches.
  • The bug tracker deserves special mention.  We hosted it for the old FHS project, and so we’re continuing to use it.  In particular, we’ll be doing triage on the old bugs there, as well as any new bugs filed.  So go ahead and file bugs, or add comments to old bugs; we’ll be taking those into account for the new update.  If you file new bugs, please file them against the “FHS” product.

We’re tentatively shooting for a goal of releasing FHS 3.0 before July, though that’s not written in stone.  But we don’t want to wait much longer than we’ve already had to.

Standards and Conversations, Part 2

Picking up where we left off last time

The LSB spec invents things without consulting distros. Like the whole init scripts thing. But that’s not as bad as depending on RPM or requiring a specific layout.

What can be very frustrating is that we do reach out to all the major distros, and a number of the less major ones.  But we don’t talk to every single person on every single distro; we can’t.  We also try to follow best practices for an open project: open version control, open IRC, open mailing lists.

Part of the problem may be that we also talk to independent software developers, and sometimes, distro people aren’t prepared to hear what developers are saying.  So, it looks like we’re pushing things on them, like predictable directory layouts, hooks for working with the user environment, different options for software installation, and the like.

We used to just listen to distros and do what they wanted.  Part of the reason there’s still a lingering perception that “the LSB failed” is that software developers saw us as irrelevant.  And they were right: we were irrelevant, because we only listened to the distros.  So now we listen to both sides, and try to get them to talk to each other, and act as a go-between when they don’t seem able to.

I had an eye-opening experience in Berlin in 2006.  We talked to packaging people, and talked about the need for cooperation between package managers and third-party installer tools. A lot of people thought that was a bad idea.  So we got them together with some major ISVs in Berlin, and told them to figure it out.  And they did figure something out, and surprise!  Communication between package managers and third-party installers became a good thing, at least if done right.

And we don’t have a problem with the “done right” part, either.  We made a few attempts at proposals for the communication system above, and someone has created an independent implementation.  Some of those proposals came under sharp criticism.  And we’re cool with that; happy, in fact, that it got attention.

So if you want to find out what’s going on with us, and what terrible things we’re going to make you do in the future, check out our project plan, sign up for our mailing list, or just come by our IRC channel (irc.linuxfoundation.org, #lsb) and ask some questions.  We try to be friendly and helpful.

Standards and Conversations, Part 1

So it looks like the project I’ve been laboring on has been getting some attention:

Ever thought it was difficult to write software for Linux? For multiple distros? InternetNews reports that the LSB is making a push for their next release (due out later this year) that should help make all that much easier.

They even link to our project status page.  Cool!

Of course, good publicity invites criticism.  This time, there seem to be two themes.  William Pitcock seems to have the most succinct summary:

To put things simply, the LSB sucks. Here’s why:

  • The LSB spec depends on RPM. I mean, come on. Seriously. Why do they need to require a specific package manager? If package handling is really required, then why not create a simple package format that can be converted on demand into the system package format? Or why care about packages at all?
  • The LSB spec invents things without consulting distros. Like the whole init scripts thing. But that’s not as bad as depending on RPM or requiring a specific layout.

(See also Scott James Remnant.)

Let’s take this one part at a time.  Today’s topic: packaging.

Part of William’s problem may be that he doesn’t understand the spec.  The LSB doesn’t require a specific package manager, or a specific package format.  It doesn’t even require that the distribution be set up using package management at all!

The spec only requires that LSB-compliant software be distributed so that any LSB-compliant distribution can install it.  That could be tarballs with POSIX-compliant install scripts, an LSB-compliant install binary, a shar archive, a Python script with embedded base64 binaries, whatever.  One of the options allowed is an RPM package, with a number of restrictions.

The restrictions are key, because they effectively define a subset of RPM that acts as, to quote William again:

…a simple package format that can be converted on demand into the system package format…

The difference being, of course, that we didn’t reinvent the wheel and create our own; we used a popular format as the basis for ours.

Scott raises another concern:

While much of the LSB can be hacked into a different distribution through compatibility layers and tools, such as alien, what ISV or other vendor wants to provide a support contract against a distribution that has such kludges?

I’m not sure if he’s referring specifically to packaging or to the standard in general.  As regards packaging: the reason we specify a strict subset is because we can test that subset, and we’ve tailored it to the needs of tools such as alien.  The theory goes that alien isn’t a kludge when it comes to LSB packages.

But, as already mentioned, if vendors aren’t comfortable with supporting RPM, they have a number of other options.  As it turns out, most of them are doing just that; the feedback we’re getting from most ISVs is that packaging (whether LSB-subset RPM, full RPM, or Debian) is just not worth the effort.

Coming up: part 2

It’s Not Like You Care About Your Documents

Recently, as part of the many antitrust/anti-competition legal actions they’re suffering under, Microsoft released specifications for the old Office binary file formats. As expected, they’re big and complex. Joel Spolsky (a former member of the Excel team) had some thoughts on their size and complexity:

With a little bit of digging, I’ll show you how those file formats got so unbelievably complicated, why it doesn’t reflect bad programming on Microsoft’s part, and what you can do to work around it.

The digging turns up reasons that make some sense: the limitations of older computers, feature creep, a complete lack of attention to the future. But it’s hard to see some of these reasons as “why it doesn’t reflect bad programming on Microsoft’s part”. Carelessness is common, sure, but we don’t call it a virtue because everybody does it.

And these are problems that should have been on someone’s radar at Microsoft. It’s one thing for a grunt programmer to hack a feature to meet a deadline; it’s another for the management to simply go along with it, or to not order a rethink when the problems come to light. When you read about hacks like the following, everything sounds nice and reasonable, until you remember what the end result is: that Microsoft Excel doesn’t have a standard format for storing and manipulating dates!

There are two kinds of Excel worksheets: those where the epoch for dates is 1/1/1900 (with a leap-year bug deliberately created for 1-2-3 compatibility that is too boring to describe here), and those where the epoch for dates is 1/1/1904. Excel supports both because the first version of Excel, for the Mac, just used that operating system’s epoch because that was easy, but Excel for Windows had to be able to import 1-2-3 files, which used 1/1/1900 for the epoch. It’s enough to bring you to tears. At no point in history did a programmer ever not do the right thing, but there you have it.

It may not have been the wrong decision, in the sense that it enabled them to ship, and shipping is everything in some circles. But as a design decision, how can anyone defend such inconsistency?

Business information technology was able to move forward in the early ’90s because older document formats like 1-2-3 and WordPerfect were simple enough to import easily into Microsoft Office. Today, when we talk about moving to open-source suites like OpenOffice or online systems like Google Docs, detractors left and right cite the pain of document conversion as a reason to hold back. But if Joel is right about the old binary formats, the pain of transition is like the pain of changing your oil: you can pay now, or you can pay a lot more later. Even Microsoft is having trouble opening its own files from long ago, with “long ago” being a period measured in years, not decades.

Maybe you didn’t write anything a decade ago you’d care to read again today; maybe you can’t imagine any of your stuff being worth reading a decade from now. Do you want to take that chance?

Thankfully, I was a geek, and kept most of my documents in plain text. Today, I take care to save important documents in formats and encodings designed for the long haul, like Unicode, ODF, and PDF. It helps that I avoid Microsoft software like the plague. (If you think they’ve changed since the bad old days, just surf the web in Firefox on Linux sometime, and see how many badly-rendered pages look much better when you switch their text encoding from Unicode to “Windows-1252”.)

If you have a lot of Office documents, even if you’re happy with Office, you might consider whether you care about opening those documents ten years from now, and whether you’d rather take the time to future-proof them while you still can.

LSB 3.2 Beta

Today, we released the first beta of LSB 3.2. If all goes well, this will hopefully be the only beta.

We’ve been working on 3.2 for a while, and we’re really excited about it. We’ve added quite a few interfaces, based on feedback from application vendors and others. There are whole new sections: printing support, Perl and Python, FreeType, Qt 4, and trial use support (our new name for “optional”) for Xrender, Xft, and the ALSA API.

Betas can only be as good as the people participating; more feedback means a better standard. So please go check out the beta. Look at the whole thing, or just parts you’re interested in. Read the spec, or check out the tests, or try building your favorite open-source app with our SDK.

We’re hoping for a release before Christmas, but that depends on the feedback we get, of course. And we’d rather know about that really big issue we forgot about and delay the beta than find out after the release. So get cracking!

LSB Distro Testing, Redux

A while back, I posted a set of instructions on how to test a Linux distribution for LSB compliance. With the 3.1 update, testing has gotten a lot easier.

The most notable enhancement in the update is the LSB Distribution Test Kit Manager, or “DTK Manager” for short. This is a framework which controls the execution of the entire test suite and collecting the results.

So, it’s time to update the instructions.

First of all, as before, your distribution must ship a few things. There must be a “lsb” package, which depends on everything required by the LSB; if it’s not installed by default, you will need to install it. Your distribution must have a facility for installing RPM packages; this will usually either be RPM itself, or a converter such as alien. (The alien utility is used mostly by Debian-based systems, such as Debian, Xandros, or Ubuntu.) Once those are in place, you should install the “Xvfb” or “xvfb” package provided by your distribution. Since Xvfb is a part of X.org, it is almost always available.

When your system is ready, download and install the “lsb-dtk-manager” package, found on the Linux Foundation’s download page. Several bundles are available; find the one that matches the architecture of the distribution you are testing. You may use the “lsb-dist-testkit-manager” or “lsb-dist-testkit” tarballs. Once the tarball is downloaded, unpack it, change to the directory it creates, and run the install.sh script.

After install.sh is done, start DTK Manager. This is done with the following command:

/opt/lsb/test/manager/bin/lsb-tef-start.sh [port-number]

This will start the manager back end, and attempt to run a browser to present the user interface. If this doesn’t work, point a browser at the port number you gave the script. The port number defaults to 8888 if you don’t give one.

If this has all worked, you will see a welcome page in your browser. Click “Get Certified”, fill in the information it requests, such as your name and the name of your distribution; this information will be stored in the test results. Then click “Run tests”.

And that’s it! The tests will take quite a while to run; the browser will display a status window. You can close the browser and do other things while the tests runs; point the browser back at the DTK Manager port and click “Execution” to see progress.

When the tests are done running, you will be presented with a results page, which tells you how the tests went. You can fix any problems you find and re-run the tests by going back to the “Get Certified” link. If you pass most tests and fail only a few, you can use “Custom Tests” to run just the test suites with failures.

Of course, you can still run the tests the old way if you prefer; the journals are all that we need for certification. Give the new DTK Manager a try, though, to see if it’s easier.

It’s certainly made my job easier. Besides the ease with which the tests are run (meaning fewer requests for help from testers), it’s possible to completely automate test runs, which will ensure that we can test the next release of the LSB more extensively and learn about problems sooner in the development process.

Like what you see? Thank the Insitute for System Programming at the Russian Academy of Sciences. They’ve done an excellent job.

Getting the Message Out

From a Fluendo employee:

Are we evil that we don’t take more hours out of our day to build on glibc 2.3 ? You bet, we are cold heartless bastards. But in reality 90% of the people on glibc 2.3 are users that have an upgrade path to a more recent version of their distro; the other people are future Debian Etch users. I’m sure the Etch releasers have convinced themselves of the usefulness of not releasing with a glibc 2.4 that is more than 15 months old, and instead opt for an even older series, even before they actually release. But I am starting to wonder more and more who the people are that are waiting for a release like this.
Realistically speaking, it is possible that we may add glibc 2.3 plugins in the future if we see that more than just Debian is affected. We are not against taking your money for giving you a service that works. But the hours in our day are just as scarce as they are in yours. I just wanted to explain this to people that want to know, to take away your incentive to complain about a nameless faceless Company being Evil to you.

Elsewhere, we learn why Debian is so “backward”. In sum: upgrades from the current stable would break with 2.4, and not all Debian architectures are supported well by 2.4.

I suspect the market for Linux multimedia plugins isn’t a huge one, and Debian is still popular both for end users and as the basis for other efforts. Given that, doesn’t it make sense not to artificially exclude a whole chunk of your potential market?

Of course, I think I know of someone who could help here…

Same As The Old Boss

OK, I’ve been busy, which is why I haven’t said much recently. But in case you haven’t noticed, my old employer, the Free Standards Group, has merged with another open-source consortium (Open Source Development Labs, or OSDL) to form the Linux Foundation. I’ve survived the merger, and am still doing most of what I was doing before.

The reaction has been pretty good so far, and even the criticism we are getting has resulted in a good amount of support in our defense.

Suspicion isn’t out of place. The new group will have to earn its reputation, just as the old organizations did. But I think we’re in a good position to show that we’re not about “combin[ing] open source with all the worst aspects of the proprietary commercial software industry” (see the critical link above, last paragraph). Our management is the same as the old groups, and both groups have established records of improving the quality of open source. Is it that hard to believe that we would do more of the same after the merger?

As for me, I’m going to be doing pretty much the same stuff I was doing before, only with a few more helpers. What could be bad about that?

UPDATE (2007-02-16): I had given the impression that OSDL’s management was entirely gone from the new organization, which is not true.  Sorry, guys!

What Do We Want From Microsoft?

Jason Matusow of Microsoft wants to know:

That said, the real voice of the community is…well…from those of you I don’t know. I have to tell you that the issues with getting this covenant right are incredibly complex and there are real concerns on all sides. Our design goal is to get language in place that allows individual developers to keep developing.

(This is in response to the recent patent deal between Microsoft and Novell, and the poor reception it’s getting from the free software community.)

Unfortunately, he got GrokLaw-ed, and his comment system isn’t taking the heat well. So, here’s my feedback; hopefully, he’s paying attention to views outside his comments.

The big problem, if you ask me, is the distinction between “commercial” and “non-commercial” that Matusow (and everyone else I hear from Microsoft) is making.

In our world, that distinction is a lot less important than the distinction between “proprietary” and “open”. For us, “commercial” is just another way software can be used, and restrictions on commercial use are like restrictions on use by women, or by people in Illinois, or by people who have ever picked their nose in public. Why are businessmen any less deserving of our software as a class than housewives, or Haitians, or other free software developers?

Matusow claims not to be interested in any of this:

We are not interested in providing carte blanche clearance on patents to any commercial activity – that is a separate discussion to be had on a per-instance basis. As you comment, please keep in mind that we are talking about individuals, not .orgs, not .com, not non-profits, not…well, not anyone other than individual non-commercial coders.

Dialogue often means meeting the other person where they’re at, not where you want them to be. They would, presumably, not take us seriously if we insisted on a blanket patent license as a condition for any kind of conversation. Fair enough; but then why should we taken them seriously when they insist on us turning our backs on one of our bedrock principles?

But does the conversation have to be either-or? I’m betting that Matusow’s blog post is evidence that it doesn’t. People at his level are not the types to waste time on wild goose chases.

And is it all that strange to think there might be value in the conversation? There’s a mighty thin line between “proprietary” and “commercial”, so thin even we get them confused sometimes. Does Microsoft really care all that much about for-profit use and improvement of free and open tech? If so, they’re prominent members of a small and shrinking club. If not, then it seems to me that we have a lot of common ground for discussion.

Blog Update

Well, it’s been over a month since the last entry. So much for posting more often!

Today, I’ve updated the blog to WordPress 2.0.3, and installed a new theme. I wasn’t too happy with the old Steam theme, but it was a variable-width theme, and I can’t stand fixed-width themes. (Why buy a better monitor if all the Web pages are forced to 600 pixels?) But with the new and improved theme support in 2.0, there are some nice themes that use your whole browser window.

(Posted to all known aggregators, too; I hope Planet doesn’t decide all my posts are new now.)

UPDATE: Well, that was fun; the nice-looking theme happens to be completely invalid. Expect theme changes over the next short while.

UPDATE: Wow, that’s depressing; the state of valid XHTML in WordPress themes is, uh, underwhelming. So I switched back to the nice theme, and edited it to be valid XHTML 1.0 Transitional and valid CSS. I’ve set up a Bazaar-NG repository for my changes.

Multimedia on Linux

Another interesting topic for standardization in the LSB involves multimedia. It’s clear that we need to give developers a good story on how to do multimedia on Linux in their applications; what’s less clear is what that story should be.

There’s been an interesting conversation regarding GStreamer, and its status in the KDE desktop. Apparently, burned by their experiences with their previous sound framework, the KDE folks are writing a new system, called Phonon. The idea is that Phonon would provide a clean, stable API layer for KDE apps to use for the vast majority of simple multimedia-ish things, like playing a sound clip.

Christian Schaller, a GStreamer hacker, isn’t too thrilled with this, and posted an unflattering analysis of Phonon to his blog. This prompted the kind of response you’d expect, including criticisms of GStreamer:

All other arguments aside, GStreamer doesn’t offer a stable API. I can understand why that’s the case, but as such, because of the (sane) library policies within the KDE project on binary compatibility we cannot simply use a GStreamer binding as our multimedia solution. Period. I was a little surprised by Christian’s posting because we’ve talked about this multiple times.

This piqued my interest, because there’s been talk within the LSB to add a multimedia framework, and GStreamer is one of the candidates. So Christian’s response is very important:

I consider Scott a friend and I think his entry is well considered. My general response is that the bigger and more complex an API gets the chances of getting it right the first time goes down.

I’m not sure how to react to this. If GStreamer’s ABI is still in flux, it may not be a good candidate for inclusion in the LSB. On the other hand, are there credible alternatives? Phonon’s scope is too limited, and it will likely be tied strongly to KDE, which makes it less desireable. There are other frameworks, but I’m not seeing that any of them have the credibility of GStreamer.

Perhaps the best we can do, at this time, is standardize at a low level (ALSA and Video for Linux) and wait for a clear winner in the multimedia space to emerge.

That sounds like an endorsement of the Phonon approach, and in a sense it is. But we have to be careful that we don’t create another source of complexity for the Linux desktop. If Phonon encourages all of us to play around with two or three separate multimedia frameworks, to the point that we can’t really have multimedia on our desktop without having to mess with more than one, then the Phonon supporters will have done the Linux desktop a disservice.

Future Standardization Directions

Now that LSB 3.1 is out, there’s been some discussion of future directions for the LSB to take. Not surprisingly, desktop componentry beyond the graphical toolkits (GTK+, Qt) has been of interest.

If you’re interested in this, the results of Sun’s evaluation of the GNOME interfaces for inclusion in Solaris provide a lot of good information about what parts of GNOME are stable enough for inclusion.

Some candidates for standardization were left out due to uncertainty over their status as standards:

We would also like to add the icon integration specification as “Stable”, but the fact that the FreeDesktop Standards website makes a weak stability claim by saying, “freedesktop.org is not a standards body” makes us a bit unsure which specifications should be considerd Stable. It would be good, I think, if the FreeDesktop community could make a stronger claim about the specific specifications that are needed for desktop integration, such as those recommended for use in the GNOME System Admin Guide, and they should probably be referenced on the GNOME Developer Standards page as well.

Well, if the freedesktop folks are nervous about being a standards body, perhaps they could work with a standards body to codify the things they consider standards. I know of at least one candidate for that position…

LSB Distro Testing

I’ve seen several requests for a simple set of instructions that test a distribution against the LSB. I wrote some Debian-specific instructions in a mailing list post back in October, and thought they’d do better as more general instructions.

Before you start, you have to find out two things about the distribution you want to test:

  • How to install the LSB on your distribution. At minimum, your distro should provide something like a “lsb” package. Make sure that’s installed.
  • How to install LSB packages on your distribution. RPM-based distros have it easy here, since the procedure is likely to be the same as installing any other package on your system. Debian users (and users of Debian derivatives) need to use alien; I’ve found that you get the best results with alien -ick [package]. Other distributions will have their own ways, possibly involving alien as well.

Go to the LSB download page and start downloading everything under the Runtime Tests for the LSB version and architecture you want to test. Make sure you don’t get the betas unless that’s what you want to test. You will also need the lsb-python package from the Application Battery list. Install all of these packages.

Once all of the packages are installed, run the tests:

  • /opt/lsb/bin/lsblibchk
  • /opt/lsb/bin/lsbcmdchk
  • Add /usr/opt/lsb/appbat/bin to your PATH, run gcc -dumpmachine to get your machine triplet, and run /opt/lsb/share/test/qmtest_libstdcpp_[version]/run_tests. Answer the questions as appropriate.
  • For the X test suites, install enough of XFree86 or X.org on your system for proper client support, as well as the Xvfb X server. Debian users can install the x-window-system-core and xvfb packages, but see below about problems with Xvfb. After this is done, go to the directory /opt/lsb/test/vsw4 and run the run_vsw4.sh script found there.
  • For the main runtime test, make sure the loop driver is loaded. (This is mostly a problem on systems using udev.) After the test package is installed, set passwords for the “vsx0”, “vsx1”, and “vsx2” users, log in as vsx0 at a system console, and run run_tests. Logging in is required; su or sudo won’t work. The default answers to the questions asked are mostly OK, but a few don’t have good defaults. In particular, you’ll have to tell the script the right passwords for the vsx users, and you should make sure the test block device is /home/tet/test_sets/nonexistb. Be prepared to type the root password several times to set up various things the tests need.

Most of the tests are quick, taking less than 15 minutes usually per test. The runtime test takes somewhere in the neighborhood of six hours, and often looks like it has hung. Don’t assume the runtime test has hung until it’s run overnight. If you’re using an emulator, don’t give up on runtime until it’s run for at least 24 hours.

The tests create two kinds of files: journal files, and the official runtime report (created by the runtime test). Where you ran the tests, look for files with names starting with “journal” in the current directory, or for paths like results/0001e/journal. The C++ tests create a directory called “qmtest_libstdcpp_[version]”. There’s a handy utility called tjreport in /opt/lsb/bin; run that on the journal to get a quick summary of the results. If you want to post results, use tjreport. The official runtime report (in /home/tet/test_sets/results) has some additional information: a list of FIP (Further Information Provided) results.

What to do with failures: First, make sure it’s really a failure of the environment, and not a failure of the tests. The LSB publishes a list of official waivers for test failures; check that your failures aren’t on that list. For failures not on that list, Google is your best resource; most likely, someone else has experienced the same failure, and will have more information about it. If you really can’t figure it out, come over to one of the LSB’s mailing lists and ask around.

On Xvfb: The Xvfb versions shipped with most distributions have bugs that can cause problems. Anything running XFree86 will likely not be able to complete the X tests, and anything running X.org may see a number of failures. Because Xvfb is not required by the LSB, you can replace Xvfb with one that doesn’t have these bugs. Recent Debian xvfb packages for etch don’t have these bugs, so Debian and derivatives can use them for testing. (The sarge backport of Xvfb can be installed on vanilla sarge without upgrading any of the other XFree86 packages; missing packages can also be pulled from the sarge backport without affecting the packages that do ship.)

Please post questions, problems, criticisms, etc. in the comments. A version of this might end up on the LSB site someday, so any improvements would be appreciated.

New Job

Now that the right people have been told, I can make it public: as of January, I’ll be a full-time developer for the Free Standards Group, producers of the LSB.

This is, perhaps, one of the most difficult job decisions I’ve had to make. In every other case where I’ve changed jobs so far, I’ve done so only when it becomes clear that the old job isn’t going anywhere: either by going under, getting radically reorganized, treating me poorly, or otherwise being a dead end. None of that is true here. I still believe in what Progeny is doing. The co-workers are superb (yes, even that one), and management has always treated me well, even in difficult circumstances.

But sometimes, opportunities are too good to pass up. I think that the next year or two will be pivotal to the future of free standards, and I’ll be in a unique position to influence the direction those standards go. Plus, I’ll be able to work with another group of brilliant and talented people, with the hope that some of that brilliance and talent will rub off. I’m also positive about working primarily from home.

So, you can expect more blogging (he says, a month after his last post!), especially about standards in the free software world. I’ve created a new category for that topic, in case you’re interested in following just that conversation.