haha, swap writes go brrrrrt —

Torvalds warns the world: Don’t use the Linux 5.12-rc1 kernel

Please, please don't use cowboy kernels in production—especially not this one!

Penguins aren't all equally trustworthy.
Enlarge / Penguins aren't all equally trustworthy.

In a message to the Linux Kernel Mailing List yesterday, founding developer Linus Torvalds warned the world not to use the 5.12-rc1 kernel in his public git tree.

Hey peeps - some of you may have already noticed that in my public git tree, the "v5.12-rc1" tag has magically been renamed to "v5.12-rc1-dontuse". It's still the same object, it still says "v5.12-rc1" internally and it is still is signed by me, but the user-visible name of the tag has changed.

As it turns out, when Linus Torvalds flags some code dontuse, he really means it—the problem with this 5.12 release candidate broke swapfile handling in a very unpleasant way. Specifically, the updated code would lose the proper offset pointing to the beginning of the swapfile. Again, in Torvalds' own words, "swapping still happened, but it happened to the wrong part of the filesystem, with the obvious catastrophic end results."

If your imagination is insufficient, this means that when the kernel paged contents of memory out to disk, the data would land on random parts of the same disk and partition the swapfile lived on... not as files, mind you, but as garbage spewed directly to raw sectors on the disk. This means overwriting not only data in existing files, but also rather large chunks of metadata whose corruption would likely render the entire filesystem unmountable and unusable.

Torvalds goes on to point out that if you aren't using swap at all, this problem wouldn't bite you. And if you're using swap partitions, rather than swap files, you'd be similarly unaffected. Unfortunately, he then reminds us that while he knows an absolute ton about the kernel, he isn't necessarily all that familiar with all the plumbing a normal end user is concerned with:

And, as far as I know, all the normal distributions set things up with swap partitions, not files, because honestly, swapfiles tend to be slower and have various other complexity issues.

Many distributions still default to swap partitions, rather than files. But Ubuntu—which is perhaps the single most widely deployed Linux distribution on the planet—has been installing swapfiles by default for more than four years now. If you're an Ubuntu user (or user of an Ubuntu-derived distro, such as Mint), you've probably got a swapfile, and this bug would probably trash your entire root filesystem.

Torvalds' warning matters above and beyond what individual users might do with a release candidate kernel, however. It's even more important that kernel developers not base their own work around that release and potentially carry a very nasty bug forward further down the line.

I want to make sure that nobody starts new topic branches using that 5.12-rc1 tag. I know a few developers tend to go "Ok, rc1 is out, I got all my development work into this merge window, I will now fast-forward to rc1 and  use that as a base for the next release". Don't do it this time. It may work perfectly well for you because you have the common partition setup, but it can end up being a horrible base for anybody else that might end up bisecting into that area.

This also leads into one of my own rather frequent warnings to fellow Linux users: don't blindly leap ahead into cowboy code that hasn't yet been sufficiently tested. Linux kernel release candidates are usually very, very solid, and it's tempting to dive into new features as early as possible—but doing so can have very, very ugly consequences. And many of those consequences could have been avoided by waiting for the code to enter production status in the first place.

Channel Ars Technica