I’ve started using the metaphor of open source libraries as “free as in puppy” (blog) because adopting a dependent library implies ongoing responsibility to feed/vaccinate/walk it.

I just read:

Stepping beyond the immediate issue, and assuming that the reporting here is correct (never a sure thing), there’s some greater insight which can be got from this episode which is applicable to open source project management generally.

The following year, JiaT575 submited a patch over the xz Utils mailing list, and almost immediately, a never-before-seen participant named Jigar Kumar joined the discussion and argued that Lasse Collin, the longtime maintainer of xz Utils, hadn’t been updating the software often or fast enough. Kumar, with the support of Dennis Ens and several other people who had never had a presence on the list, pressured Collin to bring on an additional developer to maintain the project.

It’s not uncommon to see open source projects and tools collect complaints that they’re “abandoned” or that the maintainers are somehow not dedicating enough time to the project. We’ll see this sort of thing pop up in these forums from time to time… such and such library hasn’t been updated in too long a time, etc. It’s not unreasonable for such questions or concerns to be raised by the community, but in this case it looks like this was an engineered concern for the purposes of getting malicious individuals into maintainer roles on the project. They weaponized the good intentions of the long time maintainer to get the access they desired.

I think one lesson here is for open source maintainers, or those that would become maintainers. Finding that a framework, library, or application that you’ve open sourced and maintain has found popularity is going to put you into the firing line of both deserved and undeserved public scrutiny. There is a confidence that you must possess to avoid letting unwarranted, but perhaps considerable, peer pressure forcing your hand into foolish actions. I also expect that lone maintainers or very small, ad hoc project maintenance groups will be more vulnerable to this sort of thing compared to better established groups of maintainers with maintenance processes in place.

Another lesson is for those of us that use open source tools in that we can’t just blindly install the most convenient thing and think that we’re done. A sufficiently important, but perhaps obscure, open source project may exist somewhere in the supply chain that you have allowed yourself to rely upon and it may have the kind of well intentioned, but ad hoc maintenance management that is vulnerable to this sort of social engineering attack. Consider how this was found:

Andres Freund, a developer and engineer working on Microsoft’s PostgreSQL offerings, was recently troubleshooting performance problems a Debian system was experiencing with SSH, the most widely used protocol for remotely logging into devices over the Internet. Specifically, SSH logins were consuming too many CPU cycles and were generating errors with valgrind, a utility for monitoring computer memory.

Through a combination of sheer luck and Freund’s careful eye, he eventually discovered the problems were the result of updates that had been made to xz Utils. On Friday, Freund took to the Open Source Security List to disclose the updates were the result of someone intentionally planting a backdoor in the compression software.

Someone using a tool, that included a tool with a maintenance management problem, had to take it upon themselves to figure out what was going wrong and in so doing ultimately found the attack. Andres Freund, a PostgreSQL core team member, isn’t a maintainer of xz, but nonetheless took on the responsibility to understand the root of the problem that he was seeing with a tool that he was using. When we decide to rely upon someone else’s software, we’re in essence making a trade for increased short term productivity at the cost of increased responsibility for making sure our choices pan out and continue to be correct over the lifetime of our software.

Official note, created by Lasse Collin, with other helpful links is available here: XZ Utils backdoor

In Gentoo there is already an easy workaround. Simply newest ebuild has been masked and the one with for older version that is considered secure has been restored. The downgrade is as easy as regular updates. I have almost not noticed it as it does not causes any problems.