[Rpm-maint] [rpm-software-management/rpm] Reproducible builds improvements (Issue #2894)

Tue Feb 27 22:14:50 UTC 2024

> a coherent (re)design for reproducible builds, where you basically just flick it on and be done with it

I agree that the current set of options is a bit ad-hoc. But I think it's completely unrealistic to achieve a single switch, because different build systems and ways of managing the distro create different tradeoffs, and different people want to strike the balance between useful metadata and ease-of-reproducibility. Even with the 4 options listed above, Fedora and OpenSUSE disagree on two.

In Fedora, we had a discussion about setting `%use_source_date_epoch_as_buildtime 1` and setting a fake `%_buildhost` value and many people were against. People find those informations useful, and very convenient to have them directly in the rpm, even though in they could be extracted from the build system in other ways. So this approach was abandoned, as you know, and we still have `%use_source_date_epoch_as_buildtime 0` and use real hostnames.

As @Conan-Kudo mentioned above, we have to strip metadata *anyway*. At least `HEADERIMMUTABLE`, `SIGSIZE`,
`SIGMD5`, `SHA1HEADER`, `SHA256HEADER` are "irreproducible", and `OPTFLAGS` and `PLATFORM` are often different because a "random" noarch package is selected. If you have to strip/ignore/treat-in-a-special-way those fields, then it doesn't make much difference to also handle `BUILDTIME` and `BUILDHOST` in the same way. We have to strike a balance between having useful metadata and ease of reproducibility. Since bit-for-bit reproducibility is impossible with signatures, then I think the current balance of using real `BUILDTIME` and `BUILDHOST` is good.

> We should not use clamping mtimes, outright setting them has fewer possibilities for errors (like incorrect clock). 

Please, no. As @Conan-Kudo wrote above, this has clear downsides, because it forces constant churn between rebuilds. But it also destroys useful metadata, for example timestamps in documentation that specifies when the file was last modified. I'm sure there's a million other examples. And I find the worry about incorrect clocks rather unconvincing. In the current era of signatures being checked all the time, a build system or a developer making releases with a completely wrong time would create huge problems. If you find a case like this, fix it at the source, instead of forcing a heavy-weight workaround.

> My understanding is that @mlschroe prefers to have $SOURCE_DATE_EPOCH_MTIME set the build time and if it is set rpm would ignore but pass through $SOURCE_DATE_EPOCH.

I don't think this has been thought through. `$SOURCE_DATE_EPOCH` now has the meaning of "the timestamp of the sources", i.e. the single timestamp that defines the inputs to the build. We can clamp all the files that are created after that timestamp to `$SOURCE_DATE_EPOCH` because we know that all those files are created from the original inputs, so their timestamps are not important, because they are reproducible and we can recreate them exactly independently of time. So `$SOURCE_DATE_EPOCH`  has a clear meaning and any timestamps later than that are redundant.

The second problem is that we have a timestamp that is used to clamp mtimes. If we introduce a second timestamp and use that for clamping, we have to redesign/fix/update everything to use the new timestamp. Once we have gone through that churn, we are in almost the same situation, except that the variable name is different (and we also have a two variables with a slightly different meaning).

All that said, I think it's fine if a way to e.g. add new options how to clamp/set mtimes and whatnot. But please make those opt-in, so that different distros that have slightly different approaches can make their choices.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2894#issuecomment-1967726605
You are receiving this because you are subscribed to this thread.

Message ID: <rpm-software-management/rpm/issues/2894/1967726605 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20240227/2035fe52/attachment-0001.html>