Re: [PING^4][PATCH v2] Generate reproducible output independently of the build-path

[Ximin Luo <infinity0 at pwned dot gg>]

Yury Gribov:
> [..]
> 
> In general any distro provides some way to set default parameters for
> every package.  Adding hacks to compiler to work around limitations of
> particular build system does not sound right.
> 
> Here's a relevant discussion from binutils ML:
> https://sourceware.org/ml/binutils/2009-05/msg00086.html
> 

I would argue that what is being worked around, is not a limitation of a
particular buildsystem but rather a limitation of the compiler and the
language. It is GCC itself that generates the unreproducible output, and this
issues affects all buildsystems. They do not explicitly request unreproducible
output, so the fix should not be the responsibility of buildsystem.

GCC takes the input from an environment-like object (the filesystem). Part of
this input (some prefix) differs between different builds, through no fault of
any particular buildsystem but rather through the "fault" of the user running
the build - they put the source and/or build directory under that path.

Since no buildsystem is responsible for this part of the filesystem layout, it
does not seem appropriate to me, to require buildsystems to add logic to deal
with this - e.g. to pass this information down through layers of the tooling
stack, to be eventually consumed by the GCC command line. Instead, it seems
much cleaner for the "fix" to be supplied by the user (the same entity that
controls the filesystem) via some other environment-like object (here, a UNIX
envvar), and for GCC to read this directly just like it reads the abspath() /
realpath() from the filesystem environment.

CFLAGS/CXXFLAGS is also not appropriate because these flags are meant to
directly affect how GCC works. By contrast, these prefix-map options are meant
to *cancel out* already-existing sources of unreproducibility from the
filesystem environment; they do not affect the output pro-actively like other
first-class options do. Also, adding -fdebug-prefix-map to CFLAGS/CXXFLAGS
causes unreproducibility elsewhere.

(from another email)
> I believe it would much easier to reason about environment variable
> behavior when it boils down to "prepend some standard flag to
> command-line flags".  It would also simplify maintenance of local
> compiler patch as core functionality can be merged to mainline GCC
> whereas debatable environment variable part stays in the distro.

To re-iterate my point: I don't think it's right to think of these prefix maps
as the same as other command line flags. They *cancel out* already-existing
sources of unreproducibility from the filesystem environment that GCC is
*already reading*, that already makes the output hard-to-reason-about (i.e.
unreproducible). So I think it is more appropriate to pass them in an
environment-like object, rather than via command-line flags.

Normally, system-wide CFLAGS etc are a static expression of policy. What I mean
by that is, writing logic to determine CFLAGS for particular package, would
"look like" the actual wording of that policy. So for example, "all packages
should have debug" => "CFLAGS+=-g", "packages that match property X should have
flag Y" => "if X then CFLAGS += Y". OTOH if your policy is "packages should be
reproducible" and we have to add these prefix-remapping flags to CFLAGS, you're
adding a dependency from the value of CFLAGS itself, onto the build-time
filesystem. So CFLAGS is no longer a static expression of policy, it depends on
information not part of the policy.

So, prefix-map is really different from other CFLAGS, it adds extra dependency
relationships that other flags don't add.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git