Unstable build/host qsorts causing differing generated target code

[Cory Fields <lists at coryfields dot com>]

Quick disclaimer: I'm 100% new to GCC code and the dev process, so
there are bound to be some faulty assumptions below.

I recently worked on a build of gcc, x86_64-pc-linux-gnu ->
x86_64-pc-linux-musl. In order to boost my confidence in musl, I
decided that I'd like to ensure that 3 (and 4) stage bootstraps
succeed and compare equal.

I quickly ran into failed object comparisons at stage3. The issue, as
it turned out, was that musl's qsort algorithm differs significantly
from gcc's, though both (as far as I can tell) are perfectly legal.
The c spec allows for different results in the cast of unstable
arrays.

So otherwise-identical gcc builds linked against differing libc's
potentially produce different binaries for the target. And at least in
the case of glibc/musl, I've verified that it's happening. Due to the
qsorts used in the gen* build programs, I assume it's even possible
for cross-compiled target code to vary based libc used by the native
system compiler that builds the cross-compiler.

After tracking down the issue, I set out to fix the unstable sorts.
This turned out to be tricky, given the dozens or hundreds of uses of
qsort in the code-base. As someone not familiar with the gcc code or
typical debugging procedures, my big-hammer solution was to copy
glibc's qsort implementation into musl, run both versions for every
call to qsort, and assert that the resulting arrays are equal. This
turned up dozens of failures, which I fixed case-by-case until a
bootstrap succeeded. I then removed the patches one-by-one to narrow
down exactly which ones were causing compilation differences. This
ended up only being 2 small changes for trunk, 4 for 7.2.x. I will
submit those to the -patches list as a follow-up.

More generally, many of these unstable sorts remain, and fixing them
up individually seems futile. Before working on this further, I'm
wondering if it makes sense to pull a qsort into libiberty and poison
the libc function. That way if unstable sorts do sneak in, at least
they would be consistently unstable.

Input would be much appreciated.

Regards,
Cory Fields