[Bug target/65369] [5 Regression] nettle test failure on powerpc64le-linux-gnu when built with -O3

["msebor at gcc dot gnu.org" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65369

--- Comment #7 from Martin Sebor <msebor at gcc dot gnu.org> ---
The cause of the failing tests observed on RHEL 7.1 is in the second definition
of nettle's HAVE_NATIVE_64_BIT configuration macro: 

$ grep HAVE_NATIVE_64_BIT config.*
config.h:# define HAVE_NATIVE_64_BIT 1
config.h:# define HAVE_NATIVE_64_BIT (SIZEOF_LONG * CHAR_BIT >= 64)

The macro relies on CHAR_BIT. Uses of HAVE_NATIVE_64_BIT of the form

  #if !HAVE_NATIVE_64_BIT

in files that don't include <limits.h> that defined CHAR_BIT end up compiling
the conditional blocks that are intended to be included only when the target
has no native support for 64 bit arithmetic.  One such file that comes into
play in the failing tests is camellia-absorb.c.  Since ppc64le obviously does
have such support, the code is unnecessary and, as it turns out, introduces
errors.

Once <limits.h> is included (I simply added the include directive to macros.h),
all of nettle's tests pass even on ppc64le-redhat-linux with the default -O2.

However, after rebuilding at -O3 I see the following failure:

hash input address: 0x10007410471
Got:

dd4111dd66913ae2 f39adfbc438214d6

Expected:

e33b4ddc9c38f219 9c3e7b164fcc0536
/src/nettle-3.0/run-tests: line 57: 57170 Aborted                 "$1"
$testflags
FAIL: md4

The failure disappears and is replaced by the errors below if I rebuild
everything with -fsanitize=address -fsanitize=bounds -fsanitize=undefined:

...
/src/nettle-3.0/blowfish.c:363:19: runtime error: left shift of 244 by 24
places cannot be represented in type 'int'
PASS: blowfish
...
/src/nettle-3.0/des.c:176:42: runtime error: index 42 out of bounds for type
'int8_t [26][4]'
PASS: des
/src/nettle-3.0/des.c:176:42: runtime error: index 52 out of bounds for type
'int8_t [26][4]'
PASS: des3
/src/nettle-3.0/des.c:176:42: runtime error: index 35 out of bounds for type
'int8_t [26][4]'
PASS: des-compat
...
/src/nettle-3.0/twofish.c:200:54: runtime error: left shift of 184 by 24 places
cannot be represented in type 'int'
PASS: twofish