How to fix GCC Upgrade issues(which always disappeared by add some trace prints)?

[Zhu, Siqiang (NSB - CN/Nanjing)]

Hello GCC experts,

Anyone can guide us to fix this issue on GCC v9.3?

                Our products used ARM64 target cross-compile GCC v6.5 before. In recent weeks, we upgrade it to GCC v9.3 by company request.

                However, we met some strange problems, such as

  1.  Linux eMMC driver met CRC error when booting up.
  2.  Linux some FPGA driver met structure member assignment failed when booting up.
  3.  OPTEE some module access registers error.

All of these problem can disappeared if we add some trace prints around the issue code blocks, but any code change may re-trigger them. ☹
If we try to reuse GCC v6.5 to compile these modules, these issues cannot be reproduced any more.

                The GCC v9.3 configuration shows below:
                $ ./aarch64-poky-linux-gcc-9.3.0 -v
Using built-in specs.
COLLECT_GCC=./aarch64-poky-linux-gcc-9.3.0
COLLECT_LTO_WRAPPER=/var/fpwork/siqianz/remote/nsn-yocto/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/9.3.0-r0/sysroot-destdir/var/fpwork/siqianz/remote/nsn-yocto/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/9.3.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/../../libexec/aarch64-poky-linux/gcc/aarch64-poky-linux/9.3.0/lto-wrapper
Target: aarch64-poky-linux
Configured with: ../../../../../../work-shared/gcc-9.3.0-r0/gcc-9.3.0/configure --build=x86_64-linux --host=x86_64-linux --target=aarch64-poky-linux --prefix=/host-native/usr --exec_prefix=/host-native/usr --bindir=/host-native/usr/bin/aarch64-poky-linux --sbindir=/host-native/usr/bin/aarch64-poky-linux --libexecdir=/host-native/usr/libexec/aarch64-poky-linux --datadir=/host-native/usr/share --sysconfdir=/host-native/etc --sharedstatedir=/host-native/com --localstatedir=/host-native/var --libdir=/host-native/usr/lib/aarch64-poky-linux --includedir=/host-native/usr/include --oldincludedir=/host-native/usr/include --infodir=/host-native/usr/share/info --mandir=/host-native/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/host-native --enable-clocale=generic --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --disable-multilib --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=aarch64-poky-linux- --without-local-prefix --enable-lto --disable-libssp --enable-libitm --disable-bootstrap --disable-libmudflap --with-system-zlib --with-linker-hash-style=sysv --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-gxx-include-dir=/not/exist/usr/include/c++/9.3.0 --with-sysroot=/not/exist --with-build-sysroot=/host --enable-poison-system-directories --with-system-zlib --disable-static --disable-nls --with-glibc-version=2.28 --enable-initfini-array --enable-__cxa_atexit : (reconfigured) ../../../../../../work-shared/gcc-9.3.0-r0/gcc-9.3.0/configure --build=x86_64-linux --host=x86_64-linux --target=aarch64-poky-linux --prefix=/host-native/usr --exec_prefix=/host-native/usr --bindir=/host-native/usr/bin/aarch64-poky-linux --sbindir=/host-native/usr/bin/aarch64-poky-linux --libexecdir=/host-native/usr/libexec/aarch64-poky-linux --datadir=/host-native/usr/share --sysconfdir=/host-native/etc --sharedstatedir=/host-native/com --localstatedir=/host-native/var --libdir=/host-native/usr/lib/aarch64-poky-linux --includedir=/host-native/usr/include --oldincludedir=/host-native/usr/include --infodir=/host-native/usr/share/info --mandir=/host-native/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/host-native --enable-clocale=generic --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --disable-multilib --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=aarch64-poky-linux- --without-local-prefix --enable-lto --disable-libssp --enable-libitm --disable-bootstrap --disable-libmudflap --with-system-zlib --with-linker-hash-style=sysv --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-gxx-include-dir=/not/exist/usr/include/c++/9.3.0 --with-sysroot=/not/exist --with-build-sysroot=/host --enable-poison-system-directories --with-system-zlib --disable-static --disable-nls --with-glibc-version=2.28 --enable-initfini-array --enable-__cxa_atexit
Thread model: posix
gcc version 9.3.0 (GCC)

                The previous gcc v6.5 configuration is:
                $ ./aarch64-poky-linux-gcc-6.5.0 -v
Using built-in specs.
COLLECT_GCC=./aarch64-poky-linux-gcc-6.5.0
COLLECT_LTO_WRAPPER=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/sysroot-destdir/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/../../libexec/aarch64-poky-linux/gcc/aarch64-poky-linux/6.5.0/lto-wrapper
Target: aarch64-poky-linux
Configured with: ../../../../../../work-shared/gcc-6.5.0-r0/gcc-6.5.0/configure --build=x86_64-linux --host=x86_64-linux --target=aarch64-poky-linux --prefix=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr --exec_prefix=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr --bindir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux --sbindir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux --libexecdir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/libexec/aarch64-poky-linux --datadir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/share --sysconfdir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/etc --sharedstatedir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/com --localstatedir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/var --libdir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/lib/aarch64-poky-linux --includedir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/include --oldincludedir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/include --infodir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/share/info --mandir=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native --enable-clocale=generic --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --disable-multilib --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=aarch64-poky-linux- --without-local-prefix --enable-lto --disable-libssp gcc_cv_libc_provides_ssp=yes --enable-libitm --disable-bootstrap --disable-libmudflap --with-system-zlib --with-linker-hash-style=sysv --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-gxx-include-dir=/not/exist/usr/include/c++/6.5.0 --with-sysroot=/not/exist --with-build-sysroot=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot --without-long-double-128 --enable-poison-system-directories --with-mpfr=/var/fpwork/siqianz/remote/nsn-yocto-check/build_fzhub20/tmp/work/x86_64-linux/gcc-cross-aarch64/6.5.0-r0/recipe-sysroot-native/usr --with-system-zlib --disable-static --disable-nls --with-glibc-version=2.28 --enable-initfini-array --enable-__cxa_atexit
Thread model: posix
gcc version 6.5.0 (GCC)

                Thanks a lot!
                Best Regards!

                Simon.