First of all, many apologies for not producing a small test case - I couldn't find a simple example to demonstrate the issue. The problem is that in_csum() (in lib/utils.c) returns an incorrect value when the first call of in_csum() is inlined in vrrp_build_vrrp_v3() (in keepalived/vrrp/vrrp.c). It appears then when the first call of in_csum() is inlined, the wrong value of the addr parameter is used. This can be observed with the following log output: Calling in_csum, 0x10a 0x300 added 0x5629 to get 0x5629 added 0x0 to get 0x5629 added 0x903d to get 0xe666 added 0x5c2e to get 0x14294 added 0x5629 to get 0x198bd added 0x1000 to get 0x1a8bd Calling in_csum, 0x3331 0x265 added 0x3331 to get 0x1dbee added 0x265 to get 0x1de53 added 0x6400 to get 0x24253 added 0x0 to get 0x24253 added 0x7f to get 0x242d2 added 0xf00 to get 0x251d2 added 0x2d15 to get 0x27ee7 added 0x2be to get 0x281a5 The two 16 bit values printed after "Calling in_csum" are the first two values that should be included in the checksum calculation, and the values after added are the values being added to the checksum. In the first call of in_csum (inlined) the first two values at 0x10 and 0x300, but the first two values output in in_csum() do not match. In the second call to in_csum() (not inlined), it can be seen that the values match. I have tried changing log_message(LOG_INFO, "added 0x%x to get 0x%x", addr[-1], csum); to log_message(LOG_INFO, "addr %p added 0x%x to get 0x%x", addr, addr[-1], csum); in in_csum() to see what value is being used for addr, but doing so stops the problem occurring. When LTO is not enabled, the equivalent to to debug output above is a follows (and these are the correct values): Calling in_csum, 0x10a 0x300 added 0x10a to get 0x10a added 0x300 to get 0x40a added 0x10a to get 0x514 added 0x100 to get 0x614 added 0x7000 to get 0x7614 added 0x1000 to get 0x8614 Calling in_csum, 0x3331 0x265 added 0x3331 to get 0xb945 added 0x265 to get 0xbbaa added 0x6400 to get 0x11faa added 0x0 to get 0x11faa added 0x7f to get 0x12029 added 0xf00 to get 0x12f29 added 0x2d15 to get 0x15c3e added 0x2be to get 0x15efc This problem has been seen with GCC 11.2 (Ubuntu 22.04), GCC 12.1.1 (Fedora 36), GCC 11.3.1 (Fedora 34), GCC 12.1.1 (Fedora 37) and GCC 12.2.1 (Fedora 38 - Rawhide). The problem does not occur when LTO is not enabled, nor with clang (with or without LTO). The problem also does not occur if in_csum() has __attribute__((noinline)). The output below and attached files were generated on Fedora 34 using gcc 11.3.1. The attached tar file contains the .i files and make.log, the full log of building keepalived. The output of gcc -v -save-temps when compiling lib/utils.c is: Using built-in specs. COLLECT_GCC=gcc OFFLOAD_TARGET_NAMES=nvptx-none OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-11.3.1-20220421/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 11.3.1 20220421 (Red Hat 11.3.1-2) (GCC) COLLECT_GCC_OPTIONS='-D' 'HAVE_CONFIG_H' '-I' '.' '-D' '_GNU_SOURCE' '-v' '-save-temps' '-D' '_GNU_SOURCE' '-g' '-g' '-O2' '-flto=auto' '-v' '-save-temps' '-Wall' '-Wextra' '-Wunused' '-Wstrict-prototypes' '-Wabsolute-value' '-Waddress-of-packed-member' '-Walloca' '-Walloc-zero' '-Warith-conversion' '-Warray-bounds=2' '-Wattribute-alias=2' '-Wbad-function-cast' '-Wc11-c2x-compat' '-Wcast-align' '-Wcast-qual' '-Wdate-time' '-Wdisabled-optimization' '-Wdouble-promotion' '-Wduplicated-branches' '-Wduplicated-cond' '-Wfloat-conversion' '-Wfloat-equal' '-Wformat-overflow=1' '-Wformat-signedness' '-Wformat-truncation=1' '-Wframe-larger-than=5120' '-Wimplicit-fallthrough=3' '-Winit-self' '-Winline' '-Winvalid-pch' '-Wjump-misses-init' '-Wlogical-op' '-Wmissing-declarations' '-Wmissing-field-initializers' '-Wmissing-include-dirs' '-Wmissing-prototypes' '-Wnested-externs' '-Wnormalized=nfc' '-Wnull-dereference' '-Wold-style-definition' '-Woverlength-strings' '-Wpointer-arith' '-Wredundant-decls' '-Wshadow' '-Wshift-overflow=2' '-Wstack-protector' '-Wstrict-overflow=4' '-Wstringop-overflow=2' '-Wstringop-truncation' '-Wsuggest-attribute=cold' '-Wsuggest-attribute=format' '-Wsuggest-attribute=malloc' '-Wsuggest-attribute=noreturn' '-Wsuggest-attribute=pure' '-Wsync-nand' '-Wtrampolines' '-Wundef' '-Wuninitialized' '-Wunknown-pragmas' '-Wunsuffixed-float-constants' '-Wunused-const-variable=2' '-Wvariadic-macros' '-Wwrite-strings' '-fPIE' '-Wformat=1' '-Werror=format-security' '-fexceptions' '-fstack-protector-strong' '--param=ssp-buffer-size=4' '-grecord-gcc-switches' '-O2' '-g' '-O2' '-flto=auto' '-ffat-lto-objects' '-MT' 'utils.o' '-MD' '-MP' '-MF' '.deps/utils.Tpo' '-c' '-o' 'utils.o' '-mtune=generic' '-march=x86-64' /usr/libexec/gcc/x86_64-redhat-linux/11/cc1 -E -quiet -v -v -I . -MD utils.d -MF .deps/utils.Tpo -MP -MT utils.o -D HAVE_CONFIG_H -D _GNU_SOURCE -D _GNU_SOURCE -D_FORTIFY_SOURCE=2 utils.c -mtune=generic -march=x86-64 -Wall -Wextra -Wunused -Wstrict-prototypes -Wabsolute-value -Waddress-of-packed-member -Walloca -Walloc-zero -Warith-conversion -Warray-bounds=2 -Wattribute-alias=2 -Wbad-function-cast -Wc11-c2x-compat -Wcast-align -Wcast-qual -Wdate-time -Wdisabled-optimization -Wdouble-promotion -Wduplicated-branches -Wduplicated-cond -Wfloat-conversion -Wfloat-equal -Wformat-overflow=1 -Wformat-signedness -Wformat-truncation=1 -Wframe-larger-than=5120 -Wimplicit-fallthrough=3 -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical-op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs -Wnormalized=nfc -Wnull-dereference -Wold-style-definition -Woverlength-strings -Wpointer-arith -Wredundant-decls -Wshadow -Wshift-overflow=2 -Wstack-protector -Wstrict-overflow=4 -Wstringop-overflow=2 -Wstringop-truncation -Wsuggest-attribute=cold -Wsuggest-attribute=format -Wsuggest-attribute=malloc -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wsync-nand -Wtrampolines -Wundef -Wuninitialized -Wunknown-pragmas -Wunsuffixed-float-constants -Wunused-const-variable=2 -Wvariadic-macros -Wwrite-strings -Wformat=1 -Werror=format-security -flto=auto -fPIE -fexceptions -fstack-protector-strong -flto=auto -ffat-lto-objects -g -g -grecord-gcc-switches -g -fworking-directory -O2 -O2 -O2 -fpch-preprocess -o utils.i ignoring nonexistent directory "/usr/lib/gcc/x86_64-redhat-linux/11/include-fixed" ignoring nonexistent directory "/usr/lib/gcc/x86_64-redhat-linux/11/../../../../x86_64-redhat-linux/include" #include "..." search starts here: #include <...> search starts here: . /usr/lib/gcc/x86_64-redhat-linux/11/include /usr/local/include /usr/include End of search list. COLLECT_GCC_OPTIONS='-D' 'HAVE_CONFIG_H' '-I' '.' '-D' '_GNU_SOURCE' '-v' '-save-temps' '-D' '_GNU_SOURCE' '-g' '-g' '-O2' '-flto=auto' '-v' '-save-temps' '-Wall' '-Wextra' '-Wunused' '-Wstrict-prototypes' '-Wabsolute-value' '-Waddress-of-packed-member' '-Walloca' '-Walloc-zero' '-Warith-conversion' '-Warray-bounds=2' '-Wattribute-alias=2' '-Wbad-function-cast' '-Wc11-c2x-compat' '-Wcast-align' '-Wcast-qual' '-Wdate-time' '-Wdisabled-optimization' '-Wdouble-promotion' '-Wduplicated-branches' '-Wduplicated-cond' '-Wfloat-conversion' '-Wfloat-equal' '-Wformat-overflow=1' '-Wformat-signedness' '-Wformat-truncation=1' '-Wframe-larger-than=5120' '-Wimplicit-fallthrough=3' '-Winit-self' '-Winline' '-Winvalid-pch' '-Wjump-misses-init' '-Wlogical-op' '-Wmissing-declarations' '-Wmissing-field-initializers' '-Wmissing-include-dirs' '-Wmissing-prototypes' '-Wnested-externs' '-Wnormalized=nfc' '-Wnull-dereference' '-Wold-style-definition' '-Woverlength-strings' '-Wpointer-arith' '-Wredundant-decls' '-Wshadow' '-Wshift-overflow=2' '-Wstack-protector' '-Wstrict-overflow=4' '-Wstringop-overflow=2' '-Wstringop-truncation' '-Wsuggest-attribute=cold' '-Wsuggest-attribute=format' '-Wsuggest-attribute=malloc' '-Wsuggest-attribute=noreturn' '-Wsuggest-attribute=pure' '-Wsync-nand' '-Wtrampolines' '-Wundef' '-Wuninitialized' '-Wunknown-pragmas' '-Wunsuffixed-float-constants' '-Wunused-const-variable=2' '-Wvariadic-macros' '-Wwrite-strings' '-fPIE' '-Wformat=1' '-Werror=format-security' '-fexceptions' '-fstack-protector-strong' '--param=ssp-buffer-size=4' '-grecord-gcc-switches' '-O2' '-g' '-O2' '-flto=auto' '-ffat-lto-objects' '-MT' 'utils.o' '-MD' '-MP' '-MF' '.deps/utils.Tpo' '-c' '-o' 'utils.o' '-mtune=generic' '-march=x86-64' /usr/libexec/gcc/x86_64-redhat-linux/11/cc1 -fpreprocessed utils.i -quiet -dumpbase utils.c -dumpbase-ext .c -mtune=generic -march=x86-64 -g -g -grecord-gcc-switches -g -O2 -O2 -O2 -Wall -Wextra -Wunused -Wstrict-prototypes -Wabsolute-value -Waddress-of-packed-member -Walloca -Walloc-zero -Warith-conversion -Warray-bounds=2 -Wattribute-alias=2 -Wbad-function-cast -Wc11-c2x-compat -Wcast-align -Wcast-qual -Wdate-time -Wdisabled-optimization -Wdouble-promotion -Wduplicated-branches -Wduplicated-cond -Wfloat-conversion -Wfloat-equal -Wformat-overflow=1 -Wformat-signedness -Wformat-truncation=1 -Wframe-larger-than=5120 -Wimplicit-fallthrough=3 -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical-op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs -Wnormalized=nfc -Wnull-dereference -Wold-style-definition -Woverlength-strings -Wpointer-arith -Wredundant-decls -Wshadow -Wshift-overflow=2 -Wstack-protector -Wstrict-overflow=4 -Wstringop-overflow=2 -Wstringop-truncation -Wsuggest-attribute=cold -Wsuggest-attribute=format -Wsuggest-attribute=malloc -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wsync-nand -Wtrampolines -Wundef -Wuninitialized -Wunknown-pragmas -Wunsuffixed-float-constants -Wunused-const-variable=2 -Wvariadic-macros -Wwrite-strings -Wformat=1 -Werror=format-security -version -flto=auto -fPIE -fexceptions -fstack-protector-strong -flto=auto -ffat-lto-objects --param=ssp-buffer-size=4 -o utils.s GNU C17 (GCC) version 11.3.1 20220421 (Red Hat 11.3.1-2) (x86_64-redhat-linux) compiled by GNU C version 11.3.1 20220421 (Red Hat 11.3.1-2), GMP version 6.2.0, MPFR version 4.1.0-p13, MPC version 1.2.1, isl version isl-0.18-GMP GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 GNU C17 (GCC) version 11.3.1 20220421 (Red Hat 11.3.1-2) (x86_64-redhat-linux) compiled by GNU C version 11.3.1 20220421 (Red Hat 11.3.1-2), GMP version 6.2.0, MPFR version 4.1.0-p13, MPC version 1.2.1, isl version isl-0.18-GMP GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 Compiler executable checksum: 2796d05d4bc35faeaab7ddfb73cae85e COLLECT_GCC_OPTIONS='-D' 'HAVE_CONFIG_H' '-I' '.' '-D' '_GNU_SOURCE' '-v' '-save-temps' '-D' '_GNU_SOURCE' '-g' '-g' '-O2' '-flto=auto' '-v' '-save-temps' '-Wall' '-Wextra' '-Wunused' '-Wstrict-prototypes' '-Wabsolute-value' '-Waddress-of-packed-member' '-Walloca' '-Walloc-zero' '-Warith-conversion' '-Warray-bounds=2' '-Wattribute-alias=2' '-Wbad-function-cast' '-Wc11-c2x-compat' '-Wcast-align' '-Wcast-qual' '-Wdate-time' '-Wdisabled-optimization' '-Wdouble-promotion' '-Wduplicated-branches' '-Wduplicated-cond' '-Wfloat-conversion' '-Wfloat-equal' '-Wformat-overflow=1' '-Wformat-signedness' '-Wformat-truncation=1' '-Wframe-larger-than=5120' '-Wimplicit-fallthrough=3' '-Winit-self' '-Winline' '-Winvalid-pch' '-Wjump-misses-init' '-Wlogical-op' '-Wmissing-declarations' '-Wmissing-field-initializers' '-Wmissing-include-dirs' '-Wmissing-prototypes' '-Wnested-externs' '-Wnormalized=nfc' '-Wnull-dereference' '-Wold-style-definition' '-Woverlength-strings' '-Wpointer-arith' '-Wredundant-decls' '-Wshadow' '-Wshift-overflow=2' '-Wstack-protector' '-Wstrict-overflow=4' '-Wstringop-overflow=2' '-Wstringop-truncation' '-Wsuggest-attribute=cold' '-Wsuggest-attribute=format' '-Wsuggest-attribute=malloc' '-Wsuggest-attribute=noreturn' '-Wsuggest-attribute=pure' '-Wsync-nand' '-Wtrampolines' '-Wundef' '-Wuninitialized' '-Wunknown-pragmas' '-Wunsuffixed-float-constants' '-Wunused-const-variable=2' '-Wvariadic-macros' '-Wwrite-strings' '-fPIE' '-Wformat=1' '-Werror=format-security' '-fexceptions' '-fstack-protector-strong' '--param=ssp-buffer-size=4' '-grecord-gcc-switches' '-O2' '-g' '-O2' '-flto=auto' '-ffat-lto-objects' '-MT' 'utils.o' '-MD' '-MP' '-MF' '.deps/utils.Tpo' '-c' '-o' 'utils.o' '-mtune=generic' '-march=x86-64' as -v -v -I . --gdwarf-5 --64 -o utils.o utils.s GNU assembler version 2.35.2 (x86_64-redhat-linux) using BFD version version 2.35.2-6.fc34 COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/11/:/usr/libexec/gcc/x86_64-redhat-linux/11/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/11/:/usr/lib/gcc/x86_64-redhat-linux/ LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/11/:/usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/11/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS='-D' 'HAVE_CONFIG_H' '-I' '.' '-D' '_GNU_SOURCE' '-v' '-save-temps' '-D' '_GNU_SOURCE' '-g' '-g' '-O2' '-flto=auto' '-v' '-save-temps' '-Wall' '-Wextra' '-Wunused' '-Wstrict-prototypes' '-Wabsolute-value' '-Waddress-of-packed-member' '-Walloca' '-Walloc-zero' '-Warith-conversion' '-Warray-bounds=2' '-Wattribute-alias=2' '-Wbad-function-cast' '-Wc11-c2x-compat' '-Wcast-align' '-Wcast-qual' '-Wdate-time' '-Wdisabled-optimization' '-Wdouble-promotion' '-Wduplicated-branches' '-Wduplicated-cond' '-Wfloat-conversion' '-Wfloat-equal' '-Wformat-overflow=1' '-Wformat-signedness' '-Wformat-truncation=1' '-Wframe-larger-than=5120' '-Wimplicit-fallthrough=3' '-Winit-self' '-Winline' '-Winvalid-pch' '-Wjump-misses-init' '-Wlogical-op' '-Wmissing-declarations' '-Wmissing-field-initializers' '-Wmissing-include-dirs' '-Wmissing-prototypes' '-Wnested-externs' '-Wnormalized=nfc' '-Wnull-dereference' '-Wold-style-definition' '-Woverlength-strings' '-Wpointer-arith' '-Wredundant-decls' '-Wshadow' '-Wshift-overflow=2' '-Wstack-protector' '-Wstrict-overflow=4' '-Wstringop-overflow=2' '-Wstringop-truncation' '-Wsuggest-attribute=cold' '-Wsuggest-attribute=format' '-Wsuggest-attribute=malloc' '-Wsuggest-attribute=noreturn' '-Wsuggest-attribute=pure' '-Wsync-nand' '-Wtrampolines' '-Wundef' '-Wuninitialized' '-Wunknown-pragmas' '-Wunsuffixed-float-constants' '-Wunused-const-variable=2' '-Wvariadic-macros' '-Wwrite-strings' '-fPIE' '-Wformat=1' '-Werror=format-security' '-fexceptions' '-fstack-protector-strong' '--param=ssp-buffer-size=4' '-grecord-gcc-switches' '-O2' '-g' '-O2' '-flto=auto' '-ffat-lto-objects' '-MT' 'utils.o' '-MD' '-MP' '-MF' '.deps/utils.Tpo' '-c' '-o' 'utils.o' '-mtune=generic' '-march=x86-64' '-dumpdir' 'utils.'
Created attachment 53486 [details] Tar file containing the generated .i files and make.log
This is an obvious C/C++ aliasing violation. ipv4_phdr_t ipv4_phdr; ... in_csum(({ ( uint16_t *) (&ipv4_phdr); }), sizeof(ipv4_phdr), 0, &vrrp->ipv4_csum); inside in_csum: uint16_t in_csum(const uint16_t *addr, size_t len, uint32_t csum, uint32_t *acc) .... while (nleft > 1) { csum += *addr++; You need to use either -fno-strict-aliasing or you need to mark the argument addr in in_csum as may_alias.