[Bug other/78653] New: badly optimized kernel code with -fsanitize=object-size -fsanitize=null

[clyon at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78653

            Bug ID: 78653
           Summary: badly optimized kernel code with
                    -fsanitize=object-size -fsanitize=null
           Product: gcc
           Version: 7.0
               URL: https://bugs.linaro.org/show_bug.cgi?id=2354
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: other
          Assignee: unassigned at gcc dot gnu.org
          Reporter: clyon at gcc dot gnu.org
  Target Milestone: ---
            Target: arm

As described in the original bug report
(https://bugs.linaro.org/show_bug.cgi?id=2354)
parts of the Linux kernel fail to build after enabling UBSAN.

With the sample attached testcase, a simple function gets expanded to a very
complex expression. The original source for the function is

static void zl10353_calc_nominal_rate(struct dvb_frontend *fe,
                                      u32 bandwidth,
                                      u16 *nominal_rate)
{
        struct zl10353_state *state = fe->demodulator_priv;
        u32 adc_clock = 450560; /* 45.056 MHz */
        u64 value;
        u8 bw = bandwidth / 1000000;

        if (state->config.adc_clock)
                adc_clock = state->config.adc_clock;

        value = (u64)10 * (1 << 23) / 7 * 125;
        value = (bw * value) + adc_clock / 2;
        do_div(value, adc_clock);
        *nominal_rate = value;

        dprintk("%s: bw %d, adc_clock %d => 0x%x\n",
                __func__, bw, adc_clock, *nominal_rate);
}

The key here appears to be the setting of adc_clock, which the compiler knows
is not zero, but it doesn't know the actual value of.

The do_div() macro in turn is a complex way of turning a constant 64-bit
division into a multiplication, for reference the non-preprocessed version is
at
http://lxr.free-electrons.com/source/include/asm-generic/div64.h?v=4.6
The __builtin_constant_p() here should guard code that only makes sense for a
constant input:

208         if (__builtin_constant_p(__base) &&             \
209             is_power_of_2(__base)) {                    \
210                 __rem = (n) & (__base - 1);             \
211                 (n) >>= ilog2(__base);                  \

however, the ____ilog2_NaN symbol is referenced here:

#define ilog2(n)                                \
(                                               \
        __builtin_constant_p(n) ? (             \
                (n) < 1 ? ____ilog2_NaN() :     \
                (n) & (1ULL << 63) ? 63 :       \
                (n) & (1ULL << 62) ? 62 :       \
                (n) & (1ULL << 61) ? 61 :       \
                (n) & (1ULL << 60) ? 60 :       \
...
                (n) & (1ULL <<  1) ?  1 :       \
                (n) & (1ULL <<  0) ?  0 :       \
                ____ilog2_NaN()                 \
                                   ) :          \
        (sizeof(n) <= 4) ?                      \
        __ilog2_u32(n) :                        \
        __ilog2_u64(n)                          \
 )

after 'n' has been determined to be constant yet not known whether
it is less than '1', which mathematically makes no sense.