mudflap problem

[Doug Graham]

On Tue, Sep 28, 2004 at 05:07:53PM -0400, Frank Ch. Eigler wrote:
> 
> Herman ten Brugge <hermantenbrugge@home.nl> writes:
> 
> > I tried mudflap on some test programs and found that the following
> > program works:
> > [...]
> > When compiled with -fmudflap the code runs with no error. In the memory map
> > the array b is before array a. So the code above first fills array b
> > with 0 and then a.
> 
> This should not be happening, unless some intermediate compiler pass
> is lowering the ARRAY_REF (b[i]) to a plain INDIRECT_REF (* b+i) type
> expression.  (If so, the instrumentation loses its limited awareness
> of where the pointer value came from, and checks only that it refers
> to *some* valid object.)  This sort of lowering transform should
> likely be disabled in the -fmudflap case.  (It's possible that there
> is a bug elsewhere instead.)

My apologies if this is a question that would better be asked elsewhere.

Herman's question and your response got me to wondering what sort of
things the bounds-checking patches that Herman currently maintains would
catch that mudflap would not.  I know that all pointer arithmetic is
checked in his patches, and I'm fairly sure that mudflap doesn't check any
pointer arith, so I thought that there would be plenty of cases similar
(but not quite the same) to the one that he is asking about that would not
be caught by mudflap.  But then I tried this:

    int a[16];
    int b[16];
    
    static void foo(int *x)
    {
            *x = 42;
    }
    
    int main()
    {
            foo(b+16);
    }

I thought that since b+16 points to the base of a, mudflap would not
complain about the dereference in foo().  But it did somehow know that
the pointer passed in to foo() was derived from 'b' rather than 'a'.
OTOH, if I call foo(a) instead, which means that I'm passing exactly
the same pointer value to foo(), mudflap does not complain.

How did mudflap know in foo(), without using fat pointers, that the
pointer was derived from 'b'?  Herman's bounds-checking patches can catch
this as well, but they do so by adding some pad bytes between 'a' and 'b',
so that b+16 does not point to anything that can be legally dereferenced.

But then I changed main so that I could test both cases from the same
executable:

    int main(int argc, char **argv)
    {
        if (argc > 1)
                foo(b+16);
        else
                foo(a);
    }

and mudflap didn't catch the b+16 dereference???

My original intent was to suggest that maybe mudflap could (perhaps
optionally) check pointer arithmetic as well, but now I'm not sure
what advantages that would offer.

BTW, I'm using the 20040627 snapshot of GCC 3.5 (which doesn't exhibit
the problem that Herman mentioned).

--Doug.