[PATCH] C undefined behavior fix

[Linus Torvalds]

On Sat, 5 Jan 2002, mike stump wrote:
>
> Questions, does using the inline asm trick fix the problem?  Is there
> more than one place like RELOC where this can happen?  If so, how many
> places?

Well, the RELOC per se is not really what I worry about - that code was
ugly in the first place, and there were multiple work-arounds. There's
one that appears to just clean up the code, so that's fine.

What I worry about is that gcc generally is moving in the direction of
_other_ languages, where issues like these really don't exist all that
much, simply because they aren't used to the same degree to make low-level
coding where it really _is_ reasonable to know more about the machine than
the compiler will ever do.

So people make these optimizations that simply aren't sensible for any
real program, and then they ignore those _other_ real programs that really
do care about the origins of C as a low-level "portable assembly
language".

And THAT makes me nervous. The fact is that the Linux kernel cares
_deeply_ about a lot of "implementation-defined" things, and that isn't
likely to change. If the people making optimizations in gcc do not care
about the historic meaning of these constructs, that screws the kernel
over.

And the problem with optimizations like these is that they don't always
show themselves clearly. The ones people _know_ about are the easy ones.
The ones that happen and could cause things like security problems (while
normal users may never notice anything, so the problem goes unnoticed for
a long time), _those_ are the ones that worry me a lot.

In short: gcc developers not caring about low-level people is worrysome to
me. gcc developers doing non-sensical optimizations that can never be
helpful for anybody is a _really_ bad sign in my opinion.

The fact is, that particular optimization simply does not make sense. Even
if the program is "undefined", it has a strightforward meaning that would
have come out of the compiler if there wasn't an optimization that was
nonsensical.

See? _I_ worry about the fact that no gcc people seemed to care.

(Bad analogy time: you may have a car that runs smoothly and has never had
any problems. You would _still_ want to feel like if something came up,
the dealership would at least try to be understanding, even if it was
because you rode your car into wildernesses that could damage the shock
absorbers instead of just using it on paved roads. No?)

> I'm trying to envision changes to the compiler that we could put in,
> to make gcc a better compiler for users and help solve this problem.

I've always had this personal belief that pointer casts should actually
have meaning on a code generation level too, and that the perfect
behaviour for an access through a pointer cast would be to disable
optimizations. I've argued for that before, when we had a small spat about
aliasing information.

I do agree that the problem can be solved with a "asm()" statement. The
kernel already uses them quite extensively for some things like this: we
have the notion of a "barrier()", which is when there are some external
ordering constraints (usually due to multiple threads) that force things
to be stable in memory. That's done with a simple

	asm("": : :"memory");

The main problem with asms is that they do get quite ugly rather quickly,
_especially_ when used as part of expressions. It's one thing to do

	#define barrier() \
		asm("": : :"memory")

and another to do

	#define hide(x) ({ typeof(x) y; \
		asm("# hide %0":"=g" (y):"0" (x)); \
		y; })

which is not only really ugly, but also generates ugly assembly output (it
generates another pseudo for the same thing, and because gcc doesn't know
that the _value_ is the same as the old one, it will increase register
pressure etc for no good reason. I can send out examples of this on
demand).

In contrast, if gcc had a way that didn't hide the equality of the
_value_, but hid just the "associated" information, that would be much
better.

And that is, from a conceptual standpoint, what a cast to an integer type
would do - which is why it would be very attractive to me to see

	#define hide(x)	((void *)(unsigned long)(x))

that would drop the "provenance", without dropping the fact that the value
stays the same. Trust me, it does matter for the resulting code.

> I am unsure exactly how to bound these down into small cases.  Does
> anyone see of ways to bound them down into small numbers of cases?
> Would it be reasonable for the compiler to only know about a finitely
> small number of such cases?

I really think the only interesting cases are the ptr->int, int->ptr
casts. Both are defined by the C standard to be implementaion-defined
(which certainly allows the implementation to "drop" the knowledge the
compiler might have about the pointer), and neither is used in any
reasonable code that expects aliasing optimizations etc with the pointers.

		Linus