PATCH: PR target/24419: ix86 prologue clobbers memory when it shouldn't

[Jan Hubicka]

> On Wed, Oct 19, 2005 at 09:00:32PM +0200, Jan Hubicka wrote:
> > > On Wed, Oct 19, 2005 at 09:14:11AM -0700, H. J. Lu wrote:
> > > > I may have missed something. As I understand, the only problem with
> > > > 
> > > >         push %ebp
> > > >         movl %esp, %ebp
> > > >         leal -32(%ebp), %ecx
> > > >         movl $1, (%ecx)
> > > >         subl $100, %esp
> > > > 
> > > > is that signal handler may come in the middle of above and the stack
> > > > isn't protected. Am I correct?
> > > 
> > > Yes.  And you're stuck on the fact that I wrote 100 and -32,
> > > which are smaller than the redzone size.  Multiply the numbers
> > > by 10 if you must, it's still a problem.
> > 
> > I also believe that current code is correct and clobber is neccesary in
> > general case.  We probably can work bit harder and avoid the clobber on
> > when whole stack frame fits in red zone to allow better scheduling but I
> > doubt it matters at all.
> > 
> > Concerning my original comment on push instructions in PR.. I believe it
> > is moot - only case we can screw up is convertion THE sub instruction
> > appearing in between register save and end of prologue to push sequence.
> > This should never happen as prologue generation should pick up push
> > instructions at first place when it is profitable...
> > 
> 
> We are investigating the impacts of different optimizations. This
> condition may happen when red zone is enabled, x86_prologue_using_move
> is on and x86_sub_esp_X is off. Are they allowed? If not, are there
> any checks against it?
There is no explicit check against.  x86_prologue_using_move should be
on on CPUs where push is slower than memory write, x86_sub_esp_X on CPUs
where push is comparable to arithmetic on ESP that is stronger
requirement so the configuration you are testing is somewhat insane.

If you want to experiment with this, you can teach ix86_compute_frame_layout
to not set save_regs_using_mov when the stack adjustment needed is
covered by x86_sub_esp_X flags so it won't trigger the missoptimizing
case.  (it would consistently result in better code generation too).

I believe either this or trap somewhere checking the conflicting
configuration would be good idea for mainline so people won't trigger
this again, but it is rth's call.

Honza
> 
> 
> H.J.