Bug 11366

Summary: [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2
Product: gcc Reporter: Debian GCC Maintainers <debian-gcc>
Component: rtl-optimizationAssignee: Not yet assigned to anyone <unassigned>
Status: RESOLVED DUPLICATE    
Severity: critical CC: gcc-bugs, pme
Priority: P2 Keywords: wrong-code
Version: 3.3   
Target Milestone: 3.3.2   
Host: i386-linux / powerpc-linux Target: i386-linux / powerpc-linux
Build: i386-linux / powerpc-linux Known to work:
Known to fail: Last reconfirmed:

Description Debian GCC Maintainers 2003-06-29 09:19:22 UTC
This is not a complete bug report. Any hints and help to track down this report
are welcome.

[forwarded from http://bugs.debian.org/196090]

Rechecked with 3.3 CVS 20030627, with HEAD xfree86 does finish the build due to
some ICE. Setting the optimization to -O2 -fno-strict-aliasing doesn't help
either. Compiling with -O1 doesn't show the error. Seen at least on i386-linux
and powerpc-linux.

The symptom can be found in /var/log/xdm.log:
AUDIT: Fri Jun 27 20:28:48 2003: 18641 X: client 2 rejected from local host
  Auth name: XDM-AUTHORIZATION-1 ID: -1
Xlib: connection to ":0.0" refused by server
Xlib: Invalid XDM-AUTHORIZATION-1 key (failed key comparison)
/usr/X11R6/bin/xrdb: Can't open display ':0'


Bug submitter writes:

When testing XFree86 4.2.1-7 on my PowerPC I found that compiling the
tree with -O2 (per Debian Policy) and GCC 3.3 caused the X server to
break such that XDM-AUTHORIZATION-1 Xauth tokens could not be validated.

I have not narrowed this down.  I do know that using the 4.2.1-6 server
with the 4.2.1-7 libraries worked fine, so I suspect a miscompilation on
the server side, but it is conceivable that the problem is really on the
client/library side and that GCC 3.3 is exposing a very subtle error in
the code.  Or something like that.

Compiling with -O instead of -O2 made the problem go away, and that's
the optimization level that was used for the 4.2.1-7 packages for
PowerPC which were uploaded.

Anyway, I have heard rumors that the GCC team knows that there are
problems with this optimization level on PowerPC in GCC 3.3.  Something
about a bunch of new subtargets being added to the ppc/ELF backend in
this release?

Please nag me for more information if too much time passes and I have
not followed up.

Alternatively some person might be willing to track this down for me
using the recipe above.  Just take the 4.2.1-7 source package and remove
the special case optimization stepdown for powerpc from the debian/rules
file.

Compile XFree86, install the resulting packages, and use xdm to
(attempt) to start a session.  You'll also notice that the bustage keeps
the xdm greeter from working properly.


Jack Howard adds:
   Franz Sirl, the ppclinux devtool maintainer, has kindly
built the current rawhide XFree86 4.3.0 srpms against 
gcc-3.3 from 6/18/03 on Yellow Dog Linux. He is unable to
reproduce the authentication problems we see on debian
under either xdm or kdm. Perhaps we build XFree86 differently
enough from RedHat that only we tickled the gcc bug. However
I did find out that the gcc 3.3 that RedHat uses is not
a patched version of the gcc 3.3 branch but rather a special
rh branch of gcc-3.3. One wonders if they would see the
problem if they used the stock gcc-3.3 branch instead and
if there are any fixes trapped in the gcc-3.3 rh branch that
haven't been put into the main gcc-3.3 branch.
Comment 1 Volker Reichelt 2003-07-04 11:38:33 UTC
Might be related to PR 11198.
Comment 2 Matthias Klose 2003-07-09 07:31:02 UTC
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

reichelt at gcc dot gnu dot org writes:
> Might be related to PR 11198.

No. Rechecked with 3.3 CVS 20030708.
Comment 3 Franz Sirl 2003-07-14 18:17:01 UTC
Out of interest, since you are still using XFree86-4.2.1, have you made sure 
you compiled XFree86 with -fno-strict-aliasing, a thing that is done 
automatically since 4.3.0? 
 
Comment 4 james 2003-07-14 18:44:04 UTC
Subject: Re:  [3.3 regression] miscompiles
 XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

"sirl at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:

> ------- Additional Comments From sirl at gcc dot gnu dot org  2003-07-14 18:17 -------
> Out of interest, since you are still using XFree86-4.2.1, have you made sure 
> you compiled XFree86 with -fno-strict-aliasing, a thing that is done 
> automatically since 4.3.0? 

Yes, I tried this; it doesn't help.

Comment 5 Franz Sirl 2003-07-14 19:14:04 UTC
I have no other idea then, 4.2 already had -fno-merge-constants and I can't 
think of another promising option to try. Maybe -fno-strength-reduce? 
Otherwise you have to narrow down the bugreport to a single miscompiled file, 
we won't be able to help you otherwise. 
 
Comment 6 Matthias Klose 2003-07-15 05:46:07 UTC
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

sirl at gcc dot gnu dot org writes:
> I have no other idea then, 4.2 already had -fno-merge-constants and
> I can't think of another promising option to try. Maybe
> -fno-strength-reduce? Otherwise you have to narrow down the
> bugreport to a single miscompiled file, we won't be able to help you
> otherwise.

Daniel, Branden,

I see, that on Debian only -O2 is used, not any of the mentioned
flags.
Comment 7 Andrew Pinski 2003-07-16 16:08:15 UTC
Can you also try with -fno-gcse, since there was a bug in gcse where it would miscompile 
a bitfields.
Comment 8 Phil Edwards 2003-07-17 02:52:55 UTC
Ideally, what we'd really need from the Debian team are:

1) the file that gets miscompiled, and
2) the -f flag which causes it

(1) is essential.  (2) would be nice but not crucial.  The "easy" way to
find (2) is to compile with "-fFOO" where FOO is one of the flags
turned on by -O2, as listed in the manual.  Iterate over the list until
you find the FOO that causes the breakage.  Building with "-O2 -fno-FOO"
should give you all the /other/ benefits as well as working code.

This goes much faster, of course, if you can identify (1), then you only
have to recompile that file.
Comment 9 Andrew Pinski 2003-07-17 03:08:56 UTC
I would also use -W -Wall and see if there is any warnings that could be causing this.
Comment 10 Mark Mitchell 2003-07-20 00:46:35 UTC
Until we have a test case for this PR, there is nothing that we can do.

Therefore, I've postponed it until GCC 3.3.2.
Comment 11 branden@debian.org 2003-07-24 18:49:14 UTC
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

On Tue, Jul 15, 2003 at 07:45:02AM +0200, Matthias Klose wrote:
> sirl at gcc dot gnu dot org writes:
> > I have no other idea then, 4.2 already had -fno-merge-constants and
> > I can't think of another promising option to try. Maybe
> > -fno-strength-reduce? Otherwise you have to narrow down the
> > bugreport to a single miscompiled file, we won't be able to help you
> > otherwise.
> 
> Daniel, Branden,
> 
> I see, that on Debian only -O2 is used, not any of the mentioned
> flags.

Can someone give me an idea of what the proper followup to this message
might be?  Is more information required?

Comment 12 Phil Edwards 2003-07-24 19:01:05 UTC
> Can someone give me an idea of what the proper followup to this message
> might be?  Is more information required?

Yes, more information is required.  See comment #8 and comment #9.

The "other mentioned flags" are the flags mentioned in comments 3, 4, 5, and 7.
I believe those are a subset of the possible ones mentioned in comment 8.
Comment 13 Debian GCC Maintainers 2003-08-13 07:45:01 UTC
It looks like this is fixed with Jim Wilson's alias.c patch(?), at least I am
unable to reproduce the failure on the 20030812 3.3 branch.

Checked on i386-linux, need to recheck on powerpc.
Comment 14 Dara Hazeghi 2003-08-25 00:40:27 UTC
Any news about this on powerpc? Thanks.
Comment 15 branden@debian.org 2003-09-02 09:11:19 UTC
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

This problem appears to have been fixed by:

gcc-3.3 (1:3.3.2ds0-0pre0) unstable; urgency=medium

  * Update to gcc-3.3.2 CVS 20030812.
    - Fixes generation of wrong code for XDM-AUTHORIZATION-1 key generation
      and/or validation. Closes: #196090.
  * Update NEWS files.
  * Change ix86 default CPU type for code generation:
    - i386-linux       -> i486-linux
    - i386-gnu         -> i586-gnu
    - i386-freebsd-gnu -> i486-freebsd-gnu
    Use -march=i386 to target i386 CPUs.

 -- Matthias Klose <doko@debian.org>  Tue, 12 Aug 2003 10:31:28 +0200

Thanks for fixing it.

Comment 16 Andrew Pinski 2003-09-02 22:46:35 UTC
Thanks for filing this bug, I am going to mark this as a dup of bug 11319 which is meta bug for 
the aliasing bugs.

*** This bug has been marked as a duplicate of 11319 ***