Fails on 64-bit Solaris 10, 11/SPARC, too.

I get the following stacktrace:

#0  0xfee7248c in abort () from /lib/libc.so.1
#1  0x08859957 in _cpp_process_line_notes (pfile=0x8b5c468, in_comment=0) at /vol/gcc/src/hg/trunk/solaris/libcpp/lex.c:318
#2  0x0886153a in _cpp_scan_out_logical_line (pfile=0x8b5c468, macro=0x0) at /vol/gcc/src/hg/trunk/solaris/libcpp/traditional.c:384
#3  0x08861d53 in _cpp_read_logical_line_trad (pfile=0x8b5c468) at /vol/gcc/src/hg/trunk/solaris/libcpp/traditional.c:306
#4  0x0815e515 in scan_translation_unit_trad (pfile=0x8b5c468) at /vol/gcc/src/hg/trunk/solaris/gcc/c-ppoutput.c:289
#5  preprocess_file (pfile=0x8b5c468) at /vol/gcc/src/hg/trunk/solaris/gcc/c-ppoutput.c:95
#6  0x0815935b in c_common_init () at /vol/gcc/src/hg/trunk/solaris/gcc/c-opts.c:1235
#7  0x08161ee8 in c_objc_common_init () at /vol/gcc/src/hg/trunk/solaris/gcc/c-objc-common.c:74
#8  0x083f1a00 in lang_dependent_init (argc=18, argv=0x8047160) at /vol/gcc/src/hg/trunk/solaris/gcc/toplev.c:2279
#9  do_compile (argc=18, argv=0x8047160) at /vol/gcc/src/hg/trunk/solaris/gcc/toplev.c:2404
#10 toplev_main (argc=18, argv=0x8047160) at /vol/gcc/src/hg/trunk/solaris/gcc/toplev.c:2447
#11 0x0817d4ef in main (argc=18, argv=0x8047160) at /vol/gcc/src/hg/trunk/solaris/gcc/main.c:35

gdb claims note->type = 10

(In reply to comment #1)
> Fails on 64-bit Solaris 10, 11/SPARC, too.
Tossing "Regression" onto the "Summary", thanks for confirming,
Rob

Updated summary (you list the regressed branches to fix, not the working one).

note->type == 10 is:
  *d = '\n';
  /* A sentinel note that should never be processed.  */
  add_line_note (buffer, d + 1, '\n');
  buffer->next_line = s + 1;
created.  But, as this PR lacks a testcase (well, the testcase depends on a proprietary header file not included here), it is hard to guess what's going on.
Can you see if doing just
cp /usr/include/stdlib.h /tmp/stdlib.h
gcc -E -traditional-cpp /tmp/stdlib.h
If yes, can you distill a smaller testcase from it (ideally at most a few lines)?

Subject: Re:  [4.4/4.5/4.6 regression] Preprocessor ICE with -m64 and --traditional-cpp

> ------- Comment #4 from jakub at gcc dot gnu dot org  2010-06-24 11:51 -------

> created.  But, as this PR lacks a testcase (well, the testcase depends on a
> proprietary header file not included here), it is hard to guess what's going
> on.

I could include the OpenSolaris header, which is licensed under CDDL,
but I've just been able to construct a minimal testcase:

$ cat include.c
#include <stdlib.h>
$ cat stdlib.h
#pragma redefine_extname        mkstemp64       mkstemp
$ ./cc1 -E -traditional-cpp ./include.c -o include.i -I . -m64
In file included from ./include.c:2:0:
./stdlib.h:1:0: internal compiler error: Abort

This should make it easier to find what's going on.  Unlike the original
test case from the GCC testsuite, this one fails even without -m64.

	Rainer

Can't reproduce on x86_64-linux (and, #pragma redefine_extname seems to be handled on all targets, not just Solaris).

Subject: Re:  [4.4/4.5/4.6 regression] Preprocessor ICE with -m64 and --traditional-cpp

> ------- Comment #6 from jakub at gcc dot gnu dot org  2010-06-24 12:56 -------
> Can't reproduce on x86_64-linux (and, #pragma redefine_extname seems to be
> handled on all targets, not just Solaris).

This seems to be extremely sensitive to command line parameters.  E.g. on
mips-sgi-irix6.5 it works (no ICE) with

$ cc1 -E -tditional-cpp include.c -o include.i -I .

When I add either -mabi=n32 or -mabi=64, cc1 aborts.

Similarly, on sparc-sun-solaris2.11:

$ cc1 -E -traditional-cpp include.c -o include.i -I .

ICEs

$ cc1 -E -traditional-cpp `pwd`/include.c -o include.i -I .

works.

	Rainer

Subject: Re:  [4.4/4.5/4.6 regression] Preprocessor ICE with -m64 and --traditional-cpp

It occured to me that this is only failing for me with 32-bit cc1,
i.e. i386-pc-solaris2.11, sparc-sun-solaris2.11, mips-sgi-irix6.5, but
not alpha-dec-osf5.1 (which is 64-bit).

	Rainer

Thanks for working on this guys, 
Rob

It also fails with the 64-bit compiler on SPARC/Solaris, not just -m64.

Backtrace with Rainer's testcase on SPARC/Solaris 32-bit:

Program received signal SIGABRT, Aborted.
0xff3201cc in _libc_kill () from /usr/lib/libc.so.1
(gdb) bt
#0  0xff3201cc in _libc_kill () from /usr/lib/libc.so.1
#1  0xff2b594c in abort () from /usr/lib/libc.so.1
#2  0x008607d8 in _cpp_process_line_notes (pfile=0xaa3568, in_comment=0)
    at /nile.build/botcazou/gcc-head/src/libcpp/lex.c:871
#3  0x00869370 in _cpp_scan_out_logical_line (pfile=0xaa3568, macro=0x0)
    at /nile.build/botcazou/gcc-head/src/libcpp/traditional.c:384
#4  0x00869fb4 in _cpp_read_logical_line_trad (pfile=0xaa3568)
    at /nile.build/botcazou/gcc-head/src/libcpp/traditional.c:306
#5  0x0010aef0 in scan_translation_unit_trad (pfile=0xaa3568)
    at /nile.build/botcazou/gcc-head/src/gcc/c-family/c-ppoutput.c:288
#6  preprocess_file (pfile=0xaa3568)
    at /nile.build/botcazou/gcc-head/src/gcc/c-family/c-ppoutput.c:94
#7  0x00107618 in c_common_init ()
    at /nile.build/botcazou/gcc-head/src/gcc/c-family/c-opts.c:1051
#8  0x000ae7c4 in c_objc_common_init ()
    at /nile.build/botcazou/gcc-head/src/gcc/c-objc-common.c:64
#9  0x00481bb0 in lang_dependent_init (argc=5, argv=0xffbefbcc)
    at /nile.build/botcazou/gcc-head/src/gcc/toplev.c:2184
#10 do_compile (argc=5, argv=0xffbefbcc)
    at /nile.build/botcazou/gcc-head/src/gcc/toplev.c:2315

add_line_note adds a sentinel note

 done:
  *d = '\n';
  /* A sentinel note that should never be processed.  */
  add_line_note (buffer, d + 1, '\n');
  buffer->next_line = s + 1;

then _cpp_overlay_buffer changes buffer->cur and the test at the beginning of _cpp_process_line_notes

      if (note->pos > buffer->cur)
	break;

then becomes totally bogus.

I guess _cpp_overlay_buffer/_cpp_remove_overlay must temporarily invalidate the notes somehow.

Tom, what's the best approach to fixing this?

Created attachment 22795 [details]
gcc46-pr39213.patch

I wonder if this wouldn't fix it (at least, that's similar to how lex.c guards
calling of _cpp_process_line_notes).  I can't reproduce it myself, so have to guess...

> I wonder if this wouldn't fix it (at least, that's similar to how lex.c guards
> calling of _cpp_process_line_notes).  I can't reproduce it myself, so have to
> guess...

Apparently not.  I rebuilt libcpp, deleted cc1 and cpp and relinked them:

                === gcc tests ===

Schedule of variations:
    unix

Running target unix
Using /homes/brobecke/arch/congo/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /homes/brobecke/arch/congo/share/dejagnu/config/unix.exp as generic interface file for target.
Using /nile.build/botcazou/gcc-head/src/gcc/testsuite/config/default.exp as tool-and-target-specific interface file.
Running /nile.build/botcazou/gcc-head/src/gcc/testsuite/gcc.dg/cpp/trad/trad.exp ...
FAIL: gcc.dg/cpp/trad/include.c (internal compiler error)
FAIL: gcc.dg/cpp/trad/include.c (test for excess errors)

                === gcc Summary ===

# of expected passes            181
# of unexpected failures        2
# of unsupported tests          1


Same result on Rainer's testcase.

> I wonder if this wouldn't fix it (at least, that's similar to how lex.c guards
> calling of _cpp_process_line_notes).  I can't reproduce it myself, so have to
> guess...

The problem is that #pragma redefine_extname breaks the balance of calls to _cpp_overlay_buffer and _cpp_remove_overlay:

Breakpoint 1, _cpp_overlay_buffer (pfile=0x104705160,
    start=0x104714050 " redefine_extname\tmkstemp64\tmkstemp\nE64_SOURCE)\nLONG)\n& (3 - 0 >= 4))\nNned(_XOPEN_SOURCE)) || \t\tdefined(_KERNEL) || defined(_KMEMUSER) || \t\tdefined(__EXTENSIONS__)\n", len=35)
    at /nile.build/botcazou/gcc-head/src/libcpp/traditional.c:269
269       cpp_buffer *buffer = pfile->buffer;
(gdb) p pfile->overlaid_buffer
$3 = (cpp_buffer *) 0x0
(gdb) continue
Continuing.

Breakpoint 1, _cpp_overlay_buffer (pfile=0x104705160,
    start=0x104714050 "\t \nseemp64\tmkstemp\nkstemp64\tmkstemp\nE64_SOURCE)\nLONG)\n& (3 - 0 >= 4))\nNned(_XOPEN_SOURCE)) || \t\tdefined(_KERNEL) || defined(_KMEMUSER) || \t\tdefined(__EXTENSIONS__)\n", len=2)
    at /nile.build/botcazou/gcc-head/src/libcpp/traditional.c:269
269       cpp_buffer *buffer = pfile->buffer;
(gdb) p pfile->overlaid_buffer
$4 = (cpp_buffer *) 0x10470ad98

because in_deferred_pragma is true in:

/* Called when leaving a directive, _Pragma or command-line directive.  */
static void
end_directive (cpp_reader *pfile, int skip_line)
{
  if (pfile->state.in_deferred_pragma)
    ;
  else if (CPP_OPTION (pfile, traditional))
    {
      /* Revert change of prepare_directive_trad.  */
      pfile->state.prevent_expansion--;

      if (pfile->directive != &dtable[T_DEFINE])
	_cpp_remove_overlay (pfile);
    }

It is only reset to false in _cpp_lex_direct but this is apparently too late to call _cpp_remove_overlay from there:

Index: lex.c
===================================================================
--- lex.c       (revision 167901)
+++ lex.c       (working copy)
@@ -1944,6 +1944,8 @@ _cpp_lex_direct (cpp_reader *pfile)
          pfile->state.in_deferred_pragma = false;
          if (!pfile->state.pragma_allow_expansion)
            pfile->state.prevent_expansion--;
+         if (CPP_OPTION (pfile, traditional))
+           _cpp_remove_overlay (pfile);
          return result;
        }
       if (!_cpp_get_fresh_line (pfile))

yields a SIGSEGV in _cpp_remove_overlay.  The obvious change to end_directive seems to work though, revised patch to be attached.

Created attachment 22812 [details]
Revised patch

It passes the cpp.exp and trad.exp testsuites on 2 platforms.

Author: ebotcazou
Date: Tue Jan  4 23:18:12 2011
New Revision: 168490

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=168490
Log:
	PR preprocessor/39213
	* directives.c (end_directive): Call _cpp_remove_overlay for deferred
	pragmas as well in traditional mode.

Modified:
    trunk/libcpp/ChangeLog
    trunk/libcpp/directives.c

Author: ebotcazou
Date: Tue Jan  4 23:18:29 2011
New Revision: 168491

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=168491
Log:
	PR preprocessor/39213
	* directives.c (end_directive): Call _cpp_remove_overlay for deferred
	pragmas as well in traditional mode.

Modified:
    branches/gcc-4_5-branch/libcpp/ChangeLog
    branches/gcc-4_5-branch/libcpp/directives.c

Author: ebotcazou
Date: Tue Jan  4 23:18:52 2011
New Revision: 168492

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=168492
Log:
	PR preprocessor/39213
	* directives.c (end_directive): Call _cpp_remove_overlay for deferred
	pragmas as well in traditional mode.

Modified:
    branches/gcc-4_4-branch/libcpp/ChangeLog
    branches/gcc-4_4-branch/libcpp/directives.c

At long last.

(In reply to comment #21)
> At long last.
It was only 2 years... I have some older than that.

Thank you for your work on my Bug Report,
Rob