Bug 61977 - [4.8/4.9/5 Regression] powerpc preprocessor breaks on lines that end with "vector"
Summary: [4.8/4.9/5 Regression] powerpc preprocessor breaks on lines that end with "ve...
Alias: None
Product: gcc
Classification: Unclassified
Component: preprocessor (show other bugs)
Version: 4.9.1
: P3 normal
Target Milestone: 4.8.5
Assignee: Jakub Jelinek
: 65638 (view as bug list)
Depends on:
Reported: 2014-07-31 15:32 UTC by David Rivshin
Modified: 2015-04-06 17:04 UTC (History)
3 users (show)

See Also:
Target: powerpc*-*-*-*
Known to work: 4.9.0
Known to fail: 4.9.1, 4.9.2
Last reconfirmed: 2015-03-31 00:00:00

patch to revert 210055 and 211656 (3.98 KB, patch)
2014-07-31 15:32 UTC, David Rivshin
Details | Diff
gcc5-pr61977.patch (480 bytes, patch)
2015-03-31 17:54 UTC, Jakub Jelinek
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Rivshin 2014-07-31 15:32:14 UTC
Created attachment 33219 [details]
patch to revert 210055 and 211656

The powerpc-eabi (and powerpc-eabispe -mno-spe; I haven't tried other powerpc-*) preprocessor misbehaves when a line ends with "vector". I happened to hit this in assembler comments, but that does not appear to be a requirement.

Examples of lines which cause the failure:
# vector
# x vector
x # vector
; vector
# .vector
# +vector

Examples of lines which do not cause the failure:
# vectors
# vector x
# vector.
# _vector
// vector   (unless -C is given)

The symptom depends on whether that is the last non-whitespace line in the file. If it is, then the result is an ICE. If it is not, then just "vector" appears by itself on the next line.

This did not happen on GCC 4.9.0, but does happen in GCC 4.9.1 and the current trunk. Bisecting between 4.9.0 and 4.9.1 points to SVN revision 210055 as introducing this behavior. Reverting r210055 (and also r211656 which seemed to be dependant upon r210055) appears to fix the issue, and a patch (against 4.9.1) doing that is attached.


$ prefix/bin/powerpc-eabi-gcc -v
Using built-in specs.
Target: powerpc-eabi
Configured with: ../gcc-4.9.1/configure -v --prefix=/home/drivshin/gcc-powerpc-eabi/build-gcc/../prefix --target=powerpc-eabi --enable-languages=c
Thread model: single
gcc version 4.9.1 (GCC) 


$ echo -e "# comment ending in vector\n# another comment" | prefix/bin/powerpc-eabi-cpp -x assembler-with-cpp
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
# comment ending in
# 2 "<stdin>"
# another comment


$ echo -e "# comment ending in vector" | prefix/bin/powerpc-eabi-cpp -x assembler-with-cpp
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
<stdin>:1:0: internal compiler error: Segmentation fault
0x82f7bf crash_signal
0xc37b75 _cpp_lex_direct
0xc389fb _cpp_lex_token
0xc3d117 cpp_get_token_1
0x559877 scan_translation_unit
0x559877 preprocess_file(cpp_reader*)
0x558360 c_common_init()
0x4ff08d c_objc_common_init()
0x8312c6 lang_dependent_init
0x8312c6 do_compile
Comment 1 David Rivshin 2015-03-20 18:19:48 UTC
This is still happening in the latest trunk and latest 4.9 branch code.

Simplified steps to reproduce:
../gcc.svn/configure --prefix=${PWD}/../local --enable-languages=c --with-gnu-as --with-gnu-ld --disable-libstdcxx-pch --target=powerpc-eabi --disable-shared --with-newlib
make all-gcc
make install-gcc
echo -e "# comment ending in vector" | ../local/bin/powerpc-eabi-cpp -x assembler-with-cpp

I'm fairly certain this is the same root cause as bug 51654, and changeset r210055 just exposed some non-altivec powerpc targets to it. In addition to the workarounds mentioned there (bug 51654, comment 3), removing the call to init_vector_keywords() in rs6000_cpu_cpp_builtins() also works. 

Since those vector keywords only have effect if TARGET_ALTIVEC (see rs6000_macro_to_expand()), making their definition conditional upon TARGET_ALTIVEC resolves the 4.9.1 regression (as best I can tell). Although that obviously does not resolve the underlying issue, which has existed since at least 4.6 (according to bug 51654).
Comment 2 Martin Sebor 2015-03-25 02:18:35 UTC
All powerpc64*-*-*-* targets appear to be affected.
Comment 3 Martin Sebor 2015-03-31 16:34:57 UTC
See also pr65638 for a similar problem.
Comment 4 Markus Trippelsdorf 2015-03-31 16:37:27 UTC
*** Bug 65638 has been marked as a duplicate of this bug. ***
Comment 5 Jakub Jelinek 2015-03-31 17:29:14 UTC
The problem is that cpp_peek_token, if it returns CPP_EOF, is fatal in the preprocessing:
      peektok = _cpp_lex_token (pfile);
      if (peektok->type == CPP_EOF)
        return peektok;
  while (index--);
but the macro_to_expand stuff (for which cpp_peek_token has been written, BTW) really assumes that it can non-destructively peek tokens when needed.
Comment 6 Jakub Jelinek 2015-03-31 17:53:28 UTC
Obviously regression from the times when vector wasn't a conditional macro.
Comment 7 Jakub Jelinek 2015-03-31 17:54:07 UTC
Created attachment 35194 [details]

Untested fix.
Comment 8 David Rivshin 2015-03-31 19:52:01 UTC
I briefly tested the patch, and it does fix the ICE in the case where the conditional macro is the last token.

However it does not fix the situation where there are more (non-blank) lines after the conditional macro:

$ /bin/echo -e "; comment ending in vector\n; another comment" | ../local/bin/powerpc-eabi-cpp -x assembler-with-cpp
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
; comment ending in
# 2 "<stdin>"
; another comment

There is a newline which gets inserted in the stream before the conditional macro token. In many cases that is harmless, but in some (like the above single-line assembler comment) the result is to turn valid code into invalid code.
Comment 9 David Rivshin 2015-04-01 14:21:18 UTC
I think the extra newline is the result of maybe_print_line() being invoked when trying to peek past a newline in the input.

#0  maybe_print_line_1 (src_loc=134, stream=0x361e3b8800 <_IO_2_1_stdout_>) at c-ppoutput.c:352
#1  maybe_print_line (src_loc=134) at c-ppoutput.c:385
#2  do_line_change (pfile=0x1ef2910, token=0x1f1dd68, src_loc=134, parsing_args=0) at c-ppoutput.c:463
#3  cb_line_change (pfile=0x1ef2910, token=0x1f1dd68, parsing_args=0) at c-ppoutput.c:490
#4  _cpp_lex_token (pfile=0x1ef2910) at lex.c:2192
#5  cpp_peek_token (pfile=0x1ef2910, index=0) at lex.c:2085
#6  cpp_get_token_1 (pfile=0x1ef2910, location=0x7fffffffd8fc) at macro.c:2501

If I'm understanding the logic correctly, when _cpp_lex_direct() sees the newline, the processing of the line is considered complete, and therefore that line of output complete. But because of the conditional macro that's not entirely true, and the output only has the line up to (but not including) the conditional macro token itself at that point.
Comment 11 Jakub Jelinek 2015-04-02 11:55:30 UTC
Author: jakub
Date: Thu Apr  2 11:54:58 2015
New Revision: 221838

URL: https://gcc.gnu.org/viewcvs?rev=221838&root=gcc&view=rev
	PR preprocessor/61977
	* config/rs6000/rs6000-c.c (rs6000_cpu_cpp_builtins): Don't
	predefine __vector/__bool/__pixel macros nor context sensitive
	macros for CLK_ASM.
	* config/spu/spu-c.c (spu_cpu_cpp_builtins): Similarly.

Comment 12 Jakub Jelinek 2015-04-02 11:57:33 UTC
Author: jakub
Date: Thu Apr  2 11:57:02 2015
New Revision: 221839

URL: https://gcc.gnu.org/viewcvs?rev=221839&root=gcc&view=rev
	PR preprocessor/61977
	* lex.c (cpp_peek_token): Temporarily clear pfile->cb.line_change.

	* gcc.target/powerpc/pr61977-1.c: New test.
	* gcc.target/powerpc/pr61977-2.c: New test.

Comment 13 Jakub Jelinek 2015-04-02 12:08:25 UTC
Two out of the 3 patches applied to trunk, still waiting for review of the first patch.
Comment 14 Jakub Jelinek 2015-04-06 17:02:21 UTC
Author: jakub
Date: Mon Apr  6 17:01:50 2015
New Revision: 221882

URL: https://gcc.gnu.org/viewcvs?rev=221882&root=gcc&view=rev
	PR preprocessor/61977
	* lex.c (cpp_peek_token): If peektok is CPP_EOF, back it up
	with all tokens peeked by the current function.

	* gcc.dg/cpp/pr61977.c: New test.

Comment 15 Jakub Jelinek 2015-04-06 17:04:36 UTC