Bug 56825 - Preprocessor does not expand macro correctly if it is an argument and argument of a macro contains ##
Summary: Preprocessor does not expand macro correctly if it is an argument and argumen...
Status: UNCONFIRMED
Alias: None
Product: gcc
Classification: Unclassified
Component: preprocessor (show other bugs)
Version: 4.7.2
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-04-03 09:05 UTC by Evgeny Televitckiy
Modified: 2013-04-04 06:41 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed:


Attachments
Test file (171 bytes, application/octet-stream)
2013-04-03 09:05 UTC, Evgeny Televitckiy
Details
Preprocessor result (128 bytes, application/octet-stream)
2013-04-03 09:05 UTC, Evgeny Televitckiy
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Evgeny Televitckiy 2013-04-03 09:05:13 UTC
Created attachment 29786 [details]
Test file

Ok. That is a little bit hard to explain. Consider the following macros:
#define E3N(n, ar...) E3NI(_, ##ar, n, n, n)
#define E3NI(a0, a1, a2, a3, ...) a1, a2, a3

#define USE_E3_ARG_R(n, ar...) (E3N(n, ar))
#define USE_E3_ARG_W(n, ar...) (E3N(n, ##ar))

If we write something like:
 USE_E3_ARG_W(3, 4);
It expands correctly to:
 (4, 3, 3);

But if we use more complicate:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
It expands to:
 (USE_E3_ARG_W(9, 0), 8, 8);

While I would expect:
 ((0, 9, 9), 8, 8);

Mind that the following expands correctly:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
To:
  ((7, 6, 6), 5, 5);

The problem is probably related to the following paragraph in standart:
"... After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter in the replacement list, unless preceded by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is replaced by the corresponding argument after all macros contained therein have been expanded..."

But IMHO this statement is relevant to the macro itself and should not apply to the argument of the macro containing ## to remove comma (##ar in our case).
 
I know that it will not make any point but Eclipse expands all these macros correctly with internal macro expansion.

Output of the test compilation:
gcc-mp-4.7 test2.c -v --save-temps
Using built-in specs.
COLLECT_GCC=gcc-mp-4.7
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/lto-wrapper
Target: x86_64-apple-darwin12
Configured with: ../gcc-4.7.2/configure --prefix=/opt/local --build=x86_64-apple-darwin12 --enable-languages=c,c++,objc,obj-c++,lto,fortran,java --libdir=/opt/local/lib/gcc47 --includedir=/opt/local/include/gcc47 --infodir=/opt/local/share/info --mandir=/opt/local/share/man --datarootdir=/opt/local/share/gcc-4.7 --with-libiconv-prefix=/opt/local --with-local-prefix=/opt/local --with-system-zlib --disable-nls --program-suffix=-mp-4.7 --with-gxx-include-dir=/opt/local/include/gcc47/c++/ --with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local --with-ppl=/opt/local --with-cloog=/opt/local --enable-cloog-backend=isl --disable-cloog-version-check --enable-stage1-checking --disable-multilib --enable-lto --enable-libstdcxx-time --with-as=/opt/local/bin/as --with-ld=/opt/local/bin/ld --with-ar=/opt/local/bin/ar --with-bugurl=https://trac.macports.org/newticket --disable-ppl-version-check --with-pkgversion='MacPorts gcc47 4.7.2_2'
Thread model: posix
gcc version 4.7.2 (MacPorts gcc47 4.7.2_2)
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps' '-mtune=core2'
 /opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/cc1 -E -quiet -v -D__DYNAMIC__ test2.c -fPIC -mmacosx-version-min=10.8.3 -mtune=core2 -fpch-preprocess -o test2.i
ignoring nonexistent directory "/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../../../../x86_64-apple-darwin12/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/include
 /opt/local/include
 /opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/include-fixed
 /usr/include
 /System/Library/Frameworks
 /Library/Frameworks
End of search list.
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps' '-mtune=core2'
 /opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/cc1 -fpreprocessed test2.i -fPIC -quiet -dumpbase test2.c -mmacosx-version-min=10.8.3 -mtune=core2 -auxbase test2 -version -o test2.s
GNU C (MacPorts gcc47 4.7.2_2) version 4.7.2 (x86_64-apple-darwin12)
	compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.1-p2, MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU C (MacPorts gcc47 4.7.2_2) version 4.7.2 (x86_64-apple-darwin12)
	compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.1-p2, MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 7de89c8f14784e88078e79248a999e9e
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps' '-mtune=core2'
 /opt/local/bin/as -v -arch x86_64 -force_cpusubtype_ALL -o test2.o test2.s
Apple Inc version cctools-836, GNU assembler version 1.38
COMPILER_PATH=/opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/libexec/gcc/x86_64-apple-darwin12/:/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/
LIBRARY_PATH=/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/:/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../../:/usr/lib/
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.8.3' '-v' '-save-temps' '-mtune=core2'
 /opt/local/libexec/gcc/x86_64-apple-darwin12/4.7.2/collect2 -dynamic -arch x86_64 -macosx_version_min 10.8.3 -weak_reference_mismatches non-weak -o a.out -L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2 -L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../.. test2.o -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
collect2 version 4.7.2
/opt/local/bin/ld -dynamic -arch x86_64 -macosx_version_min 10.8.3 -weak_reference_mismatches non-weak -o a.out -L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2 -L/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2/../../.. test2.o -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lSystem -v
@(#)PROGRAM:ld  PROJECT:ld64-134.9
configured to support archs: i386 x86_64
Library search paths:
	/opt/local/lib/gcc47/gcc/x86_64-apple-darwin12/4.7.2
	/opt/local/lib/gcc47
	/usr/lib
	/usr/local/lib
Framework search paths:
	/Library/Frameworks/
	/System/Library/Frameworks/
Undefined symbols for architecture x86_64:
  "_USE_E3_ARG_W", referenced from:
      _main in test2.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status

Thanks,
Evgeny
Comment 1 Evgeny Televitckiy 2013-04-03 09:05:47 UTC
Created attachment 29787 [details]
Preprocessor result
Comment 2 Evgeny Televitckiy 2013-04-03 09:09:22 UTC
A small correction.
Lines:
But if we use more complicate:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
It expands to:
 (USE_E3_ARG_W(9, 0), 8, 8);

Should be changed to:
But if we use more complicate:
 USE_E3_ARG_W(8, USE_E3_ARG_W(9, 0));
It expands to:
 (USE_E3_ARG_W(9, 0), 8, 8);
Comment 3 Harald van Dijk 2013-04-03 17:31:26 UTC
For the related
  #define foo(x, y) x ## y
both foo(f,oo(,)) and foo(,foo(,)) expand similarly to foo(,) and in that case, that is the only valid expansion in standard C (C99, anyway). The macro argument foo(,) is not expanded during argument substitution because it is an operand of a ## operator. The macro argument is not expanded after argument substitution because that happens in the context of another expansion of the same foo macro.

> But IMHO this statement is relevant to the macro itself and should not apply to the argument of the macro

That is exactly what it is meant to apply to.

#define A 1
#define B 2
#define AB 3
#define C(a, b) a ## b
C(A, B)

must expand to 3. Neither A nor B is allowed to be expanded here before concatenation takes place. AB must be expanded after that, as long as no other expansion of AB is already taking place.

That said,
  , ## x
is a GNU extension that never concatenates (except for the rare case where x is an empty macro argument), so the standard cannot and does not require it to behave exactly the same way as the concatenation operator.
Comment 4 Evgeny Televitckiy 2013-04-04 06:41:38 UTC
Ye, I thought it was something along those lines. Thanks for the clarification, now I understand the reason why this paragraph appear in standard.
Yet, appealing to the end of your comment. I would expect ,##x construct to be an exception, especially that it has a totally different meaning then concatenating # or stringifying ##.
Clearly one would expect:
#define E3N(n, ar...) E3NI(_, ##ar, n, n, n)
#define E3NI(a0, a1, a2, a3, ...) a1, a2, a3

#define USE_E3_ARG_R(n, ar...) (E3N(n, ar))
#define USE_E3_ARG_W(n, ar...) (E3N(n, ##ar))

and used:
 USE_E3_ARG_R(5, USE_E3_ARG_R(6, 7));
 USE_E3_ARG_W(5, USE_E3_ARG_W(6, 7));

To produce the same result...

In addition I wanted to add that it is not some virtual problem. I actually spent 3 hours of my life in that macro hell, trying to understand what went wrong with that macro,