[Bug preprocessor/58887] Allow recursion in variadic macros?

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58887

--- Comment #11 from Max TenEyck Woodbury <mtewoodbury at gmail dot com> ---
(In reply to joseph@codesourcery.com from comment #10)
> On Mon, 28 Oct 2013, mtewoodbury at gmail dot com wrote:
> 
>> (Stop the 'we'!  Name or enumerate the group involved please.)
> 
> Well-established consensus among the GCC maintainers about what sorts of 
> features are appropriate to add and what sorts of features cause problems.  
> It's not as if the preprocessor has lots of active development with 
> disagreement among its developers about what should go in; it's rightly 
> pretty stable in terms of features with only occasional bug fixes or new 
> features (mainly coming from WG21) needed.

I think I understand consensus, but I only hear your voice here, not the voice
of a multitude.  You may be part of the consensus you speak of, but you are not
its entirety.  In that, you overstate your case.  If you had said 'As an old
hand at this' instead of 'We', I would not have objected.

>> But that bug was filed on the wrong component and has languished for YEARS
>> as a result of that miss-filing.  It looks like no one has looked at its
>> problem seriously...
> 
> That maintainers wouldn't necessarily object to the addition of a feature 
> doesn't mean any maintainer has any interest in implementing it.  There 
> are lots of "bugs" filed suggesting some vaguely reasonable new feature 
> that was of interest to the submitter but not of sufficient interest to 
> anyone wanting to implement it (but not rejected either, because the 
> feature might well be accepted if implemented).

I have worked for companies where there was a formal definition of roles, and
what you describe is something from that world.  On the other hand, I believe
open-source projects like GCC have less authoritarian structures and that such
decisions are much less formal.  In other words, membership in 'maintainers' is
consensus based on merit, not the open and shut rule of corporate structure.

On the other hand, 'unconfirmed' is really not a status that applies to
requests for enhancements.

>> (In reply to joseph@codesourcery.com from comment #8)
>>> (And for recursion, even specification at the level of standard text might 
>>> leave something to be desired; I'd think specification at the level of 
>>> X3J11/86-196, the algorithm GCC tries to follow regarding when a macro 
>>> name generated in macro expansion can itself be expanded, would be desired 
>>> as well.  Not that I think recursion is appropriate to include in GCC's 
>>> preprocessor unless it's standardized.)
>> 
>> Hmm.  Is X3J11/86-196 the pdf that shows up at the top of a Google search?
> 
> Yes.
> 
>> If so, I'll need to go over it fairly carefully.  A quick review left me with
>> the impression that determining when to allow additional expansion involved a
>> bit of hand-waving.
> 
> The point is it defines, through the pseudocode functions, exactly how 
> hide sets (the sets of macros for which expansion is currently suppressed 
> because it would be recursive) are determined - it's the pseudocode that 
> you need to study more than the surrounding text.  And it's this algorithm 
> that GCC is intended to follow.  So anything allowing new forms of 
> recursion needs to explain how this algorithm is affected.

As I said, I need to go over it carefully.  It is obviously pseudo-code and
both the code and the explanation deserve careful study.  Where it has built in
the no-recursion rule was not quite obvious at first glance, thus the remark
about hand waving.  Again, I need to study it carefully, and I have not done
that yet...

On the other hand, your summary adds vary little to what the pdf contains...

> 
>> possible.  With the proper hedges in place it would have the same kind of
>> power that variadic functions posses.  As things currently stand, variadic
>> macros have apparently arbitrary limitations that reduces their usefulness.
>> With an intelligent design, this would be where the language aught to be
>> going.
> 
> I suggest that the language ought not to be going in the direction of 
> adding much power to the preprocessor at all - that expressive power 
> belongs in the language, not the preprocessor (and that it's fine to use 
> programs to generate C program text if neither is convenient for what you 
> want to do).

That is a very old argument and fallacious on its face.  The preprocessor is
part of the language.

External code generators create new languages.  Those new languages will never
get the level of acceptance that 'C' has.  Each generator represents an
additional layer in a project's development and requires an investment that has
to be justified.  Further, that kind of development calls for skills that many
projects simply can not justify.  Adding an external code generator to an
implementation can also preclude that implementation from consideration for
legal reasons while extending 'C' by using macros would not run into the same
bar.  Po-poing the preprocessor this way just isn't on. 

> Obviously you can experiment with adding a feature to GCC's preprocessor 
> in preparation for submitting it to WG14 - and detailed definitions in 
> terms of pseudocode algorithms and proposed standard text will be helpful 
> for that as well.  But for actual inclusion in mainline GCC, you need to 
> convince the maintainers that it's desirable for such features to be 
> present in the preprocessor at all - that they are worth the maintenance 
> burden that any feature imposes (which includes being well-enough defined 
> that reimplementing a bit of the compiler won't change them incompatibly, 
> rather than doing things in ways that are accidents of the implementation 
> and so hard to specify and keep compatible over time).

So let the maintainers think about it and do not throw false arguments into the
discussion.  The process should proceed more or less along the following lines:

1)  Define how the 'no-recursion' rule can be changed so that the result is
    most useful and least problematic from the language user's point of view.

2)  Identify _what_ needs to be changed in GCC to effect that change.  To
    some extent this includes _where_ and _how_ GCC might implement the
    change, but this step really addresses high level considerations, not
    details.

3)  Asses the amount of effort that would be required to properly
    evaluate the change.

4)  If the resources identified in step 3 are available, evaluate and
    review the expected impact of the change.  This step will also
    answer questions about the expediency of making such a change.

5)  If resources identified in step 4 are available, identify a specific
    solution, evaluate its likely effectiveness and impacts.

6)  If the resources identified in step 5 are available, implement and test
    the proposed solution.  Ideally, the change should have an option that
    turns it on and off.

7)  Use the trial solution to update the specification from step 1 and turn
    it into a formal specification.

8)  Get some experience with the solution and repeat steps based on that
    experience.

9)  Formal standardization process...

We are not even into step 1 yet.

> Sometimes a development branch is a better place for gaining experience 
> with an experimental feature than mainline, until the standards committees 
> reach a conclusion on the feature and how to specify it (that was done 
> with C++ concepts, for example).

That is probably a step 5 action, so you are jumping the gun a bit here.