How do you end up registering so many pragmas?  I can't see anything in darwin specific code.

> How do you end up registering so many pragmas?  I can't see anything in
> darwin specific code.

No idea! Last successful bootstrap was r230059.

DARWIN_REGISTER_TARGET_PRAGMAS registers 5.
If I count well on Linux and yesterday's trunk, for -fopenmp -fopenacc -fcilkplus I see 10 OpenACC, 25 OpenMP, 2 Cilk+ and 22 other deferred pragmas being registered, if you add 5 to that it is 64.
The comment in c-family is clearly outdated, the C parser uses 8 bits for pragma_kind.
But, looking at parser.h, I see
  /* The kind of token.  */
  ENUM_BITFIELD (cpp_ttype) type : 8;
  /* If this token is a keyword, this value indicates which keyword.
     Otherwise, this value is RID_MAX.  */
  ENUM_BITFIELD (rid) keyword : 8;
  /* Token flags.  */
  unsigned char flags;
  /* Identifier for the pragma.  */
  ENUM_BITFIELD (pragma_kind) pragma_kind : 6;
  /* True if this token is from a context where it is implicitly extern "C" */
  BOOL_BITFIELD implicit_extern_c : 1;
  /* True if an error has already been reported for this token, such as a
     CPP_NAME token that is not a keyword (i.e., for which KEYWORD is
     RID_MAX) iff this name was looked up and found to be ambiguous.  */
  BOOL_BITFIELD error_reported : 1;
  /* True for a token that has been purged.  If a token is purged,
     it is no longer a valid token and it should be considered
     deleted.  */
  BOOL_BITFIELD purged_p : 1;
which if I count well is already 33 bits anyway, followed by 32-bit integer and pointer, therefore on 64-bit hosts there are 63 bits of padding and on 32-bit hosts 31 bits of padding.

CCing Jason on the C++ token structure layout.

Actually, the OpenMP/OpenACC/Cilk+ pragmas and 2 others are always assigned fixed numbers, and very recently one OpenACC pragma has been added, so we now have PRAGMA_FIRST_EXTERNAL 41, plus 20 generic externals, plus the 5 Darwin ones on Darwin.

Also look at vms-c.c which registers 14.

(In reply to Jakub Jelinek from comment #3)
> DARWIN_REGISTER_TARGET_PRAGMAS registers 5.
> If I count well on Linux and yesterday's trunk, for -fopenmp -fopenacc
> -fcilkplus I see 10 OpenACC, 25 OpenMP, 2 Cilk+ and 22 other deferred
> pragmas being registered, if you add 5 to that it is 64.
> The comment in c-family is clearly outdated, the C parser uses 8 bits for
> pragma_kind.
> But, looking at parser.h, I see
>   /* The kind of token.  */
>   ENUM_BITFIELD (cpp_ttype) type : 8;
>   /* If this token is a keyword, this value indicates which keyword.
>      Otherwise, this value is RID_MAX.  */
>   ENUM_BITFIELD (rid) keyword : 8;
>   /* Token flags.  */
>   unsigned char flags;
>   /* Identifier for the pragma.  */
>   ENUM_BITFIELD (pragma_kind) pragma_kind : 6;
>   /* True if this token is from a context where it is implicitly extern "C"
> */
>   BOOL_BITFIELD implicit_extern_c : 1;
>   /* True if an error has already been reported for this token, such as a
>      CPP_NAME token that is not a keyword (i.e., for which KEYWORD is
>      RID_MAX) iff this name was looked up and found to be ambiguous.  */
>   BOOL_BITFIELD error_reported : 1;
>   /* True for a token that has been purged.  If a token is purged,
>      it is no longer a valid token and it should be considered
>      deleted.  */
>   BOOL_BITFIELD purged_p : 1;
> which if I count well is already 33 bits anyway, followed by 32-bit integer
> and pointer, therefore on 64-bit hosts there are 63 bits of padding and on
> 32-bit hosts 31 bits of padding.

That should be fixed of course.  Maybe some unioning can be done as well
based on 'type' (keyword?)

This is indeed caused by revision r230072.

I'm not sure it will make much of a difference, but Thomas is planning on adding two openacc clauses bind and nohost. Is there anything I can do to help here, or is this already being taken care of?

The cp_token::pragma_kind field seems like a waste of bits; why can't we leave the pragma kind in token->u.value?  RTH, do you remember why you added this field?

I believe the tokens didn't stay around in C at the time.
But I might be wrong... it was 9 years ago...

If we can remove it, it does seem like a good idea.

(In reply to Richard Henderson from comment #10)
> I believe the tokens didn't stay around in C at the time.
> But I might be wrong... it was 9 years ago...
> 
> If we can remove it, it does seem like a good idea.

I believe they still don't stay around in C, but they do stay around in C++.
So perhaps we could just add cp_parser_get_pragma_kind routine or similar
that would for a token return us a pragma_kind and use it in the 5 or how many spots, plus adjust the c-family assert to be id < 256 and state that C FE reserves 8 bits for pragma_kind and C++ FE doesn't have an upper bound.

Is the problem that difficult to fix?

Quick temporary fix is easy, just make pragma_kind in cp/parser.h 8 bit, and
change id < 64 to id < 256 in c-family and update the comment.  This I believe shouldn't make the C++ token any larger.
And then incrementally we can improve this by dropping pragma_kind.

> Quick temporary fix is easy, just make pragma_kind in cp/parser.h 8 bit,
> and change id < 64 to id < 256 in c-family and update the comment.
> This I believe shouldn't make the C++ token any larger.
> And then incrementally we can improve this by dropping pragma_kind.

I have successfully bootstrapped r230151 with the following patch

--- ../_clean/gcc/cp/parser.h	2015-11-10 01:54:44.000000000 +0100
+++ gcc/cp/parser.h	2015-11-11 12:10:28.000000000 +0100
@@ -48,7 +48,7 @@ struct GTY (()) cp_token {
   /* Token flags.  */
   unsigned char flags;
   /* Identifier for the pragma.  */
-  ENUM_BITFIELD (pragma_kind) pragma_kind : 6;
+  ENUM_BITFIELD (pragma_kind) pragma_kind : 8;
   /* True if this token is from a context where it is implicitly extern "C" */
   BOOL_BITFIELD implicit_extern_c : 1;
   /* True if an error has already been reported for this token, such as a
--- ../_clean/gcc/c-family/c-pragma.c	2015-11-10 01:54:43.000000000 +0100
+++ gcc/c-family/c-pragma.c	2015-11-11 12:10:25.000000000 +0100
@@ -1372,7 +1372,7 @@ c_register_pragma_1 (const char *space, 
 
       /* The C++ front end allocates 6 bits in cp_token; the C front end
 	 allocates 7 bits in c_token.  At present this is sufficient.  */
-      gcc_assert (id < 64);
+      gcc_assert (id < 256);
     }
 
   cpp_register_deferred_pragma (parse_in, space, name, id,

I let people understanding the problem update the comment. IMO the comment should include a pointer to "ENUM_BITFIELD (pragma_kind) pragma_kind : n;" when updating the assert to 2**n. It would also be interesting to know how many pragmas can be added before reaching the limit.

(In reply to Dominique d'Humieres from comment #14)
> > Quick temporary fix is easy, just make pragma_kind in cp/parser.h 8 bit,
> > and change id < 64 to id < 256 in c-family and update the comment.
> > This I believe shouldn't make the C++ token any larger.
> > And then incrementally we can improve this by dropping pragma_kind.
> 
> I have successfully bootstrapped r230151 with the following patch
> 
> --- ../_clean/gcc/cp/parser.h	2015-11-10 01:54:44.000000000 +0100
> +++ gcc/cp/parser.h	2015-11-11 12:10:28.000000000 +0100
> @@ -48,7 +48,7 @@ struct GTY (()) cp_token {
>    /* Token flags.  */
>    unsigned char flags;
>    /* Identifier for the pragma.  */
> -  ENUM_BITFIELD (pragma_kind) pragma_kind : 6;
> +  ENUM_BITFIELD (pragma_kind) pragma_kind : 8;
>    /* True if this token is from a context where it is implicitly extern "C"
> */
>    BOOL_BITFIELD implicit_extern_c : 1;
>    /* True if an error has already been reported for this token, such as a
> --- ../_clean/gcc/c-family/c-pragma.c	2015-11-10 01:54:43.000000000 +0100
> +++ gcc/c-family/c-pragma.c	2015-11-11 12:10:25.000000000 +0100
> @@ -1372,7 +1372,7 @@ c_register_pragma_1 (const char *space, 
>  
>        /* The C++ front end allocates 6 bits in cp_token; the C front end
>  	 allocates 7 bits in c_token.  At present this is sufficient.  */
> -      gcc_assert (id < 64);
> +      gcc_assert (id < 256);
>      }
>  
>    cpp_register_deferred_pragma (parse_in, space, name, id,
> 
> I let people understanding the problem update the comment. IMO the comment
> should include a pointer to "ENUM_BITFIELD (pragma_kind) pragma_kind : n;"
> when updating the assert to 2**n. It would also be interesting to know how
> many pragmas can be added before reaching the limit.

With proper ChangeLog entry and changing the 6 bits to 8 bits and 7 bits also to 8 bits in the comment about the assert this is ok for trunk, but please post the patch to gcc-patches.

Author: dominiq
Date: Wed Nov 11 14:30:16 2015
New Revision: 230172

URL: https://gcc.gnu.org/viewcvs?rev=230172&root=gcc&view=rev
Log:
gcc/cp/ChangeLog
2015-11-11  Dominique d'Humieres <dominiq@lps.ens.fr>

	PR bootstrap/68271
	* parser.h (cp_token): Update pragma_kind to 8.

gcc/c-family/ChangeLog
2015-11-11  Dominique d'Humieres <dominiq@lps.ens.fr>

	PR bootstrap/68271
	* c-pragma.c (c_register_pragma_1): Update the gcc_assert to 256.


Modified:
    trunk/gcc/c-family/ChangeLog
    trunk/gcc/c-family/c-pragma.c
    trunk/gcc/cp/ChangeLog
    trunk/gcc/cp/parser.h

Ny need to keep this PR opened?

Yes, so that we don't forget to apply a real fix.

> Yes, so that we don't forget to apply a real fix.

OK, downgrading to normal.

Created attachment 37358 [details]
gcc6-pr68271.patch

Untested fix.

Author: jakub
Date: Fri Jan 15 20:57:54 2016
New Revision: 232451

URL: https://gcc.gnu.org/viewcvs?rev=232451&root=gcc&view=rev
Log:
	PR bootstrap/68271
	* parser.h (cp_token): Remove pragma_kind field.  Add comment
	with number of unused bits.
	* parser.c (eof_token): Remove pragma_kind field initializer.
	(cp_lexer_get_preprocessor_token): Don't set pragma_kind
	field, don't clear CPP_PRAGMA u.value.
	(cp_parser_pragma_kind): New function.
	(cp_parser_omp_sections_scope, cp_parser_oacc_kernels_parallel,
	cp_parser_omp_construct, cp_parser_initial_pragma,
	cp_parser_pragma): Use cp_parser_pragma_kind instead of accessing
	pragma_kind field.

	* c-pragma.c (c_register_pragma_1): Adjust comment to note that
	C++ FE no longer has limit on number of pragmas.

Modified:
    trunk/gcc/c-family/ChangeLog
    trunk/gcc/c-family/c-pragma.c
    trunk/gcc/cp/ChangeLog
    trunk/gcc/cp/parser.c
    trunk/gcc/cp/parser.h

Fixed.