This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: Special handling of "%H" (Re: Support for %d$c format specifierin diagnostics.c)
Ok, while I am waiting for the patch to support positional format
specifiers in diagnostic (and formated output) routines being merged
into mainline tree (hint, hint), I looked at
how we could enhance msgfmt in GNU gettext suite to check for GCC
diagnostic extended format character set.
It was relatively easy to fix the heart of checking routine.
format-awk.c needs to be modified as in the patch attached at the end.
msgfmt.c needs to understand addtional "--gcc" flag and sets
gcc_warning_extended_format_check to true. (I omitted msgfmt.c mods below.)
I report the rather shocking finding below.
I ran this modified msgfmt on the PO files under
gcc/gcc/po directory.
Funny, there was NO warning at all!
for f in *.po
> do
> echo $f
> done
be.po
da.po
de.po
el.po
es.po
fr.po
ja.po
nl.po
sv.po
tr.po
ishikawa@duron$ for f in *.po
> do
> /usr/local/bin/msgfmt --check --gcc $f
> done
I was suspicious of msgfmt functionality and so modified the ja.po file to
contain invalid construct: I mixed non-positional and positional
specifier in one message string. Then, msgfmt did spew
out error.
ishikawa@duron$ !for
for f in *.po ; do /usr/local/bin/msgfmt --check --gcc $f; done
ja.po:174: 'msgstr' is not a valid C format string, unlike 'msgid'. Reason: The string refers to arguments both through absolute argument numbers and through unnumbered argument specifications.
/usr/local/bin/msgfmt: found 1 fatal error
ishikawa@duron$
So msgfmt does check for errors of some sort.
Then how come we don't get errors from PO files at all?
There were obviously some invalid translations
by visual inspections in various PO files. But msgfmt didn't complain
about them.
It turns out, then, most (or all?) of the problematic translations
were marked "fuzzy" (!), and not checked at all.
Now I believe that translators without being able to obtain help about
unknown format-like specifiers such as %H, %T, %D simply marked dubious
translation as "fuzzy" so that msgfmt won't complain :-(
[Or didn't translate them at all.]
I found out that some translation simply ripped out %H, and other
non-standard format specifiers (and leave familiar
specifiers such as %s). Agh.
There was also a very minor problem in msgfmt itself in that if msgid
contained invalid specifier such as %Z, msgfmt would not complain and
not check the translation at all(!) and move on to the next message.
Considering that msgid is a (presumably) valid format string taken out
from correct(!) C source code, the chance of having invalid format
character there is small, but I would rather see a check here, too.
One other thing.
It seems that most of the PO files were rather outdated. (At least
the ones under my gcc mainline cvs check out hierarchy.)
Strings which we discussed in this mailing list, for example, those
that start with "%H..." simply don't appear in many PO files (!?):
either they don't appear at all or don't get translated since they are
new, or marked as fuzzy, etc..
No wonder we don't get warning/error from msgfmt.
We need some serious awareness campaign efforts to the translation
community to fix this state of the affairs by offering the improved
tool and hopefully the positional parameter support soon. (I attach
the draft note to translation community. Comments/feedback welcome.)
Happy Hacking,
Ishikawa, Chiaki
---
(Draft) Note to the GCC PO file translators
One complication for translators is the
appearance of format strings that need to
be understood by internal GCC diagnostic routine.
These have similar characteristics as format strings passed
to C functions such as
- sprintf
- snprintf
- fprintf
- printf
etc.
I summarize the caution to PO translators regarding
these strings.
[] Note to the GCC message translators. PO file string pecularities.
General warning.
This part of warning is applicable to all GNU programs written
using C.
Strings meant for C's printf-like function (a format string),
reference argument with a percent character "%" followed
by a specifier character.
E.g.,
%d ... integer value.
%c ... print a character
%s ... print a string.
etc.
Argument reference order must be preserved in the translation.
But you can explicitly use "postional notation" to specify
the particular argument that is referenced by the format specifier.
[..] Position of the argument is counted as follows.
Example:
printf-like function ("format string", arg1, arg2, arg3)
format string itself is considered at the postion 0.
The following argument is numbered as arg1, arg2, and arg3.
They are counted from 1 to as many as necessary.
(So generally speaking, the format-like string can
be at any postion, not necessarily the first argument to
printf-like function. It is only necessary for us to
regard the format-string argument as being the origin of
counting, and the following arguments are counted as no. 1,
no. 2, and no.3, etc..)
Another example:
The following format string assumes that
an integer value, and a string value (character pointer in C
parlance) follows in this order.
"Message referring to integer %d and string %s in this order."
In this case, an integer value argument, argument no. 1, and
a character pointer value argument, argument no. 2,
are assumed to follow this format message.
Let us explore the sutblety of the argument order references in
translated messages of this original message.
I show acceptable and non-accetable translation messages
with regard to the reference order.
OK: "Translation referring to integer %d and string %s in this order."
A translated message referring to the argument in this order is OK.
Note that "%d" and "%s" appear in this order as in the original
message.
NG: "Translation referring to string %s and integer %d"
You can't change the order of reference in translation!
The above translation refers to the argument values in wrong order.
However, as a translator, we often are faced with the original
English messages which could be translated into
more natural message in local languages if we can change the order
of references to argument values.
How can we do this?
Use of positional parameter makes it possible to
shuffle the reference order.
The following explicitly specifies that the string referred to
is the second argument and the integer referred to is the first
argument.
OK: "Translation referring to string %2$s and integer %1$d"
[..] Positional Paramter Usage Constraint.
You must use NONE positional notation or
ALL argument reference must be in POSITIONAL notation.
E.g.
Suppose we want to shuffle only the references to second and
third argument in this message.
"Message with %d, %c, and %s in this order"
NG "Message with %d %2$s, and %3$c in this order."
OK. "Message with %1$d %2$s, and %3$c in this order."
cf. Pedantically speaking,
"Message with %d, %c, and %s in this order",
"Message with %1$d, %2$c, and %3$s in this order"
should/would produce the same message.
[..] Exceptions
There are exception to the rule: %% and %m
%% which stands for a single percent line in the formated output
should remain as is.
%m should remain as is also.
%m in the format message passed to GCC diagnostic routines
stands for the standard
error message that is printed by the implictly
given errno value. (errno is
a C library value that indicates an immediately preceding error
condition. %m prints the corresponding error message string to
that value. We don't pass errno value to
diagnostic routines since it is shared throught inside the GCC
compiler frontend.)
So always use %% and %m even in the case of
positional parameter case. These two need not change the form.
[..] GCC's extended format character set.
A word of caution here. GCC's error/warning routines use
an extended set of format characters such as
%H, %T, %D, etc.
Since these are not found in standard C usage, they
may look strange, but must be treated like other format charcters.
I found out that many such format specifiers were
dropped from the translation. Or such strings are
marked fuzzy. It is wrong to do so!
(We will offer better msgfmt tool that understands
the extended format character set used by GCC's internal
diagnostic routines.)
[..] GCC format constraint/limit:
%H is one extension which is used by GCC diagnostic routines.
%H at the beginning of a string should not be moved.
It must be the first format reference in translated string.
This is how %H expectes to be used in format strings to
GCC internal diagnostic routines.
Subtle Implication.
If we need to use positional parameter notation, then
since ALL format specifier must use positional notation,
%H becomes %1$H. But it still needs to be
at the beginning of the format string.
Example:
"%H ...%x ... %y "
after translation and shuffling of the latter two
argument references.
"%1$H ... %3$y ... %2$x"
Again, this rule becomes only important when there are three or
more format specifiers and the first one being %H and we want to
shuffle the order of references to arguments using positional
notation.
We know that there are only about a dozen such messages in GCC
suite currently (2003 July), and so there should not be many
problems with this addtional rule.
Let us summarize.
The rule then is to leave the first %H reference intact at the
beginning of the string, and if we use postional notation, %H
becomes %1$H and it still must be at the beginning of the format
string.
[end of memo]
---
Crude hack to format-awk.c in gnu gettext().
(You need to modify msgfmt.c to understand --gcc and
sets gcc_warning_extended_format_check to true if the
option is given.)
*** format-awk.c.saved 2003-07-31 02:02:08.000000000 +0900
--- format-awk.c 2003-07-31 03:07:06.000000000 +0900
***************
*** 34,39 ****
--- 34,43 ----
#define _(str) gettext (str)
+
+ int gcc_warning_extended_format_check;
+
+
/* awk format strings are described in the gawk-3.1 documentation and
implemented in gawk-3.1.0/builtin.c: format_tree().
A directive
***************
*** 62,68 ****
FAT_STRING,
FAT_INTEGER,
FAT_UNSIGNED_INTEGER,
! FAT_FLOAT
};
struct numbered_arg
--- 66,103 ----
FAT_STRING,
FAT_INTEGER,
FAT_UNSIGNED_INTEGER,
! FAT_FLOAT,
!
! /* GCC extension. */
! /* We dont care what they are. Just be sure to
! cp_printer() in gcc/gcc/cp/error.c
! recognize them as individual/different specifiers.
! case 'A': result = args_to_string (x_next_tree, verbose); break;
! case 'C': result = code_to_string (x_next_tcode); break;
! case 'D': result = decl_to_string (x_next_tree, verbose); break;
! case 'E': result = expr_to_string (x_next_tree); break;
! case 'F': result = fndecl_to_string (x_next_tree, verbose); break;
! case 'L': result = language_to_string (x_next_lang); break;
! case 'O': result = op_to_string (x_next_tcode); break;
! case 'P': result = parm_to_string (x_next_int); break;
! case 'Q': result = assop_to_string (x_next_tcode); break;
! case 'T': result = type_to_string (next_tree, verbose); break;
! case 'V': result = cv_to_string (next_tree, verbose); break;
!
! c-objc-common.c uses 'D', 'F', 'T', and 'E'. They are covered by above.
! toplev.c uses 'D', 'F', and 'T'.
! */
! FAT_GCC_EXT_A,
! FAT_GCC_EXT_C,
! FAT_GCC_EXT_D,
! FAT_GCC_EXT_F,
! FAT_GCC_EXT_L,
! FAT_GCC_EXT_O,
! FAT_GCC_EXT_P,
! FAT_GCC_EXT_Q,
! FAT_GCC_EXT_T,
! FAT_GCC_EXT_V
!
};
struct numbered_arg
***************
*** 328,333 ****
--- 363,393 ----
type = FAT_FLOAT;
break;
default:
+ if(gcc_warning_extended_format_check)
+ {
+ #define SET_GCC_EXTENDED(label,c) case label : type = FAT_GCC_EXT_##c; break;
+ switch(*format)
+ {
+ SET_GCC_EXTENDED('A',A);
+ SET_GCC_EXTENDED('C',C);
+ SET_GCC_EXTENDED('D',D);
+ SET_GCC_EXTENDED('F',F);
+ SET_GCC_EXTENDED('L',L);
+ SET_GCC_EXTENDED('O',O);
+ SET_GCC_EXTENDED('P',P);
+ SET_GCC_EXTENDED('Q',Q);
+ SET_GCC_EXTENDED('T',T);
+ SET_GCC_EXTENDED('V',V);
+
+ default:
+ goto bad; /* unknown */
+ }
+ #undef SET_GCC_EXTENDED
+ break; /* get out of the outer switch. */
+
+ }
+ bad:;
+
*invalid_reason =
(*format == '\0'
? INVALID_UNTERMINATED_DIRECTIVE ()
---
--
int main(void){int j=2003;/*(c)2003 cishikawa. */
char t[] ="<CI> @abcdefghijklmnopqrstuvwxyz.,\n\"";
char *i ="g>qtCIuqivb,gCwe\np@.ietCIuqi\"tqkvv is>dnamz";
while(*i)((j+=strchr(t,*i++)-(int)t),(j%=sizeof t-1),
(putchar(t[j])));return 0;}/* under GPL */