Re: [RFC] Adding Python as a possible language and it's usage

[David Malcolm <dmalcolm at redhat dot com>]

On Wed, 2018-07-18 at 11:51 +0200, Richard Biener wrote:
> On Tue, Jul 17, 2018 at 2:49 PM Martin Liška <mliska@suse.cz> wrote:
> > 
> > Hi.
> > 
> > I've recently touched AWK option generate machinery and it's quite
> > unpleasant
> > to make any adjustments. My question is simple: can we starting
> > using a scripting
> > language like Python and replace usage of the AWK scripts? It's
> > probably question
> > for Steering committee, but I would like to see feedback from
> > community.
> > 
> > There are some bulletins why I would like to replace current AWK
> > scripts:
> > 
> > 1) gcc/optc-save-gen.awk is full of copy&pasted code, due to lack
> > of flags type classes multiple
> > global variables are created (var_opt_char, var_opt_string, ...)
> > 
> > 2) similar happens in gcc/opth-gen.awk
> > 
> > 3) we do very many regex matches (mainly in gcc/opt-functions.awk), 
> > I believe
> >    we should come up with a structured option format that will make
> > parsing and
> >    processing much simpler.
> > 
> > 4) we can come up with new sanity checks of options:
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81397
> > 
> > 5) there are various targets that generate *.opt files, one example
> > is ARM:
> > gcc/config/arm/parsecpu.awk
> > 
> > where transforms:
> > ./gcc/config/arm/arm-cpus.in
> > 
> > I guess having a well-defined structured format for *.opt files
> > will make
> > it easier to write generated opt files?
> > 
> > I'm attaching a prototype that can transform optionlist into
> > options-save.c
> > that can be compiled and works.
> > 
> > I'm looking forward to a feedback.
> 
> I guess we either need to document python as build requirement in
> install.texi then,

> it currently has
> 
> @item A POSIX or SVR4 awk
> 
> Necessary for creating some of the generated source files for GCC@.
> If in doubt, use a recent GNU awk version, as some of the older ones
> are broken.  GNU awk version 3.1.5 is known to work.
> 
> alternatively we could handle the generated files like those we still
> need flex for:

If we go down the "document Python as a build requirement" route, we
would need to decide on a minimum version, and what to do about Python
2 vs Python 3.  We could restrict ourselves to the common subset of the
two languages, or to require Python 3 (or Python 2, I suppose).

If we want somewhat conservative minimum versions, one strategy might
be to require (Python 2.* (2.6 or later) OR Python 3 (3.3 or later)),
and code to the common subset of 2.6 and 3.3.  Implicitly, this would
mean no 3rd-party modules; we'd be sticking to the Python standard
library.

Rationale:

Python 2.6 onwards is broadly compatible with Python 3.*. and is about
to be 10 years old.  (IIRC it was the system python implementation in
RHEL 6).  I'm guessing that many older systems have Python 2 installed,
but not Python 3, and anything we write is likely to be compatible with
even older Python 2.* implementations.

Python 3.3 reintroduced the 'u' prefix for unicode string literals (PEP
414), which makes it much easier to write scripts that work with both
2.* and 3.*.  Python 3.3 is almost 6 years old.

(this is just a suggestion)

Dave


> @item --enable-generated-files-in-srcdir
> Neither the .c and .h files that are generated from Bison and flex
> nor the
> info manuals and man pages that are built from the .texi files are
> present
> in the SVN development tree.  When building GCC from that development
> tree,
> or from one of our snapshots, those generated files are placed in
> your
> build directory, which allows for the source to be in a readonly
> directory.
> 
> If you configure with @option{--enable-generated-files-in-srcdir}
> then those
> generated files will go into the source directory.  This is mainly
> intended
> for generating release or prerelease tarballs of the GCC sources,
> since it
> is not a requirement that the users of source releases to have flex,
> Bison,
> or makeinfo.
> 
> We already conditionally require Perl for building for some targets
> so I wonder
> if using perl would be better ...
> 
> Do we get rid of the AWK build requirement with your changes?
> 
> Richard.
> 
> > Martin