Re: [RFC] Adding Python as a possible language and it's usage

[David Malcolm <dmalcolm at redhat dot com>]

On Tue, 2018-07-17 at 14:49 +0200, Martin Liška wrote:
> Hi.
> 
> I've recently touched AWK option generate machinery and it's quite
> unpleasant
> to make any adjustments. My question is simple: can we starting using
> a scripting
> language like Python and replace usage of the AWK scripts? It's
> probably question
> for Steering committee, but I would like to see feedback from
> community.

As you know, I'm a fan of Python.  As I noted elsewhere in this thread,
one issue is Python 2 vs Python 3 (and minimum versions).  Within
Python 2.*, Python 2.6 onwards is broadly compatible with Python 3.*,
and there's a well-known common subset that works in both languages.

To what extent would this complicate bootstrap?  (I don't think so, in
that it would appear to be just an external build-time dependency on
the build machine).

Would this make it harder for people to build GCC?  It's one more
dependency, but CPython is widely available and relatively easy to
build.  (I don't have experience of doing bring-up of a new
architecture, though).

> There are some bulletins why I would like to replace current AWK
> scripts:
> 
> 1) gcc/optc-save-gen.awk is full of copy&pasted code, due to lack of
> flags type classes multiple
> global variables are created (var_opt_char, var_opt_string, ...)
> 
> 2) similar happens in gcc/opth-gen.awk
> 
> 3) we do very many regex matches (mainly in gcc/opt-functions.awk), I
> believe
>    we should come up with a structured option format that will make
> parsing and
>    processing much simpler.
> 
> 4) we can come up with new sanity checks of options:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81397

Having some kind of .opt linting sounds useful.

> 5) there are various targets that generate *.opt files, one example
> is ARM:
> gcc/config/arm/parsecpu.awk
> 
> where transforms:
> ./gcc/config/arm/arm-cpus.in
> 
> I guess having a well-defined structured format for *.opt files will
> make
> it easier to write generated opt files?
> 
> I'm attaching a prototype that can transform optionlist into options-
> save.c
> that can be compiled and works.
> 
> I'm looking forward to a feedback.
> Martin

You named it "gcc-options.py", but I think we'll want something that
can be imported from other scripts, and this isn't valid to "import" as
a module, due to the "-".   It should have a filename that either uses
an underscore, or no separator.

> # parse content of optionlist
It's probably worth moving this into a class.  Maybe:

class OptionList:
    def __init__ (self, lines):
       # etc

or similar.

"optimization_flags" could be a member of that class.


> # start printing
This ought to be in a function, rather than having this at the top-
level.

Moving it into a function would allow for some unittest tests:
(a) tests of parsing some lines provided as string literals, to unit-
test the parser.

(b) integration tests of parsing the actual optionlist, maybe.

perhaps via a --unit-test command-line option to trigger
unittest.main().


Maybe a way to ensure no semantic changes during the transition would
be to diff the generated .c/.h files compared to the awk files, and
verifying that there are no significant whitespace changes, for all
supported configs?

Hope this is constructive.
Dave