Bug 81519 - Enhancement: Add --help=target-distcc or similar to dump clean, optimal CFLAGS without using -march=native
Summary: Enhancement: Add --help=target-distcc or similar to dump clean, optimal CFLAG...
Status: RESOLVED WONTFIX
Alias: None
Product: gcc
Classification: Unclassified
Component: driver (show other bugs)
Version: 8.0
: P4 enhancement
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-22 19:43 UTC by Daniel Santos
Modified: 2018-12-27 09:23 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2017-07-24 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Santos 2017-07-22 19:43:00 UTC
To be clear, there is a working solution and that is to run a command such as this:

gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | grep cc1 | perl -pe 's/^.* - //g;'

The problem is that it is very verbose and redundant.  You can filter out the -mno-* flags, but I'm not certain that that is always correct and still leaves many more -m<extention> flags.  Ideally, a command such as `gcc -Q -x c -march=native --help=target-distcc' would emit something like

CFLAGS="-march=sandybridge --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=sandybridge"

In addition to the above, the output should contain any -m<extension> and -mno-<extension> flags for each that actually differ from the arch and not include extensions that are already included in another extension.

Calculating this from a script would actually be possible if bug #39851 were fixed, but there is currently no (that I am aware of) way to get an output of extensions each arch consists of.  These extensions are listed in invoke.texi by marketing name, but this would still be a parsing challenge and is subject to human error since it isn't parsed from the backend code.

Alternatively, I presume that cleaning up the driver code to omit redundant options could result in the first example being sufficient, but I don't understand the implications of that and I suspect would likely break other things.
Comment 1 Martin Liška 2017-07-24 11:18:54 UTC
I can take a look later for GCC 8.0.
Comment 2 Eric Gallager 2017-07-24 23:59:35 UTC
(In reply to Martin Liška from comment #1)
> I can take a look later for GCC 8.0.

Changing status to ASSIGNED then
Comment 3 Daniel Santos 2017-07-26 00:39:38 UTC
(In reply to Martin Liška from comment #1)
> I can take a look later for GCC 8.0.

Thank you Martin!  I still don't understand enough of gcc to be able to do this in any reasonable time frame and I've only worked with the i386 backend thus far.
Comment 4 Martin Liška 2017-08-08 08:35:43 UTC
Ok, so I've briefly investigated source code and providing such information is definitely not a simple task :/

I would recommend to fix PR39851 and then one will just compare output of following 2 invocations:

gcc --help=target  -Q
gcc --help=target -march=native -Q 

Will it work for you?

Note that fully understand which ISA extensions are enable when is also quite complex.
Comment 5 Martin Liška 2017-08-10 07:46:15 UTC
Unassigning ..
Comment 6 Daniel Santos 2017-08-25 19:16:11 UTC
(In reply to Martin Liška from comment #4)
> Ok, so I've briefly investigated source code and providing such information
> is definitely not a simple task :/

Sorry for my late response and thanks for looking into this.  I too was a bit daunted when I started to investigate this just for the i386 back-end as there were a lot of twists and turns, 64-bit bitmasks that were out of bits requiring hack-arounds to add new processors, etc.

Maybe a nice-to-have would be a generic middle-end mechanism to manage this for all back-ends in a universal fashion so that this type of thing would be much easier to handle.  Of course, that would require an awfully large amount of analysis in order to create a design that works *and* makes sense for everyone, but such a design could also do a better job of things such as representing the linage of a microarchitecture, accessing the processor_costs, etc.
Comment 7 Daniel Santos 2017-08-30 05:04:12 UTC
(In reply to Martin Liška from comment #4)
> Ok, so I've briefly investigated source code and providing such information
> is definitely not a simple task :/
> 
> I would recommend to fix PR39851 and then one will just compare output of
> following 2 invocations:
> 
> gcc --help=target  -Q
> gcc --help=target -march=native -Q 
> 
> Will it work for you?
> 
> Note that fully understand which ISA extensions are enable when is also
> quite complex.

I've thought about this some more and I'm starting to think that all of this can be determined with a script that iteratively calls gcc --help=target -Q with various machine flags to determine which -mno-* flags are really needed and which -m<isa> flags include others.  So in effect, I'm thinking that we can produce optimal C(XX)FLAGS with a script and your PR39851 fix.  I'll have to test this out.

Thanks
Comment 8 Eric Gallager 2017-09-17 20:58:27 UTC
(In reply to Martin Liška from comment #5)
> Unassigning ..

You still left the status as ASSIGNED though. I'm setting it back to NEW now.
Comment 9 Eric Gallager 2018-11-18 05:17:45 UTC
(In reply to Daniel Santos from comment #7)
> (In reply to Martin Liška from comment #4)
> > Ok, so I've briefly investigated source code and providing such information
> > is definitely not a simple task :/
> > 
> > I would recommend to fix PR39851 and then one will just compare output of
> > following 2 invocations:
> > 
> > gcc --help=target  -Q
> > gcc --help=target -march=native -Q 
> > 
> > Will it work for you?
> > 
> > Note that fully understand which ISA extensions are enable when is also
> > quite complex.
> 
> I've thought about this some more and I'm starting to think that all of this
> can be determined with a script that iteratively calls gcc --help=target -Q
> with various machine flags to determine which -mno-* flags are really needed
> and which -m<isa> flags include others.  So in effect, I'm thinking that we
> can produce optimal C(XX)FLAGS with a script and your PR39851 fix.  I'll
> have to test this out.
> 
> Thanks

If you come up with a script, do you want to put it in contrib?
Comment 10 Martin Liška 2018-11-19 07:22:04 UTC
Note that I'm not planning to come up with a script. It's quite a concrete usage, so I would recommend you to put a script into your project.

Idea of the script can be:
$ gcc --help=target -Q &>/tmp/1 && gcc --help=target -Q -march=haswell &>/tmp/2 && diff -u /tmp/1 /tmp/2 | grep '^+'
+++ /tmp/2	2018-11-19 08:21:08.180761742 +0100
+  -maes                       		[enabled]
+  -march=                     		haswell
+  -mavx                       		[enabled]
+  -mavx2                      		[enabled]
+  -mavx256-split-unaligned-load 	[disabled]
+  -mavx256-split-unaligned-store 	[disabled]
+  -mbmi                       		[enabled]
+  -mbmi2                      		[enabled]
+  -mcx16                      		[enabled]
+  -mf16c                      		[enabled]
+  -mfma                       		[enabled]
+  -mfsgsbase                  		[enabled]
+  -mhle                       		[enabled]
+  -mlzcnt                     		[enabled]
+  -mmovbe                     		[enabled]
+  -mno-sse4                   		[disabled]
+  -mpclmul                    		[enabled]
+  -mpopcnt                    		[enabled]
+  -mrdrnd                     		[enabled]
+  -msahf                      		[enabled]
+  -msse3                      		[enabled]
+  -msse4                      		[enabled]
+  -msse4.1                    		[enabled]
+  -msse4.2                    		[enabled]
+  -mssse3                     		[enabled]
+  -mtune=                     		haswell
+  -mxsave                     		[enabled]
+  -mxsaveopt                  		[enabled]
Comment 11 Daniel Santos 2018-12-02 01:48:01 UTC
(In reply to Eric Gallager from comment #9)
> (In reply to Daniel Santos from comment #7)
> > (In reply to Martin Liška from comment #4)
> > > Ok, so I've briefly investigated source code and providing such information
> > > is definitely not a simple task :/
> > > 
> > > I would recommend to fix PR39851 and then one will just compare output of
> > > following 2 invocations:
> > > 
> > > gcc --help=target  -Q
> > > gcc --help=target -march=native -Q 
> > > 
> > > Will it work for you?
> > > 
> > > Note that fully understand which ISA extensions are enable when is also
> > > quite complex.
> > 
> > I've thought about this some more and I'm starting to think that all of this
> > can be determined with a script that iteratively calls gcc --help=target -Q
> > with various machine flags to determine which -mno-* flags are really needed
> > and which -m<isa> flags include others.  So in effect, I'm thinking that we
> > can produce optimal C(XX)FLAGS with a script and your PR39851 fix.  I'll
> > have to test this out.
> > 
> > Thanks
> 
> If you come up with a script, do you want to put it in contrib?

Interesting.  I thought I replied to this already, but maybe that was on a different bug report?  Anyway, I have a script at https://github.com/daniel-santos/distccflags but it has a flaw (https://github.com/daniel-santos/distccflags/issues/2) that I need to fix.  Other than that, it's fairly use-friendly.  It would probably be a lot cleaner re-written in perl.

I tried to push this off onto distcc but they wanted it re-written in C.
Comment 12 Martin Liška 2018-12-27 09:23:56 UTC
Let me close that as won't fix.