Whole-file compilation in gfortran

[Michael Matz]

Hi,

On Tue, 14 Aug 2007, FX Coudert wrote:

> A short mail about the topic of whole-file compilation.

Actually it's more about enabling whole program compilation.  Of the most 
interest (performance wise) is inlining over module borders (i.e. inlining 
a function F defined in module A, into a call in a different module B).

For that to happen it's obviously necessary to 
1) have the body of F available at the call site in module B
2) detect that the call in B actually refers to the function we think it
   refers to (namely F)

[beware of me not being a fortran expert, so my terminology might be 
completely wrong, e.g. when I talk about modules, but I hope to get the 
idea across anyway]

GCC can only compile (currently) one file at a time (let's ignore the 
C-only whole-program mode), so solving (1) normally requires quite some 
work, if the modules are compiled separately.  But fortran is a thankful 
language in that we can simply physically cat together all files of a 
program (hence all modules) and still have a valid input file for the 
compiler.  That then trivially solves problem (1), as the whole file is 
the program, hence all bodies of all functions remain available all time 
(from the point of definition).  This is obviously a hack, and instead of 
pre-catting the files before invoking the compiler it would be better if 
the compiler could do that for us.  That would also help debug generation 
as with the hack all debug info is relative to that special cat-together 
file, not referring to the originial source files.

But even with that problem solved, we still wouldn't get nice inlining 
because of problem (2).  That's because the fortran frontend generates a 
new FUNCTION_DECL for each get_extern_function_decl call, which is used 
for each function definition and each function call.  Hence all calls will 
have different FUNCTION_DECLs, in turn being all different to all decls 
for the function definitions.  Calls to the same function name will happen 
to have different FUNCTION_DECLs, but with the same name.

That is a problem because the call graph code determines the function 
"calls-this-function" based on the FUNCTION_DECL nodes itself.  So call 
graph doesn't see that we actually have a function definition for the 
calls, hence no inlining occurs.

So my patch was to target that problem (2).  The eventual goal is, that 
after the frontend is finished, there should be only exactly one 
FUNCTION_DECL node for each function definition, and that very function 
decl should be used in CALL_EXPRs (if the calls are not to undefined 
functions obviously.  In that case we need to generate a new 
FUNCTION_DECL, but it's good if other calls to the same undefined 
function would use the same FUNCTION_DECL node).

To do that we have to enter the merge-decl business.  That is because it 
can happen that we first see a call to a yet unknown function (hence 
generating a new FUNCTION_DECL), but later see a definition which would 
match that.  In that case we would have to merge the new information (for 
instance that we now have that function defined at that place) into the 
old FUNCTION_DECL node.  The same can happen if we see two calls with 
different argument lists.  As long as we don't have the definition we 
can't be sure of the formal argument types of that FUNCTION_DECL.

And as you say it might expose some deficiencies in the current compiler, 
ala ICEing when we see a call to a FUNCTION_DECL but the arguments don't 
match.  There might be other corner cases in fortran where my simple 
approach can not be used, I don't know.

> As far as I understood, there are 3 different issues to be tackled that this
> would help us :
>  (a) perform better inter-file checking of interfaces,
>  (b) inline non-contained procedures, and
>  (c) module loading performance

I believe (a) and (c) are only a problem if you want to solve my problem 
(1) for real, i.e. when you really want to support multiple files for 
whole program mode.  That would be the best of course.

> I will not talk about (a) here, as it's probably the most tricky. As far 
> as I see, (c) requires us to store modules into memory in addition to 
> writing them to disk, so that we can load symbols from them without 
> actually reading/parsing the file back. There might be more to it, Paul 
> could tell.

It's at least required to also get to the bodies of the functions in those 
other modules.  If you don't want to parse the source again, then you will 
have to have a different mean to store and read back general trees, at 
which point you'll have solved the LTO problem.  So I think before wasting 
too much time in really supporting multi-file-whole-program mode it's 
better to wait for LTO to move forward.  That doesn't mean that we 
couldn't already start working towards forming a nicer call graph by using 
just one FUNCTION_DECL per entity.


Ciao,
Michael.