Base new module format on XML - RFC and some questions

[Mikael Morin]

On Saturday 02 October 2010 19:13:30 Dennis Wassel wrote:
> I had another shot at trying to understand module.c, but there's just
> too much of it and I fail to get the greater picture. 

There is a specific read/write function for almost every gfortran internal 
structure. Those are called mio_<whatever> and you don't need to care about 
them (That's roughly half of the file) if you don't change the format. 

If you want to know how the file is organized, open a sample module file, and 
take a look at write_module. Translating from C structs to file is easier than 
from file to C structs, so it should be understandable. Use pairs of parens in 
the file and pairs of mio_lparen()/mio_rparen() in the code to orient 
yourself.

Now the big picture.

Syntax:
There are 5 types of tokens: NAME, LPAREN, RPAREN, INTEGER, STRING. 
LPAREN/RPAREN are for grouping
INTEGER are for pointers and integral/boolean values (ranks, 
attributes/flags).
STRING are for strings (identifier names)
NAME are for attributes and enums. 
 
Reading:
The entry point is gfc_use_module. But there is little interesting there, 
everything is in read_module. 
How we read: 
- we first skip all the first part of the file until the list of symbols. 
- for each symbol we read the integer for the pointer_info (more on that 
later), the name, the module it comes from, and the binding label. We skip 
the rest for now. We register the symbol location in the pointer_info for 
later use.
- After the symbol list, there is the symtree list. For each symtree, we check 
whether it is needed (think of use, only) and if it is, mark the pointer_info 
corresponding to the symbol as NEEDED. 
- Then we seek back to the beginning of the file and load operator interfaces, 
user operators, generic interfaces, commons and equivalences. While loading 
these, we will be marking associated symbols as needed. 
- Then we keep calling load_needed as long as we actually load something.
load_needed traverses the pointer_info tree. For each pointer_info marked as 
NEEDED, we seek to the symbol location, read the symbol information and fill a 
gfc_symbol from that. 
Loading symbols can make new symbols to be marked as needed (think of dummy 
arguments for procedures), so the next call to load_needed will take care of 
it. Note that write_module uses a similar approach to write only needed 
symbols. 
- Then we load the derived type extensions. I don't know why it's not done 
before. 

Now to finish the big picture, what is a pointer_info ?
As explained above integers are used to represent pointers. Each pointer that 
needs to be written is assigned an index number by which it is referred to in 
the module file. 
A pointer info contains both the index and the pointer. It is used both for 
reading and for writing writing a module file. 
For symbol pointers, it also contains the information whether the symbol is 
needed, loaded/written, or unneeded. 
While reading, if we encounter the index before the symbol is actually 
loaded/created, we save in the fixup list all the symbol pointer addresses 
pointing to the (not yet existing) symbol corresponding to this index. When 
the pointer is created, we set the symbol pointer at these addresses. 


That's what I know. For the rest, you have to read the code. 

I must admit that there are many small parts that I don't understand myself. 
But unless they are touching the very part you are interested in, you can 
ignore them. Most of the time they take care of a corner case you haven't 
thought of. Sometimes removing them and seeing what happens can help ;-).


> The "64 million
> dollar question" remains unanswered: Does the module stuff have to do
> anything more than translate between a gfc_symtree and a file?
It's more a gfc_namespace (containing a gfc_symtree) than a gfc_symtree alone. 
This includes commons, equivalences, operators and so on. 
Now it's answered; can I get my 64 million? 

> 
> One of Jerry's comments is particularly good food for thought, I think:
> > Why does the module file need to be expressive?  Fortran source is
> > already as expressive as is needed. No need to create a new programming
> > language.
> 
> Very true! This got me thinking why module files aren't just
> (annotated, stripped-down, whatever) Fortran source code files? When
> the current s-expr format is actually less concise than the source,
> what is the point?
User-written fortran may be concise, but auto-generated fortran won't be for 
sure. 
I think that would be a nice module file format however. For gfortran 5.0 ?

> the current incarnation of module.c is just too hard for me.
module.c is not that bad. As each structure has its own io function, one can  
get the picture without descending into the low level functions. 
I think there are worse places actually. 


Mikael