[Patch, fortran, RFC] PR 40958 Reduce size of module files

[Janne Blomqvist]

On Mon, Nov 28, 2011 at 11:21, Salvatore Filippone
<filippone.salvatore@gmail.com> wrote:
> On Sun, Nov 27, 2011 at 11:31 PM, Thomas Koenig <tkoenig@netcologne.de> wrote:
>>
>> Currently, module reading really is horribly inefficient.  What I
>> implemented on the fly last night was just a bit more sane one-byte
>> buffering.  As a minumum, we should implement a bigger buffer for module
>> files to reduce the seeking.

Yes, it should be quite easy to get rid of the seeking in the various
parse_*() functions, but as Mikael said, the real horror is
read_module() and, I think in particular, load_needed() which is
called by read_module().

> As Janne mentioned. an obvious solution would be to read the whole
> module file in memory and parse from there; this would be completely
> and safely orthogonal to writing the file in gzip format. A cache of
> read module files would also make sense.

I have thought a bit about this, but I'm leaning towards thinking that
I'm not sure it's that much less work than properly fixing the parsing
by inserting another parsing stage. That is, make an in-memory
representation that more or less matches the s-expr-like module syntax
(giving a simple and fast parser). Then do the rest of the module
loading work from that tree by changing the current parsing and
various fixup code to walk that tree instead of jumping back and forth
in the module file (or an in-memory array).

One might even use flex+bison for this 1st stage; in case there are
ambiguities in the grammar it's better to fix it with modest changes
to the module writing rather than making the parser more complicated.

Then one could add the module caching thing at some suitable point,
e.g. by caching the 1st-stage parsed module in case the full-blown
"module namespace" approach is too time-consuming to implement (as it
seems to be).

But alas, the usual constraint is the time to do it, rather than
coming up with high-flying ideas... :(

-- 
Janne Blomqvist