Caching one-time startup (and GNU Octave JIT)

[David Malcolm]

I've been experimenting with adding libgccjit support to GNU Octave and
I think I've found a flaw with the current libgccjit API - the lack of a
way to cache one-time startup.

GNU Octave already has a JIT compiler, using LLVM.  Octave has a
singleton class "jit_typeinfo" which performs one-time creation of the
various Octave types and callbacks, for example creating tables of
overloaded functions, mapping from (input type, function) pairs to
optimized handlers.
(See e.g.:
http://octave.sourceforge.net/doxygen/html/d2/da9/jit-typeinfo_8cc_source.html#l01072
the jit_typeinfo constructor tor, which performs this one-time
startup).

Then, when a function or loop becomes "hot", it gets JIT-compiled from a
parse-tree representation to a "jit_ir" representation, which then gets
turned into LLVM IR, and thence machine code.

Visually:

  once:
     create types, function wrappers, overload tables, etc (the
     jit_typeinfo singleton)

  when a function becomes "hot":
    parse tree -> jit_ir -> llvm -> machine code

In my experiment, I have a semi-working:

  when a function becomes "hot":
    parse tree -> jit_ir -> libgccjit -> machine code

but I'm running up against the lack of a way of doing one-time startup
that's shared between all instances of a compile.

Currently we have an idea of a context, and a compilation callback, and
all of the "contextual" entities have a lifetime that's only within the
compilation callback - you can't have e.g. a (gcc_jit_type *) that gets
stored at startup for later use by other compilations within the same
process.

This split into one-time startup vs per-hot function seems like a
natural thing for a real-world JIT compiler to want to do.

How should libgccjit support this?   Some ideas:

(A) Nested contexts: change lifetimes so that entities have the same
lifetime as the context they were created in, but can be used within
"child contexts" (with nesting).

So you can do:
   root_context: create the one-time stuff
     child_context (for one function): create stuff for just
        this function, using the stuff created in
        "root_context".
We could then perhaps eliminate result objects, so that e.g. you could
actually compile stuff on the root_context, and use it in a
child_context.

This would be one more entrypoint:
  gcc_jit_context_new_child_context
and it would make sense for anything that creates something that's
returned to take an explicit context ptr as an arg, since that affects
the lifetime of the returned thing.

[But how to implement? Perhaps change so that things become
heap-allocated "memento" objects, with a "free" of them all happening in
the release of the context (perhaps using obstack?).   Play it all back
during the compilation hook, with 2-way mappings between the mememtos
and the tree objects. Eventually we can (I hope) remove the mementos and
work directly with gcc objects].

(B) Explicit separation between one-time init vs per-compile: perhaps
have: gcc_jit_context (one-time init) vs gcc_jit_module
(an instance of a compile on a context, roughly corresponding to a
"Translation Unit").

(C) Something else?  Ideas?

I think typically you'd want to create types and imported functions
*once*, and then create exported functions as a per-compile thing.  But
I think people may want flexibility here e.g. to create some one-time
functions at startup, and then use them in the per-hot-function code.
So (A) may be superior to (B), perhaps.

Thoughts?  I'm most interested in what the API ought to look like [1]

Dave

[1] how it works internally is likely to involve actually rebuilding the
entities each time, at least until we can perform deeper surgery to how
gcc starts up e.g. being able to share GC heaps - but at least we can
have an API that *enables* such a refactoring.