Incremental compilation

[Marc Nieper-Wißkirchen]

Am So., 30. Dez. 2018 um 09:46 Uhr schrieb Basile Starynkevitch
<basile@starynkevitch.net>:
>
>
> On 12/29/18 4:31 PM, Marc Nieper-Wißkirchen wrote:
> > Hi!
> I am guessing that you use libgccjit on Linux.
> >
> > I need to compile a number of functions incrementally. Eventually, I
> > would like to compile all the functions to a dynamic library file.
> Notice that, if you are compiling each function incrementally, it has to
> be some exported function (GCC_JIT_FUNCTION_EXPORTED), otherwise you
> won't see it between your different compilations.
> >
> > Can this be achieved with libgccjit without recompiling the functions?
>
> Probably not, and I don't understand why you need that in practice.

My use case is compiling a Scheme library. It contains macro
definitions whose transformation procedures have to be compiled at
first to expand subsequent parts of the library body. Furthermore, the
compiled transformation procedures of global macro defintions have to
appear in the final dynamic library as well.

>
> The common wisdom would be to compile each function separately (perhaps
> into its own shared object on Linux).
>
> Be aware that it is perfectly acceptable to have a big lot (e.g. many
> hundred thousands) of shared objects on Linux.

That's good to know!

So what I probably need is a kind of linker for libgccjit that
combines many of the compiled shared objects into one final module
that is then written to the disk (much like what LLVM's linker class
http://llvm.org/doxygen/classllvm_1_1Linker.html seems to achieve).

Another problem I have is that the resulting shared object has to note
its dependencies. However, libgccjit's documentation currently says:
"There is currently no support for specifying other libraries to link
against."

Is there a workaround or an ETA for this feature?

Thanks a lot!

-- Marc

P.S.: Where does libgccjit look for functions of kind
GCC_JIT_FUNCTION_IMPORTED? In the main executable and all dlopen'ed
libraries?

> My manydl.c program (which does not use libgccjit) on
> https://github.com/bstarynk/misc-basile/blob/master/manydl.c is
> generating some C code, compiling it as a plugin, and dlopen-ing it. It
> can repeat that a lot of times.
>
> The only caveat when compiling small functions separately is that you
> have to spend one (or a few) pages (of 4Kbytes) on every compilation,
> since the result of a compilation fits in a separate page (e.g. in
> different *.so files). So if the function is tiny and takes only a
> hundred byte, you lose some virtual address space. In practice, that is
> no much a problem (at least on x86-64 under Linux).
>
> My guess is that you are sort-of implementing some Read Eval Print Loop
> and JIT-compiling at every interaction. In practice, that works very
> well (even when emitting C code instead of using libgccjit). I was able
> to do that in my old GCC MELT project (which generated a C code,
> compiled it into a plugin, at every REPL interaction).
>
> Happy new year to everyone.
>
> --
>
> Basile STARYNKEVITCH   == http://starynkevitch.net/Basile
> opinions are mine only - les opinions sont seulement miennes
> Bourg La Reine, France
>


-- 
Prof. Dr. Marc Nieper-Wißkirchen

Universität Augsburg
Institut für Mathematik
Universitätsstraße 14
86159 Augsburg

Tel: 0821/598-2146
Fax: 0821/598-2090

E-Mail: marc.nieper-wisskirchen@math.uni-augsburg.de
Web: www.math.uni-augsburg.de/alg/mitarbeiter/mnieper/