Build errors

[David Malcolm]

On Wed, 2013-10-23 at 04:13 -0700, Simon Feltman wrote:
> Hi,
> I think this is an exciting project, many thanks! I am interested in
> the potential to use pygccgit as a runtime Python binding generator
> within PyGI (aka PyGObject3) but have hit a few initial snags.

Sounds interesting.

I'm thinking of maybe rewriting the Python bindings to use cffi rather
than Cython - that way they would work with PyPy.  As a potential user,
any thoughts on that?
(alternatively, we could have two different implementations; not sure if
that's sane).

> I've attempted to build things following the wiki instructions and ran
> into this error:
> ../../src/gcc/jit/internal-api.c:1479:21: error: ‘error’ was not
> declared in this scope

Sorry, my bad.  Should be fixed as of:
http://gcc.gnu.org/git/?p=gcc.git;a=commit;h=0413c323a4ceefe7ab163dd3cc4505853e583854

> I removed the offending line to see if it would continue and hit this:
> make[4]: Entering directory
> `/home/simon/projects/jit/build/x86_64-unknown-linux-gnu/32/libgcc'
> make[4]: *** No rule to make target `all'.  Stop.
> make[4]: Leaving directory
> `/home/simon/projects/jit/build/x86_64-unknown-linux-gnu/32/libgcc'
> make[3]: *** [multi-do] Error 1
> make[3]: Leaving directory
> `/home/simon/projects/jit/build/x86_64-unknown-linux-gnu/libgcc'
> make[2]: *** [all-multi] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make[2]: Leaving directory
> `/home/simon/projects/jit/build/x86_64-unknown-linux-gnu/libgcc'
> make[1]: *** [all-target-libgcc] Error 2
> make[1]: Leaving directory `/home/simon/projects/jit/build'
> make: *** [all] Error 2

The above appears to be during the build of the "libgcc" library, which
gets used at runtime on the target to support the code compiled by gcc.
I'm not sure what went wrong, but things may just work with the libgcc
you already have on your machine.

> However it looks like libgccjit.so was built after all, so I'm not
> sure what the above make error is all about:
> $ file gcc/libgccjit.so
> gcc/libgccjit.so: ELF 64-bit LSB shared object, x86-64, version...
> 
> The pygccjit code on github was a bit stale so I updated it to match
> the new "get_type" API with an enum arg and submitted a pull request.

Oops.  Thanks - looks good; I'm merged it.

BTW, note that as I'm hoping for the core jit branch to eventually be
merged into gcc itself, for *that* code I have to follow the rules given
here:
http://gcc.gnu.org/contribute.html
In particular, I can only accept patches for that code from people
who've done the appropriate legal paperwork with the FSF.

But the jittest and pygccjit projects are separate projects and don't
have to follow that rubric, so I'll happily accept pull requests and the
like.

> But I'm now hitting this when running test.py:
> $ python3 test.py
> .libgccjit.so: internal compiler error: Segmentation fault
> 0x7fad42a2a0e6 crash_signal
>         ../../src/gcc/toplev.c:336
> 0x7fad426c33c5 gcc::jit::function::add_conditional(gcc::jit::location*,
> gcc::jit::rvalue*, gcc::jit::label*, gcc::jit::label*)
>         ../../src/gcc/jit/internal-api.c:944
> 0x7fad436a04ba __pyx_pf_6gccjit_8Function_8add_conditional
>         /home/simon/projects/pygccjit/gccjit.c:3795
> 0x7fad436a04ba __pyx_pw_6gccjit_8Function_9add_conditional
>         /home/simon/projects/pygccjit/gccjit.c:3766
> 0x7fad4369f1d8 __pyx_f_6gccjit__c_callback
>         /home/simon/projects/pygccjit/gccjit.c:956
> 0x7fad426c2485 gcc::jit::context::invoke_code_factory()
>         ../../src/gcc/jit/internal-api.c:1345

I'm not seeing that (with Python 3.2 fwiw).

What version of the dmalcolm/jit branch are you using?   I added a lot
of error-checking last Friday in
d4019d9ddb77354c242798596268c7043bb7f3dd.

There's also a bug in GCC's internal error handling, which is fixed on
master, and isn't yet in my branch, which I've been applying locally
(r203810 on svn trunk, for the record).  I'll merge things shortly.


> Some additional thoughts on pygccjit: In terms of API for the enums, I
> think it would nice to follow more of a hierarchy. So instead of:
>     gccjit.FUNCTION_EXPORTED
> It could be:
>     gccjit.FunctionKind.EXPORTED

(nods).   I'm actually thinking of changing that enum so it applies to
visibility in general e.g. for globals.  So that specific one might
become GCC_JIT_VISIBILITY_EXPORTED or somesuch in the C API (not sold on
that yet).  So would that become gccjit.Visibility.EXPORTED ?

> I see that Result.get_code is already using ctypes internally but I
> think it would be nice to add more public API parity, or even the
> ability to use ctypes as the front end for building out initial
> function signatures:
>     sig = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_int)
>     fn = ctxt.new_function_from_ctypes("foo", sig)

The ctypes usage is currently a horrible hack, and anything we do in
that area is likely to be an improvement :)

Your proposal looks promising.  However, note that objects in the
context callback can't be accessed from outside the callback - they are
garbage-collected.

I think it's logical for the user to (optionally) specify the signature
when creating the function, within the callback, but we need to use it
when wrapping the (void*) from the result object.   So I think we need
the Context wrapper object to store a dict, mapping from function names
to ctypes signatures, which it would then hand off to the Result wrapper
object.   The latter could then use this to create the ctypes wrapper
around the (void*) in Result.get_code (or maybe "get_callable"?), thus
handing a python callable back to the client code - if that makes sense.

> or pull in a function symbol from ctypes for calling with in jit code:
>     incref = ctxt.import_function_from_ctypes(ctypes.pythonapi.Py_IncRef)
>     ...
>     fn.add_function_call(incref, pyobj)
> 
> What do you think?

Good idea.   Though this will require some support that's currently
missing from the C API: we don't yet have a way to take a function
pointer in C and call it from the JIT code.  (It's in the TODO.rst)


Thanks!
Dave