Re: Python API (was Re: Build errors)

[Simon Feltman <s.feltman@gmail.com>]

On Wed, Oct 23, 2013 at 9:07 AM, David Malcolm <dmalcolm@redhat.com> wrote:
> I'm thinking of maybe rewriting the Python bindings to use cffi rather
> than Cython - that way they would work with PyPy.  As a potential user,
> any thoughts on that?
> (alternatively, we could have two different implementations; not sure if
> that's sane).

cffi might also make the bindings a bit easier to write/flush out the
API with as well? I have a little experience with cffi, the obvious
thing is it will be slower than static bindings. But this doesn't
really mean anything until it is tested along with the jit compilation
times in a real project. In this regard, I guess flushing out the API
with whatever you think is easier or initially more benifitial is the
way to go (this could also mean sticking with cython bindings since
they already exists). How much larger do you think the API will grow?

>> Some additional thoughts on pygccjit: In terms of API for the enums, I
>> think it would nice to follow more of a hierarchy. So instead of:
>>     gccjit.FUNCTION_EXPORTED
>> It could be:
>>     gccjit.FunctionKind.EXPORTED
>
> (nods).   I'm actually thinking of changing that enum so it applies to
> visibility in general e.g. for globals.  So that specific one might
> become GCC_JIT_VISIBILITY_EXPORTED or somesuch in the C API (not sold on
> that yet).  So would that become gccjit.Visibility.EXPORTED ?

Seems nice to generalize it for usage with globals as well. It
probably goes without saying but arguments of this type should also be
named "visibility" or better yet annotated with
"visibility:gccjit.Visibility" or documented as such.

> I think it's logical for the user to (optionally) specify the signature
> when creating the function, within the callback, but we need to use it
> when wrapping the (void*) from the result object.   So I think we need
> the Context wrapper object to store a dict, mapping from function names
> to ctypes signatures, which it would then hand off to the Result wrapper
> object.   The latter could then use this to create the ctypes wrapper
> around the (void*) in Result.get_code (or maybe "get_callable"?), thus
> handing a python callable back to the client code - if that makes sense.

That makes sense, so as noted in the code comments below?:

def cb(ctxt):
    sig = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_int)
    # the following will store sig in a table on ctxt
    fn = ctxt.new_function_from_ctypes("foo", sig)
result = ctxt.compile()  # result needs to store a ref to ctxt
# result uses sig from ctxt table to contruct ctypes callable
code = result.get_code(b"square")

With this scheme, I guess the standard API of ctxt.new_function would
internally create a CFUNCTYPE with a signature mapped from the gccjit
types for use as a callable later on.

I think "get_callable" or even "get_pycallable" might be a bit more
understandable to a reader. The "py" prefix, albeit somewhat ugly,
gives a sense of orientation in regards to a jit construct vs a python
construct.

One thing I've noticed (this also stood out to me when playing with
llvmpy in the past) is types need to be created (and bound?) to a
context. This is well beyond my knowledge of compiler architecture,
but it seems like this could be cleaner?

    the_type = ctxt.get_type(gccjit.Type.INT)
    local_i = fn.new_local(the_type, b"i")

could be:
    local_i = fn.new_local(gccjit.Type.INT, b"i")

I'm sure there are performance and reference cycle implications with
this, and would probably convolute the internals. fn would need a
reference to the context which would need to hold a table of type
instances, etc.. (actually similar to the function building problem
above). It also might be too divergent from the C API which can be a
hard thing to balance with bindings. So I guess it depends how much
you want to diverge from the C API for Python convience.

-Simon