Re: How to grow the Fortran I/O parameter struct and keep ABI compatibility

[Jakub Jelinek <jakub at redhat dot com>]

On Tue, Nov 07, 2006 at 10:59:05AM +0100, FX Coudert wrote:
> {
>   st_parameter_common common;
>   GFC_IO_INT rec;
>   [...lots of other struct members...]
>   /* Private part of the structure.  The compiler just needs
>      to reserve enough space.  */
>     union
>     {
>       struct
>         {
>       [... the private stuff ...]
>         } p;
>       /* This pad size must be equal to the pad_size declared in
>          trans-io.c (gfc_build_io_library_fndecls).  The above  
> structure
>          must be smaller or equal to this array.  */
>       char pad[16 * sizeof (char *) + 32 * sizeof (int)];
>     } u;
> }
> st_parameter_dt;
> 
> The idea is that common.flags has a bit for every possible member  
> such as rec, to indicated whether it's present or not. The question  
> is that we sometimes need to add another struct member (like rec) in  
> this structure, to implement new features. I think we can add it at  
> the end, since when code generated by older compilers calls the  
> library, the flag for that new member is not set, and the struct  
> member is never accessed.

I designed it this way so that it is as fast as possible (can be
on the caller's stack and as few fields in the memory have to
be written as possible (the older struct was always cleared as whole)).

I think you generally can only keep backwards compatibility, not forward
compatibility (so gfortran 4.3 compiled objects should work with libgfortran
4.3, 4.4 etc., but 4.4 compiled objects if there were changes in the
struct shouldn't be expected to work with libgfortran 4.3).
This can be handled by symbol versioning, which I saw patches floating
around for.

If you want to add over time new fields, if there is space in
common.flags, you just define a new bit there and add the new field
to the end of u.p structure if there is space for it.  That's the
easy part.

If you are out of common.flags bits (or just one is left), you should
add another flags field and use the last bit in common.flags to
say "flags2 field is present".

For private space (i.e. something that you want to carry over from one
call with the same st_parameter_* argument to another with the same
argument), you can reuse all fields that aren't needed by the second
and following calls nor are used for returning value to the caller,
so if there is e.g. a field used for parameter which is only needed
by the initial call, you can just reuse it.  But, when you finally
run out of space in the struct, you can just increase the padding
and use another bit in common.flags (or flags2 when you get to it)
to say the whole struct is bigger.  Usually you will need extra
space only to implement new functionality (which has to be requested
by the caller), so if all that implementation is conditionalized
on some common.flags resp. flags2 bit, you should be backwards
compatible.  If you need extra space even for the backwards compatible
stuff, first try to move private things around so that stuff
that can be conditionalized on common.flags or flags2 bit is used
in the newly added memory space.  If even that fails, you can just
malloc/free a pointer to extra area if the bit which says the struct
is too short isn't set.

In any case, with symbol versioning when adding new feature bits
you should bump symbol version and add compatibility alias.

Say in GCC 4.5 you want to add new fields to struct st_parameter_open.
There is a free bit in common.flags, so you just allocate new
IOPARM_OPEN_HAS_FOOBAR bit and add foobar field at the end.
But, because libgfortran 4.4 won't grok that bit, you really
want to avoid GCC 4.5 compiled code to use libgfortran 4.4
_gfortran_st_open (== all routines that use the st_parameter_open
structure).  So, you have
internal_st_open (...)
{
  function definition;
}
#ifdef HAVE_SYMBOL_VERSIONING
extern __typeof (internal_st_open) st_open_4_5 __attribute__((alias ("internal_st_open")));
extern __typeof (internal_st_open) st_open_4_3 __attribute__((alias ("internal_st_open")));
asm (".symver st_open_4_5, _gfortran_st_open@@GFORTRAN_4.5");
asm (".symver st_open_4_3, _gfortran_st_open@GFORTRAN_4.3");
#endif

The two versioned symbols can have the same value if the new
st_open implementation is backwards but not forward compatible.

	Jakub