i386.c and _Optlink calling convention

[Андрей Заболотный <anpaza at mail dot ru>]

Good day!
(Sorry for previous crap, I accidentally hit the Send button :-()

I'm working on a OS/2 port of gcc and one of my goals is to achieve
a relatively full compatibility with the standard OS/2 compiler -
IBM's Visual Age C++. The later has a calling convention (often used)
calld _Optlink which is quite similar with regparm but has some
differences. The difference I have most troubles with is that it passes
first four floating-point parameters through FPU registers rather than
through the stack.

Alas, regparm doesn't do anything special about FP args, so I had to
do everything on my own. I'm far from being a gcc internals guru, so
maybe some of people with good knowledge of gcc internals can enlighten
me.

I've added support for _Optlink almost everywhere where it was required
in i386.c - init_cumulative_args, function_arg and function_arg_advance.
I've added the "[QHTSDX]F_mode" cases in function_arg(), and calling a
function with FP args works perfectly now. However, I have problems with
compilation of functions with FP args.

It looks like the code in reg-stack.c doesn't know (or I don't know how
to tell him) that some FPU registers can come in already filed. So, to
avoid internal confusion the code in reg-stack.c pushes some 'fld NaN'
instructions even before the stack frame is initialized. In the
following these registers are used 'correctly' (if they wouldn't be
initialized to NaNs), so I assume everything I did in i386.c is correct.

However, if I 'dirty fix' some code inside convert_regs_entry() this
way (sorry for the long quote):

-----------------------
  for (e = ENTRY_BLOCK_PTR->succ; e ; e = e->succ_next)
    {
      basic_block block = e->dest;
      block_info bi = BLOCK_INFO (block);
      int reg, top = -1;

      for (reg = LAST_STACK_REG; reg >= regstack_first_undef_reg; --reg)
        if (TEST_HARD_REG_BIT (bi->stack_in.reg_set, reg))
          {
            rtx init;
            bi->stack_in.reg[++top] = reg;
            init = gen_rtx_SET (VOIDmode,
                                FP_MODE_REG (FIRST_STACK_REG, SFmode),
                                nan);
            insert_insn_on_edge (init, e);
            inserted = 1;
          }                                                             
       
      for (reg = regstack_first_undef_reg - 1;
           reg >= FIRST_STACK_REG; --reg)
         bi->stack_in.reg[++top] = reg;

      regstack_first_undef_reg = FIRST_STACK_REG;
      bi->stack_in.top = top;
    }
----------------------
See, I've added regstack_first_undef_reg which is by default
FIRST_STACK_REG but in _Optlink functions it could range from
FIRST_STACK_REG to FIRST_STACK_REG+4 (set for now in i386.c).

So, now I don't get those 'fld NaN' at the beginning of the function,
but instead I'm getting random ICE's in different situations (ex. in
this function: float doit (float x, float y) { return y; }).

Ok, the letter is going too long so if anyone cares to reply me
(please do :-), I will go in deeper details - just ask.

And another small problem: _Optlink requires that those args that are
passed in registers should have a equivalent space reserved on the
stack. Currently I'm returning in function_arg() a
"gen_rtx (PARALLEL,...)", which means the value is stored *both* in
stack and register. This is not a problem since it doesn't contradict
the specs, but it is a performance impact, so I would gladly accept any
hints how I could avoid that.

--
Greetings,
   Andrew