Fix RETURN_IN_MEMORY for x86 Solaris

[Richard Sandiford]

Richard Henderson <rth@redhat.com> writes:
> On Mon, Apr 23, 2007 at 09:41:43AM +0100, Richard Sandiford wrote:
>> #define RETURN_IN_MEMORY(TYPE) \
>>   (TYPE_MODE (TYPE) == BLKmode \
>>    || (VECTOR_MODE_P (TYPE_MODE (TYPE)) && int_size_in_bytes (TYPE) == 8))
>
> ...
>
>> +      /* Return in memory only if MMX registers *are* available.  This
>> +	 seems backwards, but it is consistent with the existing
>> +	 Solaris x86 ABI.  */
>> +      if (size == 8)
>> +	return TARGET_MMX;
>
> I don't see TARGET_MMX in the above definition, therefore your
> conversion is incorrect.  Note that RETURN_IN_MEMORY preempts
> whatever register function_value would have specified.

But the old definition is checking TYPE_MODE, not type_natural_mode,
whereas the new definition is checking type_natural_mode.  AIUI,
you won't get an 8-byte vector TYPE_MODE if !TARGET_MMX.

Richard