64-bit integer typedef's and -fpic lead to infinite loop and growing memory use in port to x86-32

[H.J. Lu]

On Fri, May 28, 2021 at 12:10 PM Barnes, Richard
<Richard.Barnes@stratus.com> wrote:
>
> We are porting gcc-10.2.0 to a proprietary OS called VOS with a POSIX API that runs on x86-32. We are using a prior port of gcc-3.4.6 to build the port natively. When the build gets to the point where it compiles libgcc2.c with the gcc-10.2.0 compiler, it goes into an infinite loop and eventually runs out of virtual memory. We analyzed the failure by building libgcc2.c with -da and found that the build fails while compiling __mulvsi3.c. This build fails whether our build is using the reload pass or the LRA pass to run after the IRA pass.

I have no comment on your issue.  I am just curious why you don't build your
32-bit x86 OS with x86-64.

> What is contributing to the problem is that we must compile libgcc2.c with -fpic because the VOS runtime is position-independent with a different GOT for each thread, that the VOS ABI requires that when a function is entered, %edx must contain the GOTP, and that %edx must contain the GOTP at each call point, all of which is very different than the Unix ABI. Furthermore, GNU code requires that %ebx contain the GOTP wherever @PLT or @GOT relocation is used,. Since %edx also contains the upper half of the result of the multiply that adds to the register pressure.
>
> I am attaching mulvsi3.c, which exhibits the problem.
>
> During the failed builds, it looks like spilling keeps failing while trying to color or coalesce pseudo registers. I have built my own Chaitin register allocator that we use with our native compilers, and I know that we do not color the registers until no constrained registers are left after pruning the interference graph. If pruning fails to eliminate constrained registers, we spill and rebuild the interference graph and try again. While spilling does take time, this should not take too many passes to work. The behavior that the reload and LRA passes exhibits concerns me because that violates this rule. Also, I note that comments in the assign_by_spills() function show that the author was concerned about the possibility of multi-register reload-pseudos when the hard regs pool is fragmented. Something that could easily happen on x86-32.
>
> I would appreciate any advice that would help us deal with this problem.
>
> Thanks,
> Richard Barnes
>
> Stratus Technologies
>


-- 
H.J.