[Bug jit/64722] On 2nd time libgccjit is run in-process on i686, generated code clobbers %ebx register

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64722

--- Comment #1 from David Malcolm <dmalcolm at gcc dot gnu.org> ---
After some debugging, root cause appears to be in initialization of
pic_offset_table_rtx.

In i386.h:
/* Register to hold the addressing base for position independent
   code access to data items.  We don't use PIC pointer for 64bit
   mode.  Define the regnum to dummy value to prevent gcc from
   pessimizing code dealing with EBX.

   To avoid clobbering a call-saved register unnecessarily, we renumber
   the pic register when possible.  The change is visible after the
   prologue has been emitted.  */

#define REAL_PIC_OFFSET_TABLE_REGNUM  (TARGET_64BIT ? R15_REG : BX_REG)

#define PIC_OFFSET_TABLE_REGNUM                        \
  ((TARGET_64BIT && (ix86_cmodel == CM_SMALL_PIC            \
                     || TARGET_PECOFF))                    \
   || !flag_pic                                \
   ? INVALID_REGNUM                            \
   : pic_offset_table_rtx                        \
     ? INVALID_REGNUM                            \
     : REAL_PIC_OFFSET_TABLE_REGNUM)

and in emit-rtl.c:init_emit_regs:
  if ((unsigned) PIC_OFFSET_TABLE_REGNUM != INVALID_REGNUM)
    pic_offset_table_rtx = gen_raw_REG (Pmode, PIC_OFFSET_TABLE_REGNUM);
  else
    pic_offset_table_rtx = NULL_RTX;

Adding some debugging prints there:
--- a/gcc/emit-rtl.c
+++ b/gcc/emit-rtl.c
@@ -5872,10 +5872,18 @@ init_emit_regs (void)
     = gen_raw_REG (Pmode, RETURN_ADDRESS_POINTER_REGNUM);
 #endif

+  debug (pic_offset_table_rtx);
   if ((unsigned) PIC_OFFSET_TABLE_REGNUM != INVALID_REGNUM)
-    pic_offset_table_rtx = gen_raw_REG (Pmode, PIC_OFFSET_TABLE_REGNUM);
+    {
+      printf ("calling gen_raw_REG\n");
+      pic_offset_table_rtx = gen_raw_REG (Pmode, PIC_OFFSET_TABLE_REGNUM);
+    }
   else
-    pic_offset_table_rtx = NULL_RTX;
+    {
+      printf ("not calling gen_raw_REG\n");
+      pic_offset_table_rtx = NULL_RTX;
+    }
+  debug (pic_offset_table_rtx);

   for (i = 0; i < (int) MAX_MACHINE_MODE; i++)
     {

For 32-bit PIC, there appears to be some circular logic in this
initialization; if this has already been called, then
pic_offset_table_rtx is non-NULL, so PIC_OFFSET_TABLE_REGNUM
is INVALID_REGNUM, and so on reinitialization, it transitions back to
NULL.

Indeed, with test-empty.c, we see the NULL/non-NULL-ness of
pic_offset_table_rtx toggle at each iteration:
ITERATION 1
        PASSED: test-empty.c.exe iteration 1 of 5: set_up_logging: logfile is
non-null
        NOTE: test-empty.c.exe iteration 1 of 5: writing reproducer to
./test-empty.c.exe.reproducer.c
 test-empty.c.exe iteration 1 of 5: writing reproducer to .
<nil>
calling gen_raw_REG
(reg:SI 3 bx)
intermediate files written to /tmp/libgccjit-n2vMx2
        PASSED: test-empty.c.exe iteration 1 of 5: verify_code: result is
non-null

ITERATION 2
        PASSED: test-empty.c.exe iteration 2 of 5: set_up_logging: logfile is
non-null
        NOTE: test-empty.c.exe iteration 2 of 5: writing reproducer to
./test-empty.c.exe.reproducer.c
 test-empty.c.exe iteration 2 of 5: writing reproducer to .
(reg:SI 3 bx)
not calling gen_raw_REG
<nil>
intermediate files written to /tmp/libgccjit-qF4SVX
        PASSED: test-empty.c.exe iteration 2 of 5: verify_code: result is
non-null

ITERATION 3
        PASSED: test-empty.c.exe iteration 3 of 5: set_up_logging: logfile is
non-null
        NOTE: test-empty.c.exe iteration 3 of 5: writing reproducer to
./test-empty.c.exe.reproducer.c
 test-empty.c.exe iteration 3 of 5: writing reproducer to .
<nil>
calling gen_raw_REG
(reg:SI 3 bx)
intermediate files written to /tmp/libgccjit-mw8kAT
        PASSED: test-empty.c.exe iteration 3 of 5: verify_code: result is
non-null

ITERATION 4
        PASSED: test-empty.c.exe iteration 4 of 5: set_up_logging: logfile is
non-null
        NOTE: test-empty.c.exe iteration 4 of 5: writing reproducer to
./test-empty.c.exe.reproducer.c
 test-empty.c.exe iteration 4 of 5: writing reproducer to .
(reg:SI 3 bx)
not calling gen_raw_REG
<nil>
intermediate files written to /tmp/libgccjit-h63OvP
        PASSED: test-empty.c.exe iteration 4 of 5: verify_code: result is
non-null

ITERATION 5
        PASSED: test-empty.c.exe iteration 5 of 5: set_up_logging: logfile is
non-null
        NOTE: test-empty.c.exe iteration 5 of 5: writing reproducer to
./test-empty.c.exe.reproducer.c
 test-empty.c.exe iteration 5 of 5: writing reproducer to .
<nil>
calling gen_raw_REG
(reg:SI 3 bx)
intermediate files written to /tmp/libgccjit-hzBrIL
        PASSED: test-empty.c.exe iteration 5 of 5: verify_code: result is
non-null

i.e. on odd-numbered iterations it transitions from NULL to reg bx
on even-numbered iterations it transitions from non-NULL back to NULL.