Bug 110113

BTW. Adding return statement in `raytrace`, does not change anything:

```
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
/usr/lib/gcc/x86_64-linux-gnu/13/include/d/std/math/algebraic.d:968:47: internal compiler error: Segmentation fault
  968 |             return cast(Unqual!T) (T(1) << bsr(val) + type);
      |                                               ^
0xd32f86 crash_signal
	../../src/gcc/toplev.cc:314
0x7f7144273f8f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x17f7d10 _D3dmd4root3aav15dmd_aaGetRvalueFNaNbNiPSQBnQBmQBk2AAPvZQd
	../../src/gcc/d/dmd/root/aav.d:127
0x1706b25 DsymbolTable::lookup(Identifier const*)
	../../src/gcc/d/dmd/dsymbol.d:2408
0x1706b25 ScopeDsymbol::search(Loc const&, Identifier*, int)
	../../src/gcc/d/dmd/dsymbol.d:1470
...
...
```

Also FYI, I was not able to trigger this on DMD64 D Compiler v2.104.0

It would be handy if there was a test that didn't import all of Phobos.

Test case should deterministically crash when kernel.randomize_va_space=0.

Reducing the std module down to the following always produces a crash in dmd_aaGetRvalue when running debian/ubuntu pre-compiled gdc-13 under gdb.

---
struct Tid
{
    MessageBox mbox;
}

struct ThreadInfo
{
    bool[Tid] links;
}

class MessageBox
{
}
---

Reproducing with own built gdc is proving to be elusive.

Full reduction without any imports.

---
class LUBench { }
void lup(ulong , ulong , int , int = 1)
{
    new LUBench;
}
void lup_3200(ulong iters, ulong flops)
{
    lup(iters, flops, 3200);
}
float raytrace()
{
    struct V
    {
        float x, y, z;
        auto normalize() { }
        struct Tid { }
        Tid spawnLinked() { }
        string[] namesByTid;
        class MessageBox { }
        auto cross() { }
    }
}

Same, but without any compiler errors.

This is reproducible in upstream dmd too.

dmd -lowmem -preview=dip1021 pr110113.d -o-

---
class LUBench { }
void lup(ulong , ulong , int , int = 1)
{
    new LUBench;
}
void lup_3200(ulong iters, ulong flops)
{
    lup(iters, flops, 3200);
}
void raytrace()
{
    struct V
    {
        float x, y, z;
        auto normalize() { }
        struct Tid { }
        auto spawnLinked() { }
        string[] namesByTid;
        class MessageBox { }
        auto cross() { }
    }
}

Regression caused by upstream.

https://github.com/dlang/dmd/pull/14837

(In reply to ibuclaw from comment #8)
> Regression caused by upstream.
> 
> https://github.com/dlang/dmd/pull/14837
Tracked it down to a memory corruption bug in the D front-end.

There is a call to Mem.xrealloc(ptr) inside the escape analysis code, which allocates a new GC pointer, marking the old pointer as "free" to reuse by the next GC.malloc request.

However, said pointer is to a data structure with Array(T) fields, each to which contain a pointer that references itself (a smallarray optimization).

The corruption arises as eventually there are two references to the same address one that is reading/writing to it as an `VarDeclaration**`, the other as an `aaA**`.  Segfault occurs as a result of aaGetRvalue interpreting a `VarDeclaration*` as an `aaA*`.

Fix is in review, and will backport to both GCC-13 and GCC-12.

Thank you Iain. Amazing debugging skills.

BTW. `import std;` was because dustmite reduced original import to just that. Original import was `import std.math.algebraic : sqrt;`

But you already figured this out without even using Phobos.

The releases/gcc-13 branch has been updated by Iain Buclaw <ibuclaw@gcc.gnu.org>:

https://gcc.gnu.org/g:ae3a4cefd855512b10b833a56f275b701bacdb34

commit r13-7478-gae3a4cefd855512b10b833a56f275b701bacdb34
Author: Iain Buclaw <ibuclaw@gdcproject.org>
Date:   Mon Jun 26 02:29:46 2023 +0200

    d: Fix crash in d/dmd/root/aav.d:127 dmd_aaGetRvalue from DsymbolTable::lookup
    
    Backports patch from upstream dmd mainline for fixing PR110113.
    
    The data being Mem.xrealloc'd contains many Array(T) fields, some of
    which have self references in their data.ptr field thanks to the
    smallarray optimization used by Array.
    
    Naturally then, the memcpy from old GC data to new retains those self
    referenced addresses, and the GC marks the old data as "free". Some time
    later GC.malloc will return a pointer to said "free" data. So now we
    have two GC references to the same memory. One that is treating the data
    as an Array(VarDeclaration) in dmd.escape.escapeByStorage, and the other
    as an AA in the symtab of a dmd.dsymbol.ScopeDsymbol.
    
    Fix this memory corruption by not storing the data in a global variable
    for reuse.  If there are no more live references, the GC will free it.
    
            PR d/110113
    
    gcc/d/ChangeLog:
    
            * dmd/escape.d (checkMutableArguments): Always allocate new buffer for
            computing escapeBy.
    
    gcc/testsuite/ChangeLog:
    
            * gdc.test/compilable/test23978.d: New test.
    
    Reviewed-on: https://github.com/dlang/dmd/pull/15302

The releases/gcc-12 branch has been updated by Iain Buclaw <ibuclaw@gcc.gnu.org>:

https://gcc.gnu.org/g:016047f54713dc601c661ab57c78a26da3759608

commit r12-9729-g016047f54713dc601c661ab57c78a26da3759608
Author: Iain Buclaw <ibuclaw@gdcproject.org>
Date:   Mon Jun 26 02:29:46 2023 +0200

    d: Fix crash in d/dmd/root/aav.d:127 dmd_aaGetRvalue from DsymbolTable::lookup
    
    Backports patch from upstream dmd mainline for fixing PR110113.
    
    The data being Mem.xrealloc'd contains many Array(T) fields, some of
    which have self references in their data.ptr field thanks to the
    smallarray optimization used by Array.
    
    Naturally then, the memcpy from old GC data to new retains those self
    referenced addresses, and the GC marks the old data as "free". Some time
    later GC.malloc will return a pointer to said "free" data. So now we
    have two GC references to the same memory. One that is treating the data
    as an Array(VarDeclaration) in dmd.escape.escapeByStorage, and the other
    as an AA in the symtab of a dmd.dsymbol.ScopeDsymbol.
    
    Fix this memory corruption by not storing the data in a global variable
    for reuse.  If there are no more live references, the GC will free it.
    
            PR d/110113
    
    gcc/d/ChangeLog:
    
            * dmd/escape.d (checkMutableArguments): Always allocate new buffer for
            computing escapeBy.
    
    gcc/testsuite/ChangeLog:
    
            * gdc.test/compilable/test23978.d: New test.
    
    Reviewed-on: https://github.com/dlang/dmd/pull/15302
    (cherry picked from commit ae3a4cefd855512b10b833a56f275b701bacdb34)

Fix committed to releases/gcc-13, and backported to gcc-12.

It will hit mainline on the next merge with upstream dmd.