Bug 110113 - gdc -fpreview=dip1021 crash in d/dmd/root/aav.d:127 dmd_aaGetRvalue from DsymbolTable::lookup(Identifier const*)
Summary: gdc -fpreview=dip1021 crash in d/dmd/root/aav.d:127 dmd_aaGetRvalue from Dsym...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: d (show other bugs)
Version: 13.1.0
: P3 normal
Target Milestone: ---
Assignee: Iain Buclaw
URL: https://github.com/dlang/dmd/pull/14837
Keywords: ice-on-valid-code
Depends on:
Blocks:
 
Reported: 2023-06-04 16:42 UTC by Witold Baryluk
Modified: 2023-06-26 00:53 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2023-06-07 00:00:00


Attachments
Minimized test case with dustmite (199 bytes, application/x-dsrc)
2023-06-04 16:42 UTC, Witold Baryluk
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Witold Baryluk 2023-06-04 16:42:39 UTC
Created attachment 55254 [details]
Minimized test case with dustmite

Debian Linux amd64, experimental gcc-13, gdc 13.1.0-3


This is not very deterministic. Run few times to trigger.

```
user@debian:~$ cat lup.d
class LUBench {
}
float lup(ulong , ulong , int , int = 1) {
double[] solution;
new LUBench;
return solution[0] ;
}
float lup_3200(ulong iters, ulong flops) {
return lup(iters, flops, 3200);
}
float raytrace() {
struct V {
float x, y, z;
auto normalize() {
}
import std;
auto cross() {
}
auto norm2() {
}
auto norm() {
}
auto opBinary(){
}
}
}
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
lup.d:11:7: error: function ‘lup.raytrace’ has no ‘return’ statement, but is expected to return a value of type ‘float’
   11 | float raytrace() {
      |       ^
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
lup.d:11:7: error: function ‘lup.raytrace’ has no ‘return’ statement, but is expected to return a value of type ‘float’
   11 | float raytrace() {
      |       ^
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
lup.d:11:7: error: function ‘lup.raytrace’ has no ‘return’ statement, but is expected to return a value of type ‘float’
   11 | float raytrace() {
      |       ^
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
lup.d:11:7: error: function ‘lup.raytrace’ has no ‘return’ statement, but is expected to return a value of type ‘float’
   11 | float raytrace() {
      |       ^
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
/usr/lib/gcc/x86_64-linux-gnu/13/include/d/std/math/algebraic.d:968:47: internal compiler error: Segmentation fault
  968 |             return cast(Unqual!T) (T(1) << bsr(val) + type);
      |                                               ^
0xd32f86 crash_signal
	../../src/gcc/toplev.cc:314
0x7f53b651cf8f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x17f7d10 _D3dmd4root3aav15dmd_aaGetRvalueFNaNbNiPSQBnQBmQBk2AAPvZQd
	../../src/gcc/d/dmd/root/aav.d:127
0x1706b25 DsymbolTable::lookup(Identifier const*)
	../../src/gcc/d/dmd/dsymbol.d:2408
0x1706b25 ScopeDsymbol::search(Loc const&, Identifier*, int)
	../../src/gcc/d/dmd/dsymbol.d:1470
0x17ef5b3 _D3dmd6opover15search_functionFCQBe7dsymbol12ScopeDsymbolCQCe10identifier10IdentifierZCQDhQCd7Dsymbol
	../../src/gcc/d/dmd/opover.d:1435
0x1701fe0 search_toString(StructDeclaration*)
	../../src/gcc/d/dmd/dstruct.d:51
0x180310a semanticTypeInfoMembers(StructDeclaration*)
	../../src/gcc/d/dmd/semantic3.d:1650
0x1803394 Semantic3Visitor::visit(AggregateDeclaration*)
	../../src/gcc/d/dmd/semantic3.d:1590
0x17fef19 semantic3(Dsymbol*, Scope*)
	../../src/gcc/d/dmd/semantic3.d:83
0x175dc89 ExpressionSemanticVisitor::visit(DeclarationExp*)
	../../src/gcc/d/dmd/expressionsem.d:5572
0x175dc89 ExpressionSemanticVisitor::visit(DeclarationExp*)
	../../src/gcc/d/dmd/expressionsem.d:5407
0x175eb82 expressionSemantic(Expression*, Scope*)
	../../src/gcc/d/dmd/expressionsem.d:12706
0x18096fa StatementSemanticVisitor::visit(ExpStatement*)
	../../src/gcc/d/dmd/statementsem.d:207
0x18228c1 statementSemantic(Statement*, Scope*)
	../../src/gcc/d/dmd/statementsem.d:149
0x18228c1 StatementSemanticVisitor::visit(CompoundStatement*)
	../../src/gcc/d/dmd/statementsem.d:270
0x1809112 statementSemantic(Statement*, Scope*)
	../../src/gcc/d/dmd/statementsem.d:149
0x18002a1 Semantic3Visitor::visit(FuncDeclaration*)
	../../src/gcc/d/dmd/semantic3.d:598
0x17feae4 semantic3(Dsymbol*, Scope*)
	../../src/gcc/d/dmd/semantic3.d:83
0x17feae4 Semantic3Visitor::visit(Module*)
	../../src/gcc/d/dmd/semantic3.d:205
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <file:///usr/share/doc/gcc-13/README.Bugs> for instructions.
user@debian:~$ 
```


Could not reduce further, as it is sensitive to identifiers, and due to non-deterministic nature testing requires many repetitions.
Comment 1 Witold Baryluk 2023-06-04 16:44:02 UTC
BTW. Adding return statement in `raytrace`, does not change anything:

```
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
user@debian:~$ gdc-13 -c -fpreview=dip1021 lup.d
/usr/lib/gcc/x86_64-linux-gnu/13/include/d/std/math/algebraic.d:968:47: internal compiler error: Segmentation fault
  968 |             return cast(Unqual!T) (T(1) << bsr(val) + type);
      |                                               ^
0xd32f86 crash_signal
	../../src/gcc/toplev.cc:314
0x7f7144273f8f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x17f7d10 _D3dmd4root3aav15dmd_aaGetRvalueFNaNbNiPSQBnQBmQBk2AAPvZQd
	../../src/gcc/d/dmd/root/aav.d:127
0x1706b25 DsymbolTable::lookup(Identifier const*)
	../../src/gcc/d/dmd/dsymbol.d:2408
0x1706b25 ScopeDsymbol::search(Loc const&, Identifier*, int)
	../../src/gcc/d/dmd/dsymbol.d:1470
...
...
```
Comment 2 Witold Baryluk 2023-06-04 16:45:18 UTC
Also FYI, I was not able to trigger this on DMD64 D Compiler v2.104.0
Comment 3 Iain Buclaw 2023-06-06 20:37:44 UTC
It would be handy if there was a test that didn't import all of Phobos.
Comment 4 Iain Buclaw 2023-06-06 20:49:15 UTC
Test case should deterministically crash when kernel.randomize_va_space=0.
Comment 5 Iain Buclaw 2023-06-06 22:01:56 UTC
Reducing the std module down to the following always produces a crash in dmd_aaGetRvalue when running debian/ubuntu pre-compiled gdc-13 under gdb.

---
struct Tid
{
    MessageBox mbox;
}

struct ThreadInfo
{
    bool[Tid] links;
}

class MessageBox
{
}
---

Reproducing with own built gdc is proving to be elusive.
Comment 6 Iain Buclaw 2023-06-07 16:05:02 UTC
Full reduction without any imports.

---
class LUBench { }
void lup(ulong , ulong , int , int = 1)
{
    new LUBench;
}
void lup_3200(ulong iters, ulong flops)
{
    lup(iters, flops, 3200);
}
float raytrace()
{
    struct V
    {
        float x, y, z;
        auto normalize() { }
        struct Tid { }
        Tid spawnLinked() { }
        string[] namesByTid;
        class MessageBox { }
        auto cross() { }
    }
}
Comment 7 Iain Buclaw 2023-06-07 16:08:15 UTC
Same, but without any compiler errors.

This is reproducible in upstream dmd too.

dmd -lowmem -preview=dip1021 pr110113.d -o-

---
class LUBench { }
void lup(ulong , ulong , int , int = 1)
{
    new LUBench;
}
void lup_3200(ulong iters, ulong flops)
{
    lup(iters, flops, 3200);
}
void raytrace()
{
    struct V
    {
        float x, y, z;
        auto normalize() { }
        struct Tid { }
        auto spawnLinked() { }
        string[] namesByTid;
        class MessageBox { }
        auto cross() { }
    }
}
Comment 8 Iain Buclaw 2023-06-07 17:14:37 UTC
Regression caused by upstream.

https://github.com/dlang/dmd/pull/14837
Comment 9 Iain Buclaw 2023-06-10 17:31:55 UTC
(In reply to ibuclaw from comment #8)
> Regression caused by upstream.
> 
> https://github.com/dlang/dmd/pull/14837
Tracked it down to a memory corruption bug in the D front-end.

There is a call to Mem.xrealloc(ptr) inside the escape analysis code, which allocates a new GC pointer, marking the old pointer as "free" to reuse by the next GC.malloc request.

However, said pointer is to a data structure with Array(T) fields, each to which contain a pointer that references itself (a smallarray optimization).

The corruption arises as eventually there are two references to the same address one that is reading/writing to it as an `VarDeclaration**`, the other as an `aaA**`.  Segfault occurs as a result of aaGetRvalue interpreting a `VarDeclaration*` as an `aaA*`.

Fix is in review, and will backport to both GCC-13 and GCC-12.
Comment 10 Witold Baryluk 2023-06-11 17:06:30 UTC
Thank you Iain. Amazing debugging skills.

BTW. `import std;` was because dustmite reduced original import to just that. Original import was `import std.math.algebraic : sqrt;`

But you already figured this out without even using Phobos.
Comment 11 GCC Commits 2023-06-26 00:50:52 UTC
The releases/gcc-13 branch has been updated by Iain Buclaw <ibuclaw@gcc.gnu.org>:

https://gcc.gnu.org/g:ae3a4cefd855512b10b833a56f275b701bacdb34

commit r13-7478-gae3a4cefd855512b10b833a56f275b701bacdb34
Author: Iain Buclaw <ibuclaw@gdcproject.org>
Date:   Mon Jun 26 02:29:46 2023 +0200

    d: Fix crash in d/dmd/root/aav.d:127 dmd_aaGetRvalue from DsymbolTable::lookup
    
    Backports patch from upstream dmd mainline for fixing PR110113.
    
    The data being Mem.xrealloc'd contains many Array(T) fields, some of
    which have self references in their data.ptr field thanks to the
    smallarray optimization used by Array.
    
    Naturally then, the memcpy from old GC data to new retains those self
    referenced addresses, and the GC marks the old data as "free". Some time
    later GC.malloc will return a pointer to said "free" data. So now we
    have two GC references to the same memory. One that is treating the data
    as an Array(VarDeclaration) in dmd.escape.escapeByStorage, and the other
    as an AA in the symtab of a dmd.dsymbol.ScopeDsymbol.
    
    Fix this memory corruption by not storing the data in a global variable
    for reuse.  If there are no more live references, the GC will free it.
    
            PR d/110113
    
    gcc/d/ChangeLog:
    
            * dmd/escape.d (checkMutableArguments): Always allocate new buffer for
            computing escapeBy.
    
    gcc/testsuite/ChangeLog:
    
            * gdc.test/compilable/test23978.d: New test.
    
    Reviewed-on: https://github.com/dlang/dmd/pull/15302
Comment 12 GCC Commits 2023-06-26 00:51:35 UTC
The releases/gcc-12 branch has been updated by Iain Buclaw <ibuclaw@gcc.gnu.org>:

https://gcc.gnu.org/g:016047f54713dc601c661ab57c78a26da3759608

commit r12-9729-g016047f54713dc601c661ab57c78a26da3759608
Author: Iain Buclaw <ibuclaw@gdcproject.org>
Date:   Mon Jun 26 02:29:46 2023 +0200

    d: Fix crash in d/dmd/root/aav.d:127 dmd_aaGetRvalue from DsymbolTable::lookup
    
    Backports patch from upstream dmd mainline for fixing PR110113.
    
    The data being Mem.xrealloc'd contains many Array(T) fields, some of
    which have self references in their data.ptr field thanks to the
    smallarray optimization used by Array.
    
    Naturally then, the memcpy from old GC data to new retains those self
    referenced addresses, and the GC marks the old data as "free". Some time
    later GC.malloc will return a pointer to said "free" data. So now we
    have two GC references to the same memory. One that is treating the data
    as an Array(VarDeclaration) in dmd.escape.escapeByStorage, and the other
    as an AA in the symtab of a dmd.dsymbol.ScopeDsymbol.
    
    Fix this memory corruption by not storing the data in a global variable
    for reuse.  If there are no more live references, the GC will free it.
    
            PR d/110113
    
    gcc/d/ChangeLog:
    
            * dmd/escape.d (checkMutableArguments): Always allocate new buffer for
            computing escapeBy.
    
    gcc/testsuite/ChangeLog:
    
            * gdc.test/compilable/test23978.d: New test.
    
    Reviewed-on: https://github.com/dlang/dmd/pull/15302
    (cherry picked from commit ae3a4cefd855512b10b833a56f275b701bacdb34)
Comment 13 Iain Buclaw 2023-06-26 00:53:20 UTC
Fix committed to releases/gcc-13, and backported to gcc-12.

It will hit mainline on the next merge with upstream dmd.