Bug 34828

Summary: ICE: GNU MP: Cannot reallocate memory for gfortran.dg/parameter_array_init_3.f90
Product: gcc Reporter: Tobias Burnus <burnus>
Component: fortranAssignee: Not yet assigned to anyone <unassigned>
Status: RESOLVED FIXED    
Severity: normal CC: danglin, gcc-bugs, hjl.tools
Priority: P3 Keywords: ice-on-valid-code
Version: 4.3.0   
Target Milestone: ---   
Host: ia64-suse-linux-gnu Target:
Build: Known to work:
Known to fail: Last reconfirmed: 2008-01-18 04:18:04

Description Tobias Burnus 2008-01-17 14:36:29 UTC
Executing on host: /tmp/cvs/gcc-20080117/Build/gcc/testsuite/gfortran/../../gfortran -B/tmp/cvs/gcc-20080117/Build/gcc/testsuite/gfortran/../../ /tmp/cvs/gcc-20080117/gcc/testsuite/gfortran.dg/parameter_array_init_3.f90   -O   -pedantic-errors -S  -o parameter_array_init_3.s    (timeout = 300)
GNU MP: Cannot reallocate memory (old_size=8 new_size=8)
f951: internal compiler error: Aborted

That message appeared before in PR 31427 when a not-simplified variable was passed in TRANSFER. The test file is new as of PR 34476.

On x86-64-linux I cannot find any valgrind problem.

The ia64 failure was found by Andreas Schwab,
http://gcc.gnu.org/ml/gcc-testresults/2008-01/msg00744.html
Comment 1 Jerry DeLisle 2008-01-18 04:18:04 UTC
I think this is the same bug Steve found on FreeBSD system,  I have access to that one so I will try to isolate this
Comment 2 kargl 2008-01-18 05:25:16 UTC
Looking at Andreas's log. I'd have to say that it is the same
problem.  I'm guessing that memory is getting stomped on, but
I can't find the bug.
Comment 3 Jerry DeLisle 2008-01-18 23:16:14 UTC
On x86-64:

==11867== 208 bytes in 26 blocks are definitely lost in loss record 1 of 5
==11867==    at 0x4A059F6: malloc (vg_replace_malloc.c:149)
==11867==    by 0xB4C018: __gmp_default_allocate (in /mnt/sdb2/obj43/gcc/f951)
==11867==    by 0xB3BB6D: __gmpz_init_set (in /mnt/sdb2/obj43/gcc/f951)
==11867==    by 0x4242A8: gfc_copy_shape (expr.c:339)
==11867==    by 0x424309: gfc_copy_expr (expr.c:515)
==11867==    by 0x426206: simplify_parameter_variable (expr.c:1524)
==11867==    by 0x425F80: gfc_simplify_expr (expr.c:1638)
==11867==    by 0x40B234: expand_constructor (array.c:1379)
==11867==    by 0x40B4FC: gfc_get_array_element (array.c:1696)
==11867==    by 0x40C78E: gfc_expand_constructor (array.c:1404)
==11867==    by 0x45F7A4: gfc_resolve_expr (resolve.c:4320)
==11867==    by 0x40C646: gfc_resolve_array_constructor (array.c:1514)
Comment 4 kargl 2008-01-20 02:20:59 UTC
Just a quick note.  The bug is not present on i386-*-freebsd, but is
present on x86_64-*-*freebsd.  This is most likely a 32-bit versus 
64-bit pointer issue.
Comment 5 John David Anglin 2008-01-25 23:13:24 UTC
I also see this on hppa64-hp-hpux11.11.
Comment 6 Jerry DeLisle 2008-01-28 08:02:51 UTC
Found the problem.  We are trying to treat an EXPR_FUNCTION as if its an EXPR_CONSTANT.  I doubt the compilation is correct for any platform.  Segfault occurs here at 1053.  I tried using gfc_simplify_expr with no luck.

1053          mpz_add (tmp, tmp, ar->as->upper[i]->value.integer);
(gdb) p *ar->as->upper[i]
$1 = {expr_type = EXPR_FUNCTION, ts = {type = BT_UNKNOWN, kind = 0, 
    derived = 0x0, cl = 0x0, is_c_interop = 0, is_iso_c = 0, 
    f90_type = BT_UNKNOWN}, rank = 0, shape = 0x0, symtree = 0x10aa91d0, 
  ref = 0x0, where = {nextc = 0x10ad61af "len(HEX1)) =  [(1,i=1,len(HEX1))]", 
    lb = 0x10ad6170}, inline_noncopying_intrinsic = 0, is_boz = 0, 
  con_by_offset = 0x0, representation = {length = 0, string = 0x0}, value = {
    logical = 0, integer = {{_mp_alloc = 0, _mp_size = 279614288, 
        _mp_d = 0x0}}, real = {{_mpfr_prec = 279614288, _mpfr_sign = 0, 
        _mpfr_exp = 0, _mpfr_d = 0x0}}, complex = {r = {{
          _mpfr_prec = 279614288, _mpfr_sign = 0, _mpfr_exp = 0, 
          _mpfr_d = 0x0}}, i = {{_mpfr_prec = 0, _mpfr_sign = 0, 
          _mpfr_exp = 0, _mpfr_d = 0x0}}}, op = {
      operator = GFC_INTRINSIC_BEGIN, uop = 0x0, op1 = 0x0, op2 = 0x0}, 
    function = {actual = 0x10aa9350, name = 0x0, isym = 0x0, esym = 0x0}, 
    character = {length = 0, string = 0x0}, constructor = 0x10aa9350}}
(gdb) 

I am testing this patch:

@@ -1041,6 +1042,12 @@ find_array_element (gfc_constructor *con
          goto depart;
        }
 
+      /* Make sure we are dealing with constants.  */
+      if (ar->as->upper[i]->expr_type != EXPR_CONSTANT
+         ||
+         ar->as->upper[i]->expr_type != EXPR_CONSTANT)
+       goto depart;
+
       mpz_sub (delta, e->value.integer, ar->as->lower[i]->value.integer);
       mpz_mul (delta, delta, span);
       mpz_add (offset, offset, delta);
Comment 7 Jerry DeLisle 2008-01-28 08:09:18 UTC
Correction: upper and lower

@@ -1041,6 +1042,12 @@ find_array_element (gfc_constructor *con
          goto depart;
        }
 
+      /* Make sure we are dealing with constants.  */
+      if (ar->as->upper[i]->expr_type != EXPR_CONSTANT
+         ||
+         ar->as->lower[i]->expr_type != EXPR_CONSTANT)
+       goto depart;
+
       mpz_sub (delta, e->value.integer, ar->as->lower[i]->value.integer);
       mpz_mul (delta, delta, span);
       mpz_add (offset, offset, delta);
Comment 8 Jerry DeLisle 2008-01-28 08:44:01 UTC
With this test program derived from the original test case and the patch, gfortran compiles but does not get the results right:

module abuse_mod
   implicit none
   integer i
   character(8), parameter :: HEX1 = '40490FDB'
   integer(1), parameter :: MSKa1(len(HEX1)) =  (/(i,i=1,len(HEX1))/)
   integer(1), parameter :: ARR1(len(HEX1)) = (/( MSKa1(i), i=1,len(HEX1) )/)
end module abuse_mod

program test
  use abuse_mod
  print *, MSKa1
  print *, ARR1
end program test

]$ gfc parameter_array_init_3.f90 
$ ./a.out
    1    2    3    4    5    6    7    8
    1    1    1    1    1    1    1    1
$ ifort parameter_array_init_3.f90 
$ ./a.out
    1    2    3    4    5    6    7    8
    1    2    3    4    5    6    7    8

Without the patch gfortran gives correct results on x86-64 and segfaults on compilation on ppc64.
Comment 9 Dominique d'Humieres 2008-01-28 09:09:28 UTC
> Without the patch gfortran gives correct results on x86-64 and segfaults on
> compilation on ppc64.

Without the patch, gfortran compiles  gfortran.dg/parameter_array_init_3.f90 and gives the correct results for the code in comment #8 on ppc/intel Darwin9 in both 32 and 64 bit modes.

Comment 10 Jerry DeLisle 2008-01-29 03:33:16 UTC
Here is a modified test case that illustrates the problem better.  It was just dumb luck that the test case was passing on some platforms. The following segfaults on x86-64 with current trunk.

module abuse_mod
   implicit none
   integer i
   character(8), parameter :: HEX1 = '40490FDB'
   integer(1), parameter :: MSKa1(len(HEX1)/2) =  (/(i,i=1,len(HEX1),2)/)
   integer(1), parameter :: ARR1(len(HEX1)/2) = (/( MSKa1(i), i=1,len(HEX1),2)/)
end module abuse_mod

program test
  use abuse_mod
  print '(8i2)', (/(i,i=1,len(HEX1),2)/)
  print '(8i2)', MSKa1
  print '(8i2)', ARR1
end program test

Just the right memory locations in ar->as->upper[i]->value.integer happen to be zero.  There is no ar->as->upper[i]->value.integer in this test case.  ar->as->upper[i] has expr_type == EXPR_FUNCTION.

Now to figure out the proper fix. ;)
Comment 11 Jerry DeLisle 2008-01-29 05:17:43 UTC
Code in Comment #10 is invalid.  I have a fix for that segfault though.  Continuing the hunt.  :)
Comment 12 Jerry DeLisle 2008-01-30 01:56:20 UTC
Here is first part of patch that takes care of the segfault.  This is also helpful for pr19925:

@@ -1051,18 +1054,19 @@ find_array_element (gfc_constructor *con
       mpz_mul (span, span, tmp);
     }
 
-  if (cons)
-    {
-      for (nelemen = mpz_get_ui (offset); nelemen > 0; nelemen--)
-       {
-         if (cons->iterator)
-           {
-             cons = NULL;
-             goto depart;
-           }
-         cons = cons->next;
-       }
-    }
+    for (nelemen = mpz_get_ui (offset); nelemen > 0; nelemen--)
+      {
+        if (cons)
+         {
+           if (cons->iterator)
+             {
+               cons = NULL;
+               goto depart;
+             }
+           cons = cons->next;
+         }
+      }
 
 depart:
   mpz_clear (delta);
Comment 13 Dominique d'Humieres 2008-01-30 14:16:14 UTC
I have applied the patch on top of rev. 131960 on i686-apple-darwin9. The only change I have noticed is that the ICE for he code in comment #10 has been replaced by the following (cryptic) error:

pr34828_1.f90:6.80:

teger(1), parameter :: ARR1(len(HEX1)/2) = (/( MSKa1(i), i=1,len(HEX1),2)/)
                                                                          1     
Error: Initialization expression didn't reduce (1)
...

(I can now see why the code is invalid!-).

For pr19925 codes, I still have the same ICEs as before the patch:

[karma] f90/bug% cat pr19925_1.f90
INTEGER, PARAMETER :: N=100000
INTEGER, PARAMETER :: I(N)=(/(MOD(K,2),K=1,N)/)
INTEGER, PARAMETER :: M(N)=I(N:1:-1)
END
[karma] f90/bug% gfc pr19925_1.f90
f951: internal compiler error: Bus error
...
[karma] f90/bug% cat pr19925_3.f90
   program stuff
     integer :: i_do
     integer :: i(100001) = (/ (i_do, i_do=1,100001) /)
     write (*,*) i(100001)
   end program stuff
karma] f90/bug% gfc pr19925_3.f90
pr19925_3.f90: In function 'stuff':
pr19925_3.f90:1: internal compiler error: Possible frontend bug: array constructor not expanded
...

Comment 14 Jerry DeLisle 2008-01-30 18:04:18 UTC
The pr19925 segfault is close by the segfault here but not the same.  I am working on a patch for that one.  What is becoming clear is that we are not handling expansion of constructors very well yet.

Also, I will add that the original problem with this PR shows that on ppc64, the expression types for the constructors are showing as EXPR_FUNCTION whereas on x86-64 where the test case passes, those same expressions are EXPR_CONSTANT or EXPR_VARIABLE.  I fear that a pointer related problem is resulting in cobbering some things and it could be just about anywhere that happens.

I am thinking we have to go back and scrutinize the matchers and work down through until we see where the corruption is happening.
Comment 15 Jerry DeLisle 2008-03-25 02:53:10 UTC
Un-assigning myself. I can see valgrind errors but have been unable to isolate the problem.
Comment 16 kargl 2008-04-30 22:37:11 UTC
(In reply to comment #15)
> Un-assigning myself. I can see valgrind errors but have been unable to isolate
> the problem.
> 

I may have a further piece to the puzzle.  It appears that len(HEX1) is not being properly evaluated to give MSKa1(len(HEX1)) the correct array descriptor.  A modified program shows

program z
  call a
  call b
  call d
end program z

subroutine a
   implicit none
   integer i
   character(8), parameter :: HEX1 = '40490FDB'
   integer, parameter :: MSKa1(len(HEX1)) =  [(1,i=1,len(HEX1))]
!
! If uncomment either line, then segfault.
!
!   integer, parameter :: ARR1(len(HEX1)) = [( MSKa1(i), i=1,len(HEX1) )]
!   integer, parameter :: ARR1(8) = [( MSKa1(i), i=1,len(HEX1) )]

   print '(A,1x,8(i0,1x))', hex1, mska1
end subroutine a

subroutine b
   implicit none
   integer i
   character(8), parameter :: HEX1 = '40490FDB'
!
! If 8 in MSKa1(8) is len(HEX1), then segfault.
!
   integer, parameter :: MSKa1(8) =  [(1,i=1,len(HEX1))]
   integer, parameter :: ARR1(len(HEX1)) = [( MSKa1(i), i=1,len(HEX1) )]
   print '(A,1x,16(i0,1x))', hex1, mska1, arr1
end subroutine b
!
! This works as expected!
!
subroutine d
   implicit none
   integer i
   character(8), parameter :: HEX1 = '40490FDB'
   integer, parameter :: MSKa1(8) =  [(1,i=1,len(HEX1))]
   integer, parameter :: ARR1(8) = [( MSKa1(i), i=1,len(HEX1) )]
   print '(A,1x,16(i0,1x))', hex1, mska1, arr1
end subroutine d

Comment 17 Jerry DeLisle 2008-05-26 17:05:04 UTC
With the test case in comment#16 on x86-64-linux-gnu, I see this:

==11143== 416 bytes in 52 blocks are definitely lost in loss record 5 of 14
==11143==    at 0x4A059F6: malloc (vg_replace_malloc.c:149)
==11143==    by 0x3FDCC07EF8: __gmp_default_allocate (in /usr/lib64/libgmp.so.3.4.2)
==11143==    by 0x3FDCC18050: __gmpz_init_set (in /usr/lib64/libgmp.so.3.4.2)
==11143==    by 0x4232A7: gfc_copy_shape (expr.c:338)
==11143==    by 0x42331D: gfc_copy_expr (expr.c:515)
==11143==    by 0x425266: simplify_parameter_variable (expr.c:1526)
==11143==    by 0x425003: gfc_simplify_expr (expr.c:1641)
==11143==    by 0x409F2F: expand_constructor (array.c:1437)
==11143==    by 0x40A063: gfc_get_array_element (array.c:1791)
==11143==    by 0x40B2FE: gfc_expand_constructor (array.c:1462)
==11143==    by 0x45DBC9: gfc_resolve_expr (resolve.c:4337)
==11143==    by 0x40B1CA: gfc_resolve_array_constructor (array.c:1572)
==11143== 
==11143== 
==11143== 632 bytes in 4 blocks are definitely lost in loss record 6 of 14
==11143==    at 0x4A04D1F: calloc (vg_replace_malloc.c:279)
==11143==    by 0xB4304A: xcalloc (xmalloc.c:162)
==11143==    by 0x55B877: init_emit (emit-rtl.c:5012)
==11143==    by 0x5F3E62: prepare_function_start (function.c:3906)
==11143==    by 0x5F5E98: init_function_start (function.c:3953)
==11143==    by 0x492BDD: trans_function_start (trans-decl.c:1618)
==11143==    by 0x499D6D: gfc_generate_function_code (trans-decl.c:3218)
==11143==    by 0x45570B: gfc_parse_file (parse.c:3585)
==11143==    by 0x47D81D: gfc_be_parse_file (f95-lang.c:258)
==11143==    by 0x6E49F1: toplev_main (toplev.c:965)
==11143==    by 0x3FF061E073: (below main) (libc-start.c:220)
==11143== 
==11143== 
==11143== 1,224 bytes in 9 blocks are definitely lost in loss record 8 of 14
==11143==    at 0x4A059F6: malloc (vg_replace_malloc.c:149)
==11143==    by 0xB4300B: xrealloc (xmalloc.c:177)
==11143==    by 0x8CAC80: vec_heap_o_reserve_1 (vec.c:176)
==11143==    by 0x50609D: insn_locators_alloc (vecprim.h:27)
==11143==    by 0xA9C778: tree_expand_cfg (cfgexpand.c:1850)
==11143==    by 0x6686D6: execute_one_pass (passes.c:1284)
==11143==    by 0x6688D0: execute_pass_list (passes.c:1334)
==11143==    by 0x74A705: tree_rest_of_compilation (tree-optimize.c:421)
==11143==    by 0x8FA841: cgraph_expand_function (cgraphunit.c:1148)
==11143==    by 0x8FC5E3: cgraph_assemble_pending_functions (cgraphunit.c:514)
==11143==    by 0x8FBB34: cgraph_finalize_function (cgraphunit.c:632)
==11143==    by 0x49AFF5: gfc_generate_function_code (trans-decl.c:3481)
==11143== 
==11143== 
==11143== 62,028 (26,528 direct, 35,500 indirect) bytes in 348 blocks are definitely lost in loss record 12 of 14
==11143==    at 0x4A059F6: malloc (vg_replace_malloc.c:149)
==11143==    by 0xB43097: xmalloc (xmalloc.c:147)
==11143==    by 0x52A7F2: df_install_refs (df-scan.c:2454)
==11143==    by 0x52AA8D: df_refs_add_to_chains (df-scan.c:2580)
==11143==    by 0x52E0CF: df_record_exit_block_uses (df-scan.c:3950)
==11143==    by 0x52F5E0: df_scan_blocks (df-scan.c:604)
==11143==    by 0x51EF4D: rest_of_handle_df_initialize (df-core.c:743)
==11143==    by 0x6686D6: execute_one_pass (passes.c:1284)
==11143==    by 0x6688D0: execute_pass_list (passes.c:1334)
==11143==    by 0x6688E4: execute_pass_list (passes.c:1335)
==11143==    by 0x74A705: tree_rest_of_compilation (tree-optimize.c:421)
==11143==    by 0x8FA841: cgraph_expand_function (cgraphunit.c:1148)
==11143== 
==11143== LEAK SUMMARY:
==11143==    definitely lost: 28,800 bytes in 413 blocks.
==11143==    indirectly lost: 35,500 bytes in 347 blocks.
==11143==      possibly lost: 64 bytes in 2 blocks.
==11143==    still reachable: 298,303 bytes in 1,048 blocks.
==11143==         suppressed: 0 bytes in 0 blocks.

Notice the similarity in the second trace hunk to that found in pr36214.  Also, the first trace hunk shown here, I have not seen before and involves simplify_parameter_variable.  This may be the clue we need.
Comment 18 Francois-Xavier Coudert 2008-05-28 17:26:27 UTC
For the memory leak that happens from simplify_parameter_variable(), a reduced testcase is:

  integer, parameter :: MSKa1(1) = 1
  integer, parameter :: ARR1 = MSKa1(1)
  end

The reason for it is that when we change the expression rank in simplify_parameter_variable(), we don't change the shape accordingly. I think it should be done like that:

Index: expr.c
===================================================================
--- expr.c      (revision 135515)
+++ expr.c      (working copy)
@@ -1522,12 +1522,20 @@ simplify_parameter_variable (gfc_expr *p
 {
   gfc_expr *e;
   try t;
+  int n;

   e = gfc_copy_expr (p->symtree->n.sym->value);
   if (e == NULL)
     return FAILURE;

+  /* Copy the rank and shape from p, but first clear the existing shape.  */
+  if (e->shape)
+    for (n = 0; n < e->rank; n++)
+      mpz_clear (e->shape[n]);
+
+  gfc_free (e->shape);
   e->rank = p->rank;
+  e->shape = gfc_copy_shape (p->shape, p->rank);

   /* Do not copy subobject refs for constant.  */
   if (e->expr_type != EXPR_CONSTANT && p->ref != NULL)


That patch seems to regtest fine, but I don't see how it could fix the original ICE.
Comment 19 Joost VandeVondele 2008-08-08 20:52:47 UTC
Is this still an ice-on-valid-code ? I can't reproduce that here.
Comment 20 dave 2008-08-08 21:06:57 UTC
Subject: Re:  ICE: GNU MP: Cannot reallocate memory for gfortran.dg/parameter_array_init_3.f90

> ------- Comment #19 from jv244 at cam dot ac dot uk  2008-08-08 20:52 -------
> Is this still an ice-on-valid-code ? I can't reproduce that here.

Seems fixed <http://gcc.gnu.org/ml/gcc-testresults/2008-08/msg00500.html>.

Dave
Comment 21 kargl 2008-08-09 05:15:47 UTC
(In reply to comment #19)
> Is this still an ice-on-valid-code ? I can't reproduce that here.
> 

As with Dave, I can confirm that this problem no longer occurs
on x86_64-*-freebsd.  The PR can probably be closed.
Comment 22 Joost VandeVondele 2008-08-11 07:34:44 UTC
as per comment #20 and comment #21