I have a code that has multiple large data files that are written in big endian format. To deal with this on many platforms, a script was written that, among various other things, sets a few units to read as big endian by using GFORTRAN_CONVERT_UNIT='native;big_endian:60-70,80-89'. However, when the Fortran program is called from the script I get: scale: malloc.c:2368: sysmalloc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_to p)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. This reliably happens on any Linux platform I've tested it on *except* Fedora/RHEL. I've been able to reliably reproduce the problem on several 32-bit machines (Arch Linux (GCC 4.7.1), OpenSUSE 12.2, Ubuntu 12.10 Beta 1) by creating and running these two programs: Write program: [shane@shane-laptop ~/temp/testgfortran]$ cat test_write.f90 program test_write implicit none integer, parameter :: NUM = 10 integer :: i open(unit=88,form='unformatted',convert='big_endian') do i = 1,NUM write (88) i end do close(88) end program test_write Read Program: [shane@shane-laptop ~/temp/testgfortran]$ cat test_read.f90 program test_write implicit none integer, parameter :: NUM = 10 integer :: readInt integer :: i open(unit=88,form='unformatted') do i = 1,NUM read (88) readInt write (*,*) readInt end do close(88) end program test_write And testing: [shane@shane-laptop ~/temp/testgfortran]$ ./test_write [shane@shane-laptop ~/temp/testgfortran]$ ./test_read 16777216 At line 10 of file test_read.f90 (unit = 88, file = 'fort.88') Fortran runtime error: End of file [shane@shane-laptop ~/temp/testgfortran]$ GFORTRAN_CONVERT_UNIT='native;big_endian:88' ./test_read test_read: malloc.c:2368: sysmalloc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize)) +((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. Aborted [shane@shane-laptop ~/temp/testgfortran]$ GFORTRAN_CONVERT_UNIT='big_endian' ./test_read 1 2 3 4 5 6 7 8 9 10 Obviously the first invocation of test_read fails because the endianess is wrong, but the second one should work. Running gdb yields: Program received signal SIGABRT, Aborted. 0xb7fdd424 in __kernel_vsyscall () However, the above test program does work on all of the 64-bit machines I've tested. I found that by commenting out the lines 582-583 in libgfortran/runtime/ environ.c I can get it to work. I don't really know why it's accessing an element in the elist structure that hasn't been allocated yet. This loop doesn't seem to do anything to me. Since I don't know the reprocussions of just commenting out stuff willy nilly I'll leave the implementation details to someone with more knowledge, but this seems like a good place to start.
Mine.
Patch posted.
The patch does get rid of memory corruption. However, there seem to be some problems with search_unit returning true if an exception is found when there is only one exception. If n_elist = 1, then high = low = 0, and the funtion will always return 0, even if the unit passed in to search for is in the exception list.
Am 30.09.2012 21:12, schrieb shart6 at utk dot edu: > If n_elist = 1, then high = low = 0, and the funtion will always return 0, even > if the unit passed in to search for is in the exception list. If there are n_elist exceptions, then they can be found in elist[0], ..., elist[n_elist-1]. If there is one exception, then it can be found at elist[0].
(In reply to comment #4) > Am 30.09.2012 21:12, schrieb shart6 at utk dot edu: > > If n_elist = 1, then high = low = 0, and the funtion will always return 0, even > > if the unit passed in to search for is in the exception list. > > If there are n_elist exceptions, then they can be found in > elist[0], ..., elist[n_elist-1]. > > If there is one exception, then it can be found at elist[0]. That is true, but the function will not return 1 (true, a match was found in elist) when there is only one exception stored. It will skip over the while loop, set *ip to 0, and return 0 (false, the exception was not found!). This can be illustrated: 1) Call the program, setting only one exception (in this case unit 21): [shane@shane-laptop ~/temp/testgfortran]$ GFORTRAN_CONVERT_UNIT='native;big_endian:21' gnu_wrap gdb ./test_read 2) Set a break point at where the library enquires as to the endianness of the file to be opened: Breakpoint 1, _gfortrani_get_unformatted_convert (unit=21) at ../../../libgfortran/runtime/environ.c:848 3) We will step into search_unit (with unit = 21). Everything is great. however, since n_elist = 1, high = low = 0, and the while loop is skipped over: (gdb) step 471 while (high - low > 1) (gdb) step 485 if (unit > elist[high].unit) 4) And 0 is returned (unit exception not found!) (gdb) step 490 return 0; 5) The library is told to open unit 21 (which is in the exception list at 0) as default endianness: _gfortrani_get_unformatted_convert (unit=21) at ../../../libgfortran/runtime/environ.c:856 856 return def;
Latest patch fixed it. Thanks a lot for the time you put into this!
Author: tkoenig Date: Sat Oct 6 13:04:35 2012 New Revision: 192158 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=192158 Log: 2012-10-06 Thomas König <tkoenig@gcc.gnu.org> PR libfortran/54736 * runtime/environ.c (search_unit): Correct logic for binary search. (mark_single): Fix index errors. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/runtime/environ.c
Author: tkoenig Date: Fri Oct 12 18:56:16 2012 New Revision: 192408 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=192408 Log: 2012-10-06 Thomas König <tkoenig@gcc.gnu.org> PR libfortran/54736 Backport from trunk * runtime/environ.c (search_unit): Correct logic for binary search. (mark_single): Fix index errors. Modified: branches/gcc-4_7-branch/libgfortran/ChangeLog branches/gcc-4_7-branch/libgfortran/runtime/environ.c
Author: tkoenig Date: Fri Oct 12 19:38:04 2012 New Revision: 192411 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=192411 Log: 2012-10-12 Thomas König <tkoenig@gcc.gnu.org> PR libfortran/54736 libgfortran/Changelog: Fix date of last commit. Modified: branches/gcc-4_7-branch/libgfortran/ChangeLog
Author: tkoenig Date: Sun Oct 21 13:43:32 2012 New Revision: 192653 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=192653 Log: 2012-10-21 Thomas König <tkoenig@gcc.gnu.org> PR libfortran/54736 Backport from trunk * runtime/environ.c (search_unit): Correct logic for binary search. (mark_single): Fix index errors. Modified: branches/gcc-4_6-branch/libgfortran/ChangeLog branches/gcc-4_6-branch/libgfortran/runtime/environ.c
Also fixed on 4.6, closing.