User account creation filtered due to spam.

Bug 20366 - AIX g++ -D_LARGE_FILES fails to compile #include <iostream>
Summary: AIX g++ -D_LARGE_FILES fails to compile #include <iostream>
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: target (show other bugs)
Version: 3.4.3
: P3 normal
Target Milestone: ---
Assignee: Peter O'Gorman
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-03-07 19:12 UTC by dsanderson@panasas.com
Modified: 2010-12-27 19:38 UTC (History)
5 users (show)

See Also:
Host: powerpc-ibm-aix5.2.0.0
Target: powerpc-ibm-aix5.2.0.0
Build: powerpc-ibm-aix5.2.0.0
Known to work: 4.4.0
Known to fail:
Last reconfirmed: 2008-04-04 05:31:57


Attachments
the effect fixincludes could have on aix5.2/aix5.3 (590 bytes, patch)
2007-12-19 15:57 UTC, Michael Haubenwallner
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description dsanderson@panasas.com 2005-03-07 19:12:30 UTC
	The compiler's C++ headers do not appear to work properly on AIX when
	the compiler is run with -D_LARGE_FILES.  This macro causes the
	system headers to #define symbols like fopen, while the C++ headers
	unconditionally #undef these symbols.

Environment:
System: AIX bigblue 2 5 000F699D4C00


	
host: powerpc-ibm-aix5.2.0.0
build: powerpc-ibm-aix5.2.0.0
target: powerpc-ibm-aix5.2.0.0
configured with: /usr0/slammert/gcc-3.4.3/configure --with-as=/usr/bin/as --with-ld=/usr/bin/ld --disable-nls --enable-languages=c,c++ --enable-threads --enable-version-specific-runtime-libs

How-To-Repeat:
	On AIX 5.2, compile the following program with
	3.4.3 g++ -D_LARGE_FILES:

	    #include <iostream>
	    int main(void) { return 0; }

	You will get output similar to the following:

	    In file included from /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/powerpc-ibm-aix5.2.0.0/bits/c++locale.h:43,
			     from /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/iosfwd:46,
			     from /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/ios:44,
			     from /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/ostream:45,
			     from /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/iostream:45,
			     from bug0.C:1:
	    /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/cstdio:108: error: `::fgetpos' has not been declared
	    /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/cstdio:110: error: `::fopen' has not been declared
	    /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/cstdio:115: error: `::freopen' has not been declared
	    /usr/local/lib/gcc/powerpc-ibm-aix5.2.0.0/3.4.3/include/c++/cstdio:118: error: `::fsetpos' has not been declared
Comment 1 dsanderson@panasas.com 2005-03-07 19:12:30 UTC
Fix:
	I can work around this problem and still compile a program with a
	64-bit off_t by compiling with -maix64 instead of -D_LARGE_FILES.
Comment 2 David Edelsohn 2005-03-07 19:30:30 UTC
G++ and libstdc++ on AIX currently is configured to enable _LARGE_FILE_API, 
but not _LARGE_FILES.  _LARGE_FILE_API exposes the 64-bit functions with 
separate names off64_t and fopen64(), but does not redefine the original 
functions with their counterparts supporting 64-bit types.
Comment 3 The Written Word 2006-02-22 23:43:39 UTC
Any plans to add _LARGE_FILES support to g++/libstdc++ for AIX?
Comment 4 Michael Haubenwallner 2007-12-01 22:04:15 UTC
Same here with gcc-4.2.2 and powerpc-ibm-aix5.3.0.0.
 
Some diagnostics what happens when _LARGE_FILES (and _LARGE_FILE_API) is defined:

<sys/types.h>

/* would both be 'long' without _LARGE_FILES: */
typedef long long fpos_t;
typedef long long off_t;

/* because of _LARGE_FILE_API: */
typedef long long   off64_t;

<stdio.h>

/* because of _LARGE_FILES (the 'problematic' ones): */
#define fseeko fseeko64
#define ftello ftello64
#define fgetpos fgetpos64
#define fsetpos fsetpos64
#define fopen fopen64
#define freopen freopen64

/* these actually are declared as f*64() because of defines above: */
extern FILE *fopen(const char *, const char *);
extern FILE *freopen(const char *, const char *, FILE *);
extern int   fgetpos(FILE *, fpos_t *);
extern int   fsetpos(FILE *, const fpos_t *);
extern int   fseeko(FILE *, off_t, int);
extern off_t ftello(FILE *);

/* because of _LARGE_FILE_API: */
extern int fgetpos64(FILE *, fpos64_t *);
extern FILE *fopen64(const char *, const char *);
extern FILE *freopen64(const char *, const char *, FILE *);
extern int fseeko64(FILE *, off64_t, int);
extern int fsetpos64(FILE *, const fpos64_t *);
extern off64_t ftello64(FILE *);

<cstdio>

/* remember: these actually are declared as f*64(),
 * while the original ones are undeclared.
 */
#undef fgetpos
#undef fopen
#undef freopen
#undef fsetpos

namespace std {
  using ::fpos_t;
  using ::fgetpos;
  using ::fopen;
  using ::freopen;
  using ::fsetpos;
}


I'm unsure how to 'fix' that 'correctly' - one suggestion is to 'fixincludes':

Add this just _before_ those '#define fopen fopen64' lines in <stdio.h>

  static FILE *fopen(const char *a, const char *b) { return fopen64(a,b); }

Eventually there is a better way to redirect from 'fopen' to 'fopen64' on aix also for the compiler - not just the preprocessor.
Comment 5 Michael Haubenwallner 2007-12-19 15:57:47 UTC
Created attachment 14796 [details]
the effect fixincludes could have on aix5.2/aix5.3

When applying this patch to (already fixed) stdio.h on aix5.2/aix5.3, compiling "#include <iostream>" with -D_LARGE_FILES works. But I have no clue on how to create entries with this effect in inclhack.def, or if this is the way to go.
Comment 6 Tom Coleman 2007-12-20 00:06:58 UTC
The test program will compile if you comment out the #undefs in cstdio.

//#undef fgetpos
//#undef fopen
//#undef freopen
//#undef fsetpos

I've wrestled with this problem with G++ on AIX for years.  In desparation I just had our admin load the IBM 4.0.0 RPMs - same problem!

This is the second time this error has prevented me from porting a relatively simple application to AIX.

This particular problem has gone on long enough.
 
Comment 7 dsanderson@panasas.com 2007-12-20 00:17:39 UTC
Note that my originally stated workaround -- to compile with -maix64 -- doesn't really work beyond helping the compile to succeed.  Using -maix64 gives you a 64-bit executable (e.g. with 64-bit pointers and a 64-bit off_t) that can't access large files.
Comment 8 David Edelsohn 2007-12-21 00:29:20 UTC
The issue is whether to default G++ to _LARGE_FILES, as is done for Fortran, and perform all libstdc++ I/O as 64-bit operations, which will affect performance.  Not everyone may want or need large I/O.  There is no way to give users control over this without imposing it so G++, G++ headers, and libstdc++ all are self-consistent.  The other alternative is another multilib, but there alredy are too many variants.
Comment 9 The Written Word 2007-12-21 03:59:19 UTC
(In reply to comment #8)
> The issue is whether to default G++ to _LARGE_FILES, as is done for Fortran,
> and perform all libstdc++ I/O as 64-bit operations, which will affect
> performance.  Not everyone may want or need large I/O.  There is no way to give
> users control over this without imposing it so G++, G++ headers, and libstdc++
> all are self-consistent.  The other alternative is another multilib, but there
> alredy are too many variants.

Why is this? -D_LARGE_FILES simply enables LFS. xlC works with and without -D_LARGE_FILES. Shouldn't G++ work as effectively as xlC?
Comment 10 David Edelsohn 2007-12-21 14:58:49 UTC
Subject: Re:  AIX g++ -D_LARGE_FILES fails to compile #include <iostream> 

> Why is this? -D_LARGE_FILES simply enables LFS. xlC works with and without
> -D_LARGE_FILES. Shouldn't G++ work as effectively as xlC?

	You are mixing multiple questions and issues.

	-D_LARGE_FILES redefines all I/O functions to use the 64-bit
versions. 

	xlC has its own system header files.  G++ needs to use the AIX
system header files with changes from fixincludes, combined with libstdc++
header files that perform their own redefinitions.

	"Shouldn't G++ work as effectively..." is a business question, not
a technical question.  Many major AIX customers use GCC, but none have
told IBM that they will not buy a system unless GCC works better.  That
sets investment priorities.
Comment 11 The Written Word 2007-12-21 19:14:15 UTC
(In reply to comment #10)
> Subject: Re:  AIX g++ -D_LARGE_FILES fails to compile #include <iostream> 
> 
>         xlC has its own system header files.  G++ needs to use the AIX
> system header files with changes from fixincludes, combined with libstdc++
> header files that perform their own redefinitions.

Agreed. So, all that's left is for someone to do the work. I forgot about the libstdc++ headers that would need to change. We're going to try and have someone here work on this in the next 1-2 months as we're getting bitten by this.
Comment 12 David Edelsohn 2007-12-21 23:23:04 UTC
If AIX G++ users are focussing on 64-bit apps and large files, I'm willing to switch the default to _LARGE_FILES.  The changes would be in gcc/config/rs6000/aix*.h CPP_SPEC and libstdc++/config/os/aix/os_defines.h, plus any fallout from the change.
Comment 13 The Written Word 2007-12-21 23:25:07 UTC
(In reply to comment #12)
> If AIX G++ users are focussing on 64-bit apps and large files, I'm willing to
> switch the default to _LARGE_FILES.  The changes would be in
> gcc/config/rs6000/aix*.h CPP_SPEC and libstdc++/config/os/aix/os_defines.h,
> plus any fallout from the change.

We only care about -D_LARGE_FILES to support files >2G while running a 32-bit app. We're not building any 64-bit apps.
Comment 14 Tom Coleman 2007-12-22 20:18:11 UTC
(In reply to comment #12)

I don't see it as so much a matter of "focus".  

A lot of applications test for _LARGE_FILES in the C library and will set it as a configuration option if supported.  As far as straight C goes, there's no problem with gcc on AIX.

But if _LARGE_FILES is set, the fact that g++ can't compile C++ applications using cstdio on AIX is a source of both frustration and annoyance.

Does G++ really have to be either one way or the other?  Can't it also include conditional code to check for _LARGE_FILES?  Maybe I'm missing something here.



Comment 15 David Edelsohn 2007-12-22 20:52:39 UTC
Subject: Re:  AIX g++ -D_LARGE_FILES fails to compile #include <iostream> 

>>>>> tcoleman at autowares dot com writes:

Tom> Does G++ really have to be either one way or the other?  Can't it also include
Tom> conditional code to check for _LARGE_FILES?  Maybe I'm missing something here.

	GNU libc does not use _LARGE_FILE to distinguish large offsets in
32-bit file I/O.  Why would the GCC libstdc++ community be interested in
complicating its header files with AIX-specific checks?
Comment 16 Peter O'Gorman 2007-12-31 19:20:28 UTC
(In reply to comment #15)
>         GNU libc does not use _LARGE_FILE to distinguish large offsets in
> 32-bit file I/O.  Why would the GCC libstdc++ community be interested in
> complicating its header files with AIX-specific checks?

I went back through the ChangeLog to find out where these #undef came from and why.
http://gcc.gnu.org/ml/gcc-patches/2001-02/msg00275.html
http://gcc.gnu.org/ml/gcc-patches/2000-11/msg00073.html
http://gcc.gnu.org/ml/libstdc++/2000-11/msg00041.html

It looks to me like while the undefs were at first added little by little to solve problems as they arose, they were eventually simply added en-mass without any particular bug being solved.

This was also a problem on solaris:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=4850

but it was "fixed" with the introduction of #pragma redefine_extname:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=4850

Because it is not currently possible to build an autoconf based project on aix with gcc without doing --disable-largefile, we would like to see this fixed.

How about something like:

#if defined(fopen) && fopen != fopen64
#undef fopen
#endif

for all of:
fseeko fseeko64
ftello ftello64
fgetpos fgetpos64
fsetpos fsetpos64
fopen fopen64
freopen freopen64

in libstdc++'s cstdio.

If something like this is acceptable to David, we will try it on aix and linux and submit to -patches.
Comment 17 Peter O'Gorman 2007-12-31 19:28:32 UTC
(In reply to comment #16)

> This was also a problem on solaris:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=4850

I mean

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=4886

Comment 18 The Written Word 2007-12-31 19:39:11 UTC
(In reply to comment #16)
> Because it is not currently possible to build an autoconf based project on aix
> with gcc without doing --disable-largefile, we would like to see this fixed.

More accurately, it is possible to use `gcc' and large files, just not `g++' and large files.
Comment 19 The Written Word 2008-03-03 21:12:13 UTC
Proposed patch:
http://gcc.gnu.org/ml/gcc-patches/2008-03/msg00162.html
Comment 20 Ralf Wildenhues 2008-03-07 06:54:45 UTC
Subject: Bug 20366

Author: rwild
Date: Fri Mar  7 06:53:59 2008
New Revision: 133001

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133001
Log:
fixincludes/
2008-03-07  Peter O'Gorman  <pogma@thewrittenword.com>

	PR c++/20366
	* inclhack.def (AAB_aix_stdio): Hack stdio.h for -D_LARGE_FILES.
	* fixincl.x: Regenerate.
	* tests/base/stdio.h: Add test.

gcc/testsuite/
2008-03-07  Peter O'Gorman  <pogma@thewrittenword.com>

	PR c++/20366
	* g++.dg/other/pr20366.C: New test.

Added:
    trunk/gcc/testsuite/g++.dg/other/pr20366.C
Modified:
    trunk/fixincludes/ChangeLog
    trunk/fixincludes/fixincl.x
    trunk/fixincludes/inclhack.def
    trunk/fixincludes/tests/base/stdio.h
    trunk/gcc/testsuite/ChangeLog

Comment 21 Peter O'Gorman 2010-12-27 19:38:47 UTC
Fixed since 4.4.0