Large File iostreams (was Re: [Patch] Fix libstdc++/11378...)

[Nathan Myers]

On Thu, Jul 10, 2003 at 08:35:43PM -0000, P?tur Run?lfsson wrote:
> Nathan Myers wrote:
> >> #include <ios>
> >> int main()
> >> {
> >>   std::streamoff m, n;
> >>   std::streamoff o(m - n);
> >> }
> >
> > Uh-oh.  If it's a conforming program, then adding overloads that make
> > it _not_ truncate its result is not allowed.  That looks like a defect.
> 
> I agree.
> 
> Elsewhere, Nathan wrote:
> > The type
> > safety argument dictates that we should use a class type for
> > std::streamoff containing a 64-bit integer type,
> 
> That seems absolutely neccessary so that errors such as
> 
> fstream f;
> ...
> long pos = f.tellg();
> 
> can be caught at compile time, especially since this works as
> expected with the current implementation.

Yes.

> > and publicly defining
> > (only!) the operations required in 27.4.3.2,
> 
> Unless I'm missing something, I don't see any requirement that
> streamoff or fpos<> be DefaultConstructible.

Indeed.  That seems like another defect, or two.  Is anybody getting 
these written down? :-)  I propose not to adopt those defects into 
libstdc++, since the correct resolution seems pretty obvious.

> > 27.4.3.2
> > requires that it convert explicitly (but not implicitly!) from,
> > [...] std::streamsize
> 
> The descriptions of basic_istream::tellg and basic_ostream::tellp
> seem to assume an implicit conversion from streamsize to streamoff.
> I don't see any reason to make that conversion explicit, since it
> is required to be lossless.
> 
> However, I don't see any requirement that the conversion from
> fpos<> to streamoff be implicit. Since that is a lossy conversion
> (it loses the conversion state) fpos<>::operator streamoff()
> should be dropped and streamoff should have an explicit
> constructor taking a fpos<>.

Agreed.

> > and implicitly to, std::streamsize
> 
> If streamoff is 64-bits and streamsize 32, then the conversion
> from streamoff to streamsize is lossy. It would be nice if the
> compiler could give a warning about that.
> 
> streamoff o;
> ...
> streamsize s = o; // Should get a warning pointing to this line!
> 
> One way to achieve this (not neccesarily the only, or the best way)
> is to define streamoff::operator long long(), and not
> streamoff::operator streamsize().

Another might be to define an intermediate type to consume an implicit
conversion.  The standard doesn't actually require implicit conversion
to streamsize, it is just that you can't define explicit "op T()" 
members (my fault, maybe) and you can't define an explicit constructor
for a built-in integer type :-).

> ---
> 
> All this looks so complex that I wonder if the best way to solve
> it is to just add header <ext/fstream64> that contains:
>  [...]
> This isn't so very far fetched; after all, it is what the C
> library does (off_t vs. off64_t, fseeko vs. fseeko64 etc.).
> It can even be done without breaking ABI compatibility, it
> only requires adding an overload of __basic_file::seekoff
> that takes a streamoff64 and calls lseek64.

Absolutely not!  It's a nightmarish hack.  The only reason those exist 
in libc is ABI backward-compatibility, a problem we don't have.   

I don't see any fundamental problems in getting iostreams fully 64-bit 
clean -- just niggly details, and standard defects that needed to be 
flushed out anyhow.

Nathan Myers
ncm-nospam@cantrip.org