Re: Why fixing 9533 doesn't fix 7744: revealed!

[Nathan Myers <ncm-nospam at cantrip dot org>]

On Fri, Mar 07, 2003 at 01:23:55AM -0600, Loren James Rittle wrote:
> In article <20030306040248 dot GA27802 at tofu dot dreamhost dot com> Nathan Myers writes:
> 
> > What was the reason that you chose a one-character buffer over making
> > it entirely unbuffered?  They seem equivalent, but having three modes
> > seems unnecessarily complicated.
> 
> Speaking for myself only, we might have hacked it without considering
> some other options.  Sorry.  Would disabling buffering in libc with
> setbuf(0, 0) have been enough to fully address the interactive cases?
> (Ah, I think if the abstraction error was concurrently fixed, this
> approach probably works.)

I haven't studied P?tur's explanation yet.  He's probably right.

> > Yes, this the path forward: make the native mode fast and clean, and 
> > make the hack mode (just barely) work.  I like the idea of two
> > entirely different objects, one a hack job on stdio and the other
> > purely POSIX-native, and sync_with_stdio(false) plugs the latter
> > in in place of the former.
> 
> OK, we are of like minds in how to get a possible major speed up on
> many POSIX platforms.
> 
> > <rant>
> > One thing to get away from is this idea that it's OK for C++ to be a
> > second-class citizen, dependent on C for its low-level access.  C++
> > is not a bag on C, it's a whole other language.  It has to be link-
> > compatible with C, but besides that it should stand alone.  The 
> > libstdc++.so should talk directly to the OS.  Non-POSIX users who want 
> > the library to perform well for them may supply patches making 
> > similarly low-level access for their platforms.  We shouldn't
> > (e.g.) introduce thread-unsafety just to avoid the need for low-level
> > access, or introduce undue complexity in trying to perform well where
> > we don't have low-level access.
> 
> Your view makes sense to me.  However, for the sake of argument only,
> it has been stated that libstdc++-v3 shall use the system's libc
> wherever possible (that view has had merit as well given the goals of
> the FSF).  

I don't know who made such a statement, or why anyone would make it.

Certainly it is necessary for code that has been compiled against
the native libc to continue to function without recompilation when 
libstdc++ is linked in, too.  That imposes constraints.  In 
environments with more primitive linkage than ELF, it imposes 
stronger constraints.  Further, native libc environments usually
provide non-standard extensions that we can't generally afford to
duplicate exactly.  All these come down to: don't break users'
existing code.  Within that limitation, we should be aggressive
about presenting the best native C++ environment we can. 

That might mean, for instance, providing our own implementation of 
_overflow and _underflow (or whatever it's called on the target), 
shadowing libc's, so that we can make sync_with_stdio() a no-op, 
on platforms with strong enough linkage support to enable it.

> Also, the standard C++ library includes libc by reference.

That only means that the same APIs are defined.  It does not (and
cannot) say how the required semantics are obtained.  In several
cases (e.g. strchr) it defines an interface different from C's.

> It explicitly states how e.g. C++ and C IO must interleave to comply
> with the standard.  

It leaves much implementation-defined, and doesn't even mention 
interactive semantics, never mind performance.  (In my view that's
a defect.)

> Would you advocate that `-lstdc++ -lc' (note
> order) should pick up C IO from libstdc++.so?  

Where we can make it work, that would be a Good Thing.  The C library 
is just not very big, compared to the C++ library.  Replicating all 
of it would not be a very big job, and just replicating key parts of 
it (but adapted to our needs, as mentioned above) seems well within 
our resources.

> As an aside, someone
> filed a PR against libstdc++ just today regarding global namespace
> leakage from system headers when they only included e.g. <vector>.  I
> think one could make a claim that programs that included only headers
> dictated by the C++ standard <[a-z]*> *and* <c[a-z]*> should have zero
> such leakage.  I think the root cause is the same (layering atop libc
> and the platform's system headers).  However the claim is easier said
> than done.  Multiple efforts have been conducted over the years to
> properly shadow system headers and move decls into std::, etc.

I agree with the PR filer.  That leakage is an embarrassment.  If my
one-year-old would just sleep through a night, I might be able to pick 
up the effort again.

> > Our C++ locale code should read the same files as the C library does,
> > instead of calling C library functions and trying to patch up after.  
> > For non Glibc systems, POSIX provides tools to extract the files to 
> > a standard format; the library installer can use these to generate
> > corresponding files for our library's use, in Glibc's format.  That 
> > way the code to read the files may be common to all POSIX platforms.  
> > Then, users of non-POSIX platforms can send utilities to convert their 
> > own locale files' formats (or make do with just the "C" locale).
> 
> I think these words are properly directed at me and my crazy ideas
> from last week RE hacking up generic locale support.  (FYI, I came to
> my senses and will install something that forces generic locale
> support as-is to report attempts to create unsupported locales.)  One
> issue, even on POSIX platforms, the libc might not handle multiple
> locales.  glibc does this by having parallel entry points for many IO
> calls.  But this observation plays into your observation about
> attempting to reuse libc thus making libstdc++ a second-class citizen
> on the platform.

I certainly did not mean to disparage your efforts.  I do think, though,
that we have gone far down what must be a dead-end road.

> > On many platforms we can achieve better integration with stdio by 
> > deriving streambuf from FILE and making the setg/setp/eback/gptr/
> > egptr/pptr/epptr family of functions refer to the same (inherited)
> > struct members as do getc and putc do, and make underflow and 
> > overflow use the same file descriptor member.  There's just not 
> > so much variation in stdio implementations.
> 
> Perhaps I am reading this final point too literally.  In all
> seriousness, do you know a trick to allow this in general?  E.g. one
> BSD looks like this:
> 
> typedef struct __sFILE {
>   [...standard says this is opaque...]
> } FILE;
> 
> How do I derive anything against FILE?  I mean, sure:
> 
> class foo_streambuf : FILE {};
> 
> is legal code; but (a) in general, FILE* may never be safely converted
> to foo_streambuf*; and (b) to peer in the opaqueness seems dangerous
> unless you really know the implementations well.

I would probably #if a FILE member into basic_streambuf<>, instead.

Peering into FILE is dangerous in that it exposes us to changes in 
private details of their implementation.  Still, FILE implementation 
is not exactly an area of active research.  I doubt those details change 
once in five years, and even then are carefully kept backward-compatible.  
The names vary (_overflow, __overflow, _Overflow, _IO_FILE_overflow, 
etc.) but what they do doesn't, much.  Adapters to local practice would 
fit well in the config tree, and the generic alternative is what we have 
now.  Target maintainers should have no trouble keeping their local 
configs up to date, and the performance and integration benefits would 
make it worth their while.

(I might be wrong about the practicality of using FILE directly; it 
may be that most implementations still only have three pointers instead 
of streambuf's six.  If I recall correctly, this _can_ be worked around 
by introducing an extra level of indirection in the sputc and sgetc 
inlines -- so that in filebuf the get and put pointers are aliased 
together, but in others they are not.  That might be considered a 
heavy cost to bear, per character.  On the other hand, it would only 
need to be borne on those platforms, and not on (e.g.) GNU systems.) 

Nathan Myers
ncm at cantrip dot org