[Bug libstdc++/45574] ifstream::getline() is extremely slow

[tstarling at wikimedia dot org]

------- Comment #8 from tstarling at wikimedia dot org  2010-09-08 01:34 -------
(In reply to comment #5)
> For sure we cannot add virtual functions to basic_streambuf without breaking
> the ABI. 

I'm mostly looking for a long-term fix, to improve the speed of libstdc++
applications generally, especially those that don't have developers who would
go to the trouble to track down the source of slowness in their programs. The
short-term fix is to call ios::sync_with_stdio(false). So it's fine for me to
wait for the next major version. 

> Also, getline certainly isn't just fgets, takes a delim char, uses
> traits, etc. 

The delim char can be taken care of with getdelim(). I don't think it's
unreasonable to specialise for default traits, that would take care of 99% of
use cases.

> Sure, anyway, in principle you can often speed-up special cases,
> but also given that in ~5-7 years nobody else reported anything about the
> performance of the synced getline, I don't think anything is going to happen
> anytime soon, I could keep this open, but it would be futile, we have a lot of
> work to do, for C++0x, in particular.

OK, let's keep it open. 

(In reply to comment #6)
> By the way, I don't know anything about your testcase (it would be a good idea
> attaching something here, just in case), but on my machines, i7 mostly, I don't
> see anything similar to your performance gap, I see something more similar to
> 9-10x, which, considering that a real synced mode must be unbuffered, seems
> completely reasonable to me.

Probably the main difference is the number of bytes per line in the input file.
I'm using a file with 1M lines and an average of 429 bytes per line. Using less
bytes per line would bring more pressure on to the constant per-line overhead,
and less on the inner loop. 

But a 9-10x difference doesn't sound reasonable to me. The synced mode is not
unbuffered, before or after my suggested change, it uses the internal buffer in
glibc.

(In reply to comment #7)
> It's well known (though maybe not well enough) that you should use
> sync_with_stdio(false) to get good performance, unless you specifically need
> the synchronisation.

Maybe you should tell that to Paolo Carlini, who closed bug 15002 as "resolved
fixed" in 2004, or to Loren Rittle, who closed bug 5001 as "resolved fixed" in
2003, declaring "This issue was addressed by gcc 3.2.X such that
sync_with_stdio was no longer required for reasonable performance." 


-- 

tstarling at wikimedia dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
         Resolution|INVALID                     |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45574