Bug 16401 - [3.4 Regression] ostringstream in gcc 3.4.x very slow for big data
Summary: [3.4 Regression] ostringstream in gcc 3.4.x very slow for big data
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libstdc++ (show other bugs)
Version: 3.4.1
: P2 normal
Target Milestone: 3.4.2
Assignee: Paolo Carlini
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-07-07 08:10 UTC by Jan Starzynski
Modified: 2004-10-30 21:10 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work: 3.3.3 4.0.0
Known to fail: 3.4.0
Last reconfirmed: 2004-07-07 09:06:34


Attachments
Patch for mainline/3_4 (739 bytes, patch)
2004-07-07 10:08 UTC, Paolo Carlini
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Starzynski 2004-07-07 08:10:00 UTC
Hello, 
 
I'm using ostringstream to write big amounts of data (some 10MB). For simplicity the 
data is written as single bytes. For gcc 3.3.x the time per character needed is almost 
constant for different amounts of data. But for 3.4.x this time increases dramatically:  
the time / char - ratio for 10 MB is 8 times that for 1 MB. 
 
This performance-decrease does not happen to string-concatenation using +=. 
 
Appended is an example program that has a switch to measure the performance of 
both string and ostringstream. 
 
I'm using linux 2.4.20 with SuSE 8.2 on a pentium 4, 512 MB Ram, 2.4 GHz. The gcc 
is 3.4.1, but the results are similar to 3.4.0. gcc -v tells:  
 
configured with: ../gcc-3.4.1/configure --enable-threads=posix --prefix=/usr 
--with-local-prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man 
--libdir=/usr/lib --enable-languages=c,c++,f77,objc,java,ada --disable-checking 
--enable-libgcj --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib 
--with-system-zlib --enable-shared --enable-__cxa_atexit i486-suse-linux 
Thread-Modell: posix 
gcc-Version 3.4.1 
 
///// compile with ///// 
g++ -O2 -march=pentium4 -Wall -o strstream_t strstream_t.cpp 
 
///// strstream_t.cpp ///// 
 
/// 0: use ostringstream 
/// 1: use string 
#define STR_ONLY 0 
 
#include <iostream> 
 
#include <sys/time.h> 
#include <sys/resource.h> 
#include <unistd.h> 
 
#if STR_ONLY 
# include <string> 
#else 
# include <sstream> 
#endif 
 
using namespace std; 
 
static double time2dbl(long sec, long usec) 
{ 
  return sec + usec * 1e-6; 
} 
 
/// gives time in seconds and microseconds 
static void FineTimeInt(unsigned int *sec, unsigned int *musec) 
{ 
  static struct timeval last_tv; 
  struct timeval tv; 
 
  gettimeofday(&tv,NULL); 
 
  last_tv = tv; 
  *sec = tv.tv_sec; 
  *musec = tv.tv_usec; 
} 
 
/// gives time in seconds 
static double FineTime(void) 
{ 
  unsigned int sec, musec; 
  FineTimeInt(&sec, &musec); 
  return time2dbl(sec, musec); 
} 
 
int main() 
{ 
  double oldr = 1; 
  for(int n = 1; n <=  10000000; n *= 10) { 
#if STR_ONLY 
    string str; 
#else 
    ostringstream str; 
#endif 
    double t0 = FineTime(); 
    for(int i = 0; i < n; i++) { 
#if STR_ONLY 
      str += char(i); 
#else 
      str << char(i); 
#endif 
    } 
    double t1 = FineTime(); 
    double t = t1 - t0; 
#if STR_ONLY 
    double l = str.length(); 
#else 
    double l = str.str().length(); 
#endif 
    double r = t / l; 
    cout << n << " " << l << " " <<  t << " " << r << " " <<  r / oldr << endl; 
    oldr = r; 
  } 
}
Comment 1 Paolo Carlini 2004-07-07 09:06:34 UTC
Confirmed, sigh. The fix is easy, just grow the internal string object
exponentially and not one page at a time: I tried to be too smart during the
redesign of sstream... Anyway, I should have a patch shortly. Thanks for your 
report and sorry.
Comment 2 Jan Starzynski 2004-07-07 09:49:41 UTC
Subject: Re:  ostringstream in gcc 3.4.x very slow for big data

> Anyway, I should have a patch shortly. Thanks for your
> report and sorry.

It's a pitty I did'nt report that earlier as I saw with 3.4.0 and: shit 
happens ;-)

Comment 3 Paolo Carlini 2004-07-07 10:08:59 UTC
Created attachment 6705 [details]
Patch for mainline/3_4
Comment 4 Paolo Carlini 2004-07-07 10:10:47 UTC
;) Anyway, I have attached a patch (already regtested) that you can use in your
local tree, before 3.4.2 is out. Of course, in case you encounter any problem
please let us know ASAP (this time ;)
Comment 5 GCC Commits 2004-07-07 21:48:05 UTC
Subject: Bug 16401

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	paolo@gcc.gnu.org	2004-07-07 21:48:01

Modified files:
	libstdc++-v3   : ChangeLog 
	libstdc++-v3/include/bits: sstream.tcc 
Added files:
	libstdc++-v3/testsuite/27_io/basic_stringbuf/overflow/char: 1.cc 
	libstdc++-v3/testsuite/performance/27_io: stringbuf_overflow.cc 

Log message:
	2004-07-07  Paolo Carlini  <pcarlini@suse.de>
	
	PR libstdc++/16401
	* include/bits/sstream.tcc (overflow): When reallocating _M_string
	use an exponential grow policy.
	* testsuite/27_io/basic_stringbuf/overflow/char/1.cc: New.
	* testsuite/performance/27_io/stringbuf_overflow.cc: New.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/ChangeLog.diff?cvsroot=gcc&r1=1.2564&r2=1.2565
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/include/bits/sstream.tcc.diff?cvsroot=gcc&r1=1.39&r2=1.40
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/testsuite/27_io/basic_stringbuf/overflow/char/1.cc.diff?cvsroot=gcc&r1=NONE&r2=1.1
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/testsuite/performance/27_io/stringbuf_overflow.cc.diff?cvsroot=gcc&r1=NONE&r2=1.1

Comment 6 GCC Commits 2004-07-14 16:53:56 UTC
Subject: Bug 16401

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_4-branch
Changes by:	paolo@gcc.gnu.org	2004-07-14 16:53:49

Modified files:
	libstdc++-v3   : ChangeLog 
	libstdc++-v3/include/bits: sstream.tcc 
Added files:
	libstdc++-v3/testsuite/27_io/basic_stringbuf/overflow/char: 1.cc 
	libstdc++-v3/testsuite/performance/27_io: stringbuf_overflow.cc 

Log message:
	2004-07-14  Paolo Carlini  <pcarlini@suse.de>
	
	PR libstdc++/16401
	* include/bits/sstream.tcc (overflow): When reallocating _M_string
	use an exponential grow policy.
	* testsuite/27_io/basic_stringbuf/overflow/char/1.cc: New.
	* testsuite/performance/27_io/stringbuf_overflow.cc: New.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.2224.2.141&r2=1.2224.2.142
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/include/bits/sstream.tcc.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.37.4.2&r2=1.37.4.3
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/testsuite/27_io/basic_stringbuf/overflow/char/1.cc.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=NONE&r2=1.1.4.1
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc++-v3/testsuite/performance/27_io/stringbuf_overflow.cc.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=NONE&r2=1.1.4.1

Comment 7 Paolo Carlini 2004-07-14 16:55:09 UTC
Fixed for 3.4.2.