Re: libstdc++ test suite still drives machine into swap

[Phil Edwards <pedwards at disaster dot jaj dot com>]

I'm revisting this topic because my machine is dying.

The results from mkcheck don't match the results from the dejagnu check.
Which probably needs looking into, but I don't have the dejagnu knowledge
to do so myself.  (Nor, to be brutally honest, the desire.  I could express
my professional optinon about dejagnu, but Mark has asked us to not use
those words on the public lists.)

Anyhow, I noticed that the machine swapped itself to pieces, which lead
me to this thread from 26 June.

There is a minor patch at the end.  The real point of this message is to
get the patch approved.  If no discussion is generated, that's okay.  :-)

A month ago, Zack Weinberg wrote:
> The libstdc++ test suite still drives my machine, which has plenty of
> RAM thank you, into swap.  I suspect this is because ulimit -d does
> not actually work on Linux, because libc allocates huge blocks with
> anonymous mmap, and anonymous mmap ignores that particular limit.

Somewhere is a thread from me, Gaby, and a couple other people explaining
that ulimit doesn't work with dejagnu anyhow.  The ulimit is executed
in a subshell, which instantly exits.  DG doesn't allow executation of
arbitrary commands before running a test executable in the same shell as
the executable.  The same technique works fine in the mkcheck script.

On Linux, you must use -v instead of -d.  Which we do, in mklimit.
Still doesn't help DG.


> The last time I brought this up I tried to explain why these tests
> shouldn't be done at all and no one got it.

I may be wrong, but I think you're mistaken.

> What actually happens is that execution never gets past the
> std::string constructor.  Malloc lies.  It gives you a pointer to
> memory that does not exist, on the theory that you won't *really* use
> all of it.  Then the operating system kills the process - if you're
> lucky - while it's trying to write 'A' over a gigabyte of virtual
> memory that the computer does not actually have.[1]
[...]
> [1] If you're unlucky, the computer crashes, or the operating system
> kills half a dozen random innocent processes before it hits the one
> that's eating all the memory.

Maybe on your OS.  This doesn't happen to me under Solaris, or Linux, or
OSF/1, or IRIX.  Even without resource limits being applied to the user's
shell and inherited by subprocesses.  Granted that many implementations
of malloc overcommit, I just don't see random processes dying.  More on
this below.


> The *only* way you can test the constraints on max_size, size,
> capacity, etc. is if you can somehow create a basic_string<>
> instantiation for which max_size is small enough that you *can*
> allocate that much memory without bringing the computer to its knees.[2]
[...]
> [2] I tried to figure out how to do that and got lost in a maze of
> twisty little template classes.  But I don't really speak C++.  I
> would suggest that #define __STD_STRING_MAX_SIZE or some such should
> override the default for std::string's max_size() method.

I like this idea on the face of it.  I haven't really thought it through,
but this sounds useful.


> [Sample code above was taken from 21_strings/ctor_copy_dtor.cc.  That
> particular test has been #if 0'ed out with a comment referencing one
> of the previous threads on this subject.  However, not all the tests
> with this problem have been disabled.  Right now I'm not sure which
> test is the current problem.]

We're doing the same thing in 21_strings/insert.cc; we create a maximum-sized
string and then copy it (fortunately it's refcounting and only goes into
terminal thrash once).  Included is a patch that does the same limiting
we did last October.  Comments anyone?

> [I might add that surely it is only necessary to test this sort of
> thing *once*, not the five or six times it appears to be done in the
> current test suite.]

And here's the real issue, or close to the real issue.  Do we really
need to test these corner cases, especially when we know that malloc
might overcommit, or that the kernel is useless when it comes to properly
killing processes that exceed their resource limits?  Do we need to test
"identical" cases using different member functions?

I say yes, for two reasons:  first, the cases might not actually be
identical, and the point of the testsuite is to let us mess with the
implementation without changing the semantics.  Let's say we test a
"too large" string using its constructor; do we need to test "too large"
semantics when inserting data into an already-close-to-maximal string?
I submit that we do.  (Zack's suggestion about artificially limiting
max_size()/npos would make this painless, I think, but I don't know how
to pass arbitrary -D options inside dejagnu.)

Secondly:  some user out there is going to try and allocate (size_t)(-1)
bytes for a string, and we have to behave correctly.  This argues /against/
limiting max_size()/npos artificially.  We need to know what actually
happens in these cases.  And if that causes the kernel to assassinate
random processes, then the solution IMO isn't to change the testcase,
the solution is to tell the user "on such-and-such OS, don't do that".


'Night,
Phil


2001-07-31  Phil Edwards  <pme@sources.redhat.com>

	* insert.cc:  Don't allocate large amounts of memory; see change
	to ctor_copy_dtor.cc on 2000-10-21.


Index: insert.cc
===================================================================
RCS file: /home/pme/Repositories/GCC/gcc/libstdc++-v3/testsuite/21_strings/insert.cc,v
retrieving revision 1.3
diff -u -3 -p -r1.3 insert.cc
--- insert.cc	2001/05/12 16:51:41	1.3
+++ insert.cc	2001/08/01 01:11:05
@@ -79,12 +79,14 @@ int test01(void)
 
   csz01 = str01.max_size();
   try {
-    std::string str04(csz01, 'b'); 
+//    std::string str04(csz01, 'b'); 
+std::string str04(1, 'b'); 
     str03 = str04; 
     csz02 = str02.size();
     try {
       str03.insert(0, str02, 0, 5);
-      VERIFY( false );
+//      VERIFY( false );
+VERIFY( str03.size() == 6 );
     }		 
     catch(std::length_error& fail) {
       VERIFY( true );