a request with help with modifying the libstdc++ test suite harness to analyze a copy of libstdc++ other than the one in the source unpacking and build directory

[Abe]

First: my apologies to Jonathan if he gets this in duplicate.  If so,
please just tell me, and I will not CC again when I reply to the list
as well.


> The docs at http://gcc.gnu.org/onlinedocs/libstdc++/manual/test.html say:
[…]
> runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite

Thanks for that!  I had forgotten about that.  IIRC I tried that a
long-enough time ago that I had already forgotten about that command,
but when I tried it the first time I didn`t have DejaGNU installed yet
so it didn`t work at that time.  ;-)


> Have you tried that method? i.e. not running 'make check' within the
> build tree (which causes it to use the just-built compiler) but
> launching DejaGNU from another location, where it finds GCC in your PATH?

I have now, and thanks _very_ much for your help.  However, not all
the problem[s] is/are solved, I think.  Now [when running the test
suite in the abovementioned way], I get these errors that I`m pretty
sure I didn`t get from "make check":

  ERROR: tcl error sourcing
/usr/src/gcc-4.7.2/libstdc++-v3/testsuite/libstdc++-abi/abi.exp.
  ERROR: can't read "baseline_subdir_switch": no such variable
      while executing
  "eval exec $cxx $baseline_subdir_switch"
      invoked from within
  "set baseline_subdir "[eval exec $cxx $baseline_subdir_switch]""
      (file "/usr/src/gcc-4.7.2/libstdc++-v3/testsuite/libstdc++-abi/abi.exp"
line 27)
      invoked from within
  "source /usr/src/gcc-4.7.2/libstdc++-v3/testsuite/libstdc++-abi/abi.exp"
      ("uplevel" body line 1)
      invoked from within
  "uplevel #0 source
/usr/src/gcc-4.7.2/libstdc++-v3/testsuite/libstdc++-abi/abi.exp"
      invoked from within
  "catch "uplevel #0 source $test_file_name""
  Running /usr/src/gcc-4.7.2/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
...
  […]


I also seem to get several failures [i.e. "FAIL" in the log] that I
don`t normally get, i.e. that I didn`t get with "make check".

Can somebody please tell me how to cleanly [i.e. w/o modifying any
non-settings files] get it to use another instance of GCC, in the
above context [i.e. "runtest --tool <…>"], i.e. other than the
currently-default "g++" that`s on the path?  Based on my reading of
"man expect" [manpage dated "29 December 1994"] and poking my nose
inside at least one of the Expect files, I tried "runtest --tool
libstdc++ --srcdir=/usr/src/gcc-4.7.2/libstdc++-v3/testsuite/
cxx=g++_4.7.2", but that gave me “Illegal Argument "cxx=g++_4.7.2"”
and “try "runtest --help" for option list”; the output from "runtest
--help" is very confusing.  I looked around the Web, and found stuff
that seemed to indicate that I should try writing a "site.exp" file,
so I did, with each of these lines [only 1 not-commented-out line at a
time]:

  # set cxx "g++_4.7.2"
  # set cxx [transform "g++_4.7.2"]
  set CXX "g++_4.7.2"

… but "no dice": it still only ran "g++", despite echoing "Found
/Volumes/[…]/GCC_STL_testing/testing_KO_00001000/site.exp" [which
might have required "-v" or "--verbose"].  Ditto WRT "~/.dejagnurc": I
got "Found ~/.dejagnurc" on the console, but it ignored my setting
attempts.  Methinks perhaps at least one of the things I tried was
correct, but "libstdc++.exp" is overwriting user settings blindly [IDK
how to fix that].  When I modified "libstdc++.exp" [on the relevant
system it`s in "/usr/src/gcc-4.7.2/libstdc++-v3/testsuite/lib"] as per
this diff:

 	  verbose -log "LD_LIBRARY_PATH = $env(LD_LIBRARY_PATH)"
 	}
     } else {
-	set compiler [transform "g++_4.7.2"]
+	set compiler [transform "g++"]
     }

     # Set the default timeout for v3 tests.
     set tool_timeout 600

     # Default settings.
-    set cxx [transform "g++_4.7.2"]
+    set cxx [transform "g++"]
     set cxxflags "-g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0"
     set cxxpchflags ""
     set cxxldflags ""


… it finally gave in. ;-)  Is there a way I should have tried that I
missed, or is this just an inflexibility bug in "libstdc++.exp"?  And
what does the "transform" in “[transform "g++"]” mean?  I tried Google
and Expect and Tcl documentation, but I came up blank on that one.

Also about "libstdc++.exp": can somebody please tell me what`s the
difference between the variables "compiler" and "cxx"?


>> […] the modified GCC STL I want to run the test
>> suite over is modified as a copy of the libstdc++ directory I got
>> after doing "make install" […]

> It shouldn't matter as long as the compiler can find the headers that
> are included by the tests.

Cool!  Thanks for the feedback.


>> [I`m guessing it`s _not_ the case that some test driver code is inspecting my proposed alternative
>> directories, finding the contents/layout different than expected, and then reverting to the defaults […]

> That is not the case.

Thanks.


>> My apologies for the length of this message; I figured it would be
>> better to get all the background information into the request for help
>> rather than having people need to "drag it out of me" before they can help.

> Thank you, that does make it easier :-)

You are welcome!  Thank _you_ for helping.  This stuff is important to
me, as you might have guessed.

Regards,

Abe