Re: ABI regression tester, version 0.0.0pre2.alpha7

[Phil Edwards <phil at jaj dot com>]

On Tue, Aug 20, 2002 at 01:08:52AM -0700, Benjamin Kosnik wrote:
> 
> > I'm testing it by adding a 50-integer array to the declaration of
> > std::nothrow_t.  So tomorrow's log should show a "problem" when the
> > std::nothrow object has changed in size from 1 byte to 200.  After verifying
> > that everything can work without human handholding, I'll fix the code back
> > and start getting real reports on Tuesday.
> 
> It looks like this works, thanks. This is the test with switching the
> libstdc++.so binary between trees?

No, this is the one that uses Uli's readelf-pipe-sed-pipe-awk script on the
two trees, then diffs them.  If the diff is nonzero then we have problems.
(I need to make the diff output more readable, but normally there should
be none at all.)

The .so swap should be another kind of test, but I haven't automated
that one.


> 1) build abi-checker app
> 2) generate current list of exported symbols from libstdc++.so
> 3) run abi-checker app against known baseline and test list, report issues
> 
> Thoughts? This should allow people on x86/linux to verify changes don't
> mess with the "stable" 3.2.0 libstdc++ abi.

You've preempted the "thoughts I was going to post on putting this into
the testsuite."  Great minds...

The two .sh scripts in your tarball are only different in that one reads
libstdc++.so.5.0.0, and the other reads .5.0.1.  Since we use symlinks
in the build/install directory, you could just read libstdc++.so, or
libstdc++.so.5, and need to only maintain a single script.


> 1) as written, sizes of exported object are for x86/linux and may not be
> portable to other oses and cpu's... I'm hoping I can get somebody like
> Loren (please) to test on x86/BSD to see how far off I am. This isn't a
> super big deal to me, because all I was really trying to check was the
> list of exported symbols, which should remain constant across cpus (?).

I think this is right.

> It's easy to change what is being compared, so comparing sizes of
> objects can be turned off.

If we start tracking more numbers, probably a "--with-cpu-specific-stuff"
flag should go into that script.  Or "--only--portable," etc.


> 2) testsuite integration
> (util subdir? makefile work? checking for all the readelf deps? anyway......) 

The baseline files could go into config.  We have a system in place now to
make decisions based on cpu/os combinations; we could take advantage of that.


> 3) doesn't test vtable or object layout, or alignment issues. It would
> be possible to add at some later date, but the formalization of the abi
> checking has to start somewhere. 

Yeah, this is giving me fits.  I'd /really/ like some decent BFD
documentation; with that we could take apart the binary ourselves.


> 4) generating the exported symbol list. Weak symbols cannot be
> completely ignored, as a bunch of the template linkage is weak. So, I've
> included them.

Ah.  I should start including those in my autocrasher, then.


> I'm still not exactly sure I'm getting all the exported
> symbols but I think this is really close.

The difference between the output of your script, and simply readelf'ing
the entire thing, should point out what we're missing.


Phil

-- 
I would therefore like to posit that computing's central challenge, viz. "How
not to make a mess of it," has /not/ been met.
                                                 - Edsger Dijkstra, 1930-2002