Re: [PATCH 00/17] RFC: Addding a unit testing framework to gcc

[Jeff Law <law at redhat dot com>]

On 06/10/2015 11:16 AM, David Malcolm wrote:
> On Wed, 2015-06-10 at 17:34 +0200, Jakub Jelinek wrote:
> > On Wed, Jun 10, 2015 at 11:24:41AM -0400, David Malcolm wrote:
> > > I picked the Google Test framework:
> > >    http://code.google.com/p/googletest/
> >
> > I must say I'm not very excited about using this, it won't integrate
> > very well with dejagnu
>
> Why is that a goal?  I've been using DejaGnu's unittesting API for
> testing the jit, and it is... suboptimal, to put it mildly.
Everyone on the project is already using dejagnu.  Adding another 
framework means everyone has to learn new ways of doing the same thing. 
 I think Jakub touched on multilibs, flag passing, result parsing, etc. 
   Note these things are embedded in the daily practices of many 
developers, build bots and even within our Makefiles for parallization 
of testing.

I'd be less likely to object if the google test framework was wrapped by 
dejagnu and had those low level things wired up properly.  I'd also be 
less likely to object if we converted *everything* to googletest.  The 
former is most likely orders of magnitude easier :-)


> > whether talking about results (will it provide
> > some *.log/*.sum file with FAIL/XFAIL/PASS/XPASS etc. lines?),
>
> It doesn't have an output formatter for the DejaGnu format, but I guess
> I could write one.  The gtest standard output format is IMHO superior to
> DejaGnu's since it tells you start-of-test/end-of-test on separate
> lines, so you can see which test killed things in the event of total
> failure, and the per-test timings (which can be disabled if you want to
> do diffs).
You can still get all that output if you wrap googletest with dejagnu -- 
that low level information would go into the .log file and the .sum 
would just have the pass, fail, xfail, xpass, etc messages.


> >   choosing
> > what options to use, e.g. global
> > RUNTESTFLAGS='--target_board=unix/\{-m32,-m64\}'
> > to test everything twice for 32-bit and 64-bit, will that run just
> > all unittests twice the same?,
>
> Fair enough; yes, as written, RUNTESTFLAGS is ignored; it will run
> everything once.  However we could express such a loop directly in the
> "main" of the
But using the syntax above is already well known by developers and 
build/test bots.  I'm not keen to change it.


And note, I *hate* dejagnu.

Jeff