This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] Temporarily enable ac_fold_checking for --enable-checking=yes.


On 6/23/07, Mark Mitchell <mark@codesourcery.com> wrote:
Sebastian Pop wrote:
> Here are more numbers from my laptop i686-linux, bootstrapping
> with --enable-checking=yes,fold
> Elapsed (wall clock) time (h:mm:ss or m:ss): 2:32:58
> That was with a small patch for PR32461 on which I'm working now.

I love automated testing.

But, I'm concerned about some of the things that we're doing.  We have a
few testcases in the C testsuite that take up huge fractions of the
total time.  The libstdc++ testsuite now runs for a long time after the
G++ testsuite is done, even running both in parallel.  Additional
testing during bootstraps makes things slower.

I have two concerns.  First, I think some of the testing we do is not
delivering value proportional to cost.  Is 20061226-1.c (sp?) really
delivering anything close to the same percent of our quality that it
costs in testing?  If not, should we let it take up several percent of
our testing?  Is the libstdc++ testsuite delivering C++ front-end
quality proportional to its cost?  Second, do we all need to do all of
the testing all of the time?  Or, could some of us do it some of the
time, and could those of us who cause problems fix any problems revealed?

I am *not* complaining about the existence of libstdc++ testsuite or
20061226-1.c.  In fact, I'm very impressed by the libstdc++ testsuite,
and the fact that the maintainers have worked hard to expand it.  My
concern (as a C++ maintainer) is that it very rarely finds problems with
my changes, but does impact my ability to make those changes.  However,
I'm not comfortable suggesting that we not run the libstdc++ testsuite
for G++ front-end changes; it finds problems often enough that I'd feel
a little nervous not running it.

I agree in general with the above.


The reason for this musing is that test time does impact productivity.
It takes hours to test a change.  Even assuming unlimited machine
resources, it's hard to keep track of multiple pending patches.  If I
fix one thing, run tests, go fix another thing, run more tests, go fix
another, run more tests, and the first tests aren't done yet, I start to
lose track of what I'm doing.  And, for many people, even in this modern
age, machine resources aren't unlimited.  If we were talking about
making a 1-minute build/test cycle into a 2-minute build/test cycle,
that wouldn't worry me, but we're starting to stretch it out to the
point where a build/test cycle can take the better part of a day.

But while this at first looks like a problem we maybe just need "the tools" to handle working on things in parallel. I am able to work on about three patches in parallel, interleaving the testing time (which takes about two hours for all,ada, but disabled multilibs and libjava on x86_64 for me).

Maybe we could have one or two people running autotesters with lots and
lots of checking turned on, and then reinforce the idea that a solemn
responsibility of contributing GCC is fixing regressions that you introduce?

And I am definitely against this. I would even make our current rules stricter and require bootstrap even for frontend-only changes. Automatic testing is cheap - even if it takes a while. Going through testresults, filing bugs and going after the people that introduced them is not.

Let's not waste other peoples time for the sake of your own ease.

Richard.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]