[Bug testsuite/25241] DejaGNU does not distinguish between errors and warnings

["manu at gcc dot gnu dot org" <gcc-bugzilla at gcc dot gnu dot org>]

------- Comment #12 from manu at gcc dot gnu dot org  2007-01-31 18:10 -------
(In reply to comment #11)
> The other DejaGnu procs that are wrapped in the GCC testsuite, like dg-test,
> are used within the .exp files, not within tests; that's why dg-error and
> dg-warning are different.  Developers are familiar with test directives but
> most have probably never looked inside the .exp files.

I don't think that they need to, except for the people responsible for
maintaining the testsuite. I have no idea of Tcl and the behaviour of expect is
still a mistery (even with --verbose --verbose --verbose --verbose --verbose
--debug --debug --debug). I wished I'd never had to look at it.

Some people just assume a behaviour and their tests are useless against a
warning changed to an error, interactions between the messages in different
lines, duplicated messages. Other people try to workaround these issues, so
they end up making difficult to fix the problem.

The question is: do we want dg-gcc-warning dg-gcc-error obligatory from now on?
Or we prefer to fix the testcases and wrap dg-warning and dg-error? I believe
doing nothing was already discarded.

> Ben Elliston is also a DejaGnu maintainer, and we've talked about fixing things
> like this in DejaGnu and moving GCC to a new version.  One problem is that it's
> difficult to move GCC to newer tools; another is that in this case, the
> behavior isn't compatible if we keep the same names.

The behaviour is not compatible because some testcases are broken (the fewer
from what I have seen), others are using workarounds (that we could try to
workaround, for example matching "error" any number of times for dg-error), and
others are using dg-warning to match messages from the compiler that are not
warnings nor errors.

However, I don't see how we can avoid to have our own directives (either
wrapped dg-* or either custom dg-gcc-*), since the difference between them
depend on GCC, so it will make dejagnu not useful for other projects. We even
have differences within GCC: gfortran uses its own style for diagnostics, that
is why my patch has to restore the original directives for gfortran.exp!

> Perhaps we should back up and design new test directives for diagnostics from
> scratch, starting by coming up with a list of requirements for them.  That
> might be easier to do on the wiki.

I agree with this but... can't we do something meanwhile? Isn't my patch an
improvement over the current situation? I am afraid there is no much interest
in fixing this. And as time goes by, the testsuite grows fast and changing the
current situation gets more impossible. :-(

I just want to know whether keep working in the patch and fixing testcases or
just forget about it and wait for the new directives. 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25241