This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: comparing DejaGNU results


On May 31, 2006, at 11:13 AM, James Lemke wrote:
My current version is attached.  If others find it useful I can
contribute it.  Comments and suggestions are welcome.

Please do. I'd welcome it (and scripts to generate html, to track known issues, to trim log files, to drive things and do on)... I think having a few different styles would be good, then people can try them all out and see which ones they like and why. Anyway, for me, it isn't yet better:


dg-cmp-results.sh: Verbosity is 0, Variant is "unix"

Older log file: 06031600/gcc.sum
sed: illegal option -- r
usage: sed script [-Ean] [-i extension] [file ...]
sed [-an] [-i extension] [-e script] ... [-f script_file] ... [file ...]
Newer log file: 06052905/gcc.sum
sed: illegal option -- r
usage: sed script [-Ean] [-i extension] [file ...]
sed [-an] [-i extension] [-e script] ... [-f script_file] ... [file ...]
sed: illegal option -- r
usage: sed script [-Ean] [-i extension] [file ...]
sed [-an] [-i extension] [-e script] ... [-f script_file] ... [file ...]
sed: illegal option -- r
usage: sed script [-Ean] [-i extension] [file ...]
sed [-an] [-i extension] [-e script] ... [-f script_file] ... [file ...]
sort: unrecognized option `--'
Try `sort --help' for more information.


:-(

When I fixed it up like this:

Attachment: t.txt
Description: Text document


I got:


Script error: trying to process:
O: PASS: objc.dg/bad-receiver-type.m (test for warnings, line 13)
N:PASS:objc.dg/bad-receiver-type.m (test for warnings, line 13)


:-(

mrs $ grep 'objc.dg/bad-receiver-type.m' 06031600/objc.sum 06052905/ objc.sum
06031600/objc.sum:PASS: objc.dg/bad-receiver-type.m (test for warnings, line 13)
06031600/objc.sum:PASS: objc.dg/bad-receiver-type.m (test for warnings, line 13)
06031600/objc.sum:PASS: objc.dg/bad-receiver-type.m (test for excess errors)
06052905/objc.sum:PASS: objc.dg/bad-receiver-type.m (test for warnings, line 13)
06052905/objc.sum:PASS: objc.dg/bad-receiver-type.m (test for warnings, line 13)
06052905/objc.sum:PASS: objc.dg/bad-receiver-type.m (test for excess errors)


Mine btw gives:

mrs $ cmp_logs  06031600/objc.sum 06052905/objc.sum
New tests that PASS:

objc.dg/dwarf-2.m (test for excess errors)
objc.dg/dwarf-2.m scan-assembler 0x10[^0-9a-f].*DW_AT_language
objc.dg/member-1.m  (test for errors, line 3)
objc.dg/member-1.m (test for excess errors)

Old tests that passed, that have disappeared: (Eeek!)

objc.dg/stubify-2.m scan-rtl-dump-not symbol_ref.*"objc_msgSend"


I tried all the results and couldn't find any that your script could analyze, except libgomp, but only because libgomp doesn't run on my system, so there are no results in it? Is it just me? I hand edited my results to be the first 30 or so from the Objective C++ testsuite, and then I got it to analyze and not dump out.


When I tried gcc, I had a chance to notice the timings, your version:

real    8m44.413s
user    2m0.714s
sys     7m54.847s

mine:

real    0m1.994s
user    0m1.756s
sys     0m0.556s

:-) Maybe you only run the script on toy results? Or, do you just drink lots of coffee? Now, I know mine is well more than 10-100x slower than it could be, but waiting 2 seconds isn't a hardship for me. Waiting 8 minutes strikes me as just a little slow.

The output of yours doesn't seem to be targeted for human eyes, the verbosity (at the least verbose setting) is about 121x more than mine for two random sets I had lying around, and that is with it cutting the output off really early due to the above problem. I predict that in normal day to day use, it is well better than 120,000x larger. What use do you get out of it?

Mine was designed to do two things, figure out if the results are interesting and not send email, if they are not, and to show engineers the `interesting' detailed results in priority order. It's meant to be run daily, and on good days, it produces no output. On normal days, output is 20-30 lines at most. It tries to balance the complaints with the occasional atta boy, to help with moral.
Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]