Re: announce: testsuite summaries

[Marc Lehmann <pcg at goof dot com>]

On Sun, Nov 22, 1998 at 04:29:41PM -0200, Alexandre Oliva wrote:
> > Do you grep around for things that look like test_summary output? I
> > ask because my testing script sticks warn_summary output in there as
> > well...  is that breaking things for you?
> 
> Well, since he listed me as a good submitter, and I also append the
> output of warn_summary, I don't think it breaks anything...

Good submitter = parsable subject line, mostly ;)

The only thing missing in the test reports is a line with the compiler
version (this is the only thing I have to parse from the Subject: line),
maybe test_summary could be changed to include that? (e.g.

Compiler version haifa-enabled egcs-2.92.21 19981119 (gcc2 ss-980609 experimental)

The only real _problem_ is that we get bug reports for various cvs versions
without these being marked clearly. These are somewhat killing the
comparisons.


> What do you think of setting up an alternate e-mail address we could
> add to/replace in test_summary, so that you don't have to add them
> manually, and we can stop cluttering the list with these reports?

If you want to submit a test to the database without sending it to the list,
you can send it to:

testsuite-results@gcc.ml.org

But I don't think its a good idea in general. I still think people will want
to have a look at them "directly", i.e. when they browse the list (possibly
off-line). I'm deeply annoyed when I have to go on-line for some information
I _could_ have had in my inbox.

(this is only what I *think*, though, if people don't care for the
reports...)

On Sun, Nov 22, 1998 at 10:22:24PM +0000, Nix wrote:

> btw, my own compilation/test reports are about to expand slightly; I'm
> sticking a (non-context!) diff between this run's warn_summary/
> test_summary output and the last run's in there, before the full

Oh. Ok, what I do is: stop parsing on a few "keywords", like "Counting". It
would be nice if I could use a more specific marker ("WARN_SUMMARY"), so you
can use the magic word "Counting" without doing harm to your report ;)

I also start parsing again if a line contains "configure flags", so, since
most reports end with that line, you can also put comments below that line.

Again, this could be standardized, but it works quite ok in practise.

> run. I'm also thinking of changing it so that only weekly or so is the
> full report spammed to the egcs list; the rest of the time only diffs
> are sent.

"diffs"? To test reports??? Ok, they are quite large, so this would make
some sense, but how do this diffs look like?

Also, I'd like to have test-reports for snapshots _only_, if possible, as
comparing test reports is difficult enough with the same version, having
test reports for every day is IMO overkill.

> I would be very glad to have these as well; I feel vaguely guilty
> about quasi-spamming the list.

Would you feel better non-quasi-spamming my phone line? ;) Apart from that,
sending reports for random cvs versions is no good (and currently I ignore
them, depending on the subject)

> But on the other hand, having the reports going past everyone's eyes
> has one advantage - *everyone* sees them, so they can jog loose

Yes!

On Sun, Nov 22, 1998 at 10:49:14PM +0000, Nix wrote:
> Ah. I'll stop my occasional practice of putting long comments after it
> then.

(you can put them below the configure flags line, if your report
includes it at the bottom)

> > but fortunately you are very consistent in your reports!
> 
> If I was being inconsistent it would be damn surprising. It *is*
> mostly scripted ;)

But not scripted enough. People with low Isi/Diot ranking still send bogus
subjects sometimes ;)

> > | count(testreport.id) | name                        | address                               |
> > +----------------------+-----------------------------+---------------------------------------+
> > |                  185 | Alexandre Oliva             | oliva@dcc.unicamp.br                  |
> > |                   43 | Nix                         | nix-egcs@esperi.demon.co.uk           |
> 
> Gah. I think I may be submitting a few too many, I haven't been going
> nearly as long as Alexandre but I've got nearly as many tests...

I've not taken his the reports from the archives, I took them form my
personal egcs-folder, and I had the habit of removing test reports since
they were quite large. Somehow, Alexandres reports mostly survived this
unnatural selection ;)

> .... however, except for the list-spammage associated with this there
> is nothing wrong with that, I suppose.

Me, too.

> > |                   24 |                             | manfred@s-direktnet.de                |
> [cut]
> >          Now, if we only could save manfred from anonymity ;)
> 
> Time to spruce up the RFC822 From line parser, methinks.

Surprisingly (even for me) it turned out that I'm using the reply-to: header
(the logic behind this is that I might want to reply to the submitter. Also,
the reply-to is often more stable than the From:, since its often the
canonical address, and I don't want to have the same person with ten
entries in the submitters table)

The ideal way would be to use From: for name and comments and reply-to for
the address.

BTW: Manfred, if you read this, I updated your name into the database, since
     you were the only using reply-to without submitting a name ;)

      -----==-                                              |
      ----==-- _                                            |
      ---==---(_)__  __ ____  __       Marc Lehmann       +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com       |e|
      -=====/_/_//_/\_,_/ /_/\_\                          --+
    The choice of a GNU generation                        |
                                                          |