Re: GNATS vs. Bugzilla [Get raw message]

["Joseph S. Myers" <jsm28 at cam dot ac dot uk>]

On Mon, 19 Nov 2001, MattyT wrote:

> > 1. Works in a generally mail-based system: bug submission by a gccbug
> > script, bugs sent to gcc-bugs / gcc-prs mailing lists, follow-ups that go
> > to a bug tracking system address stored with the bug and sent on the
> > gcc-prs, full audit trails of every change to a bug sent to gcc-prs.
> > 2. Handles MIME in mail.
> > 5. Follow-ups sent to gcc-gnats for existing PRs get filed properly in any
> > new system.
> 
> I'm not really sure the benefit of this is that great.  Bugzilla has an
> email interface in its contrib directory, but it is rarely used I
> believe, I've never looked at it, and we don't guarantee it even works. 
> People just don't care enough to want an email interface when they're
> using Bugzilla.  I'm not sure if GNATS had an email interface and the
> reasoning for this requirement is basically inertia, but if you rule out
> Bugzilla because it doesn't do things the same, you're doing yourself a
> great disservice.  Many people are probably familiar with Bugzillas
> anyway, and in all likelihood they would prefer it.

What I know is that GCC works by mail and CVS.  This works fine for
discussing bugs and patches, and works fine for people reading and
replying to mail offline.  What's needed is that replies to bug reports
(sent to some special address) will get appended to them automatically -
and that some converter is in place so that replies to old GNATS reports
(with those PR numbers in the subject) will enter the new database
properly.  Being able to do other things by mail was listed as a
preference, not essential, but replies by mail getting into the database
is essential.  Mail readers and text editors tend to be better at doing
their jobs - editing text and replying to messages - than web browsers.  
Given that the mail may include patches, it should be able to extract the
plain text of messages (TABs intact, etc.) from the database - the
existing system falls down on this.

Mail works fine for GCC.  GNATSweb is what doesn't.

Grep and the existing search engine work fine for searching the list
archives for past discussions (on any list) of an issue - whether bug
reports, patches, or other discussions - having everything go to the lists
avoids the need to use multiple search mechanisms.

The gccbug script is the recommended method of submitting GCC bug reports,
allows offline submission, and will automatically include information
about host/build/target systems and configuration.  Since many people
forget to include the necessary information in bug reports if it isn't put
there for them, I think the provision of such a script (or maybe a more
intelligent program that actually asks for the source files, verifys
results if possible, ...) is necessary.  In particular it lets people
compose their reports with their normal text editor rather than being
constrained to enter things into a web browser.

One thing I forgot to note: there would also need to be a converter that
converts bug reports received from the existing deployed gccbug script
into entries for Bugzilla, as well as a new gccbug script.  Since we have
released GCC 3.0 with the current script, this converter will need to be
in place on gcc-gnats for a few years, at least.

> Bugzilla does use email for notifying people of changes, and very
> effectively.  People can choose what bugs and what changes they are
> interested in.  Then it is very simple to follow a link into the web
> interface if you want to do something.  Other bug systems are painful in
> comparison.
> 
> You could make some small hacks to Bugzilla to generate an email for
> each new bug or each change or whatever, and send it to a mailing list. 
> However, you don't really need such a facility when you can set up your
> own email preferences.

Can this be set up with a gcc-bugs or gcc-prs pseudo-user (or similar)
that will receive messages for every new bug and every change,
irrespective of any per-PR configuration anyone else does?

GNATS sends notifications in some cases, but falls down on the details -
for examples, changes of PR category and priority do not get messages sent
out.  Further, people can change existing PR text (which I don't think
should be allowed at all - only appending should be, not modifying or
deleting what was originally written) and this doesn't generate messages, 
nor an audit trail showing the previous text.

I'd like the audit trail send to the lists to be sufficient to both (a)
allow one to determine any previous version of the bug report from the
current one and (b) allow one to determine the current one from any
previous one (e.g. from a backup).  However, I don't think any automated
system is needed to apply this audit trail to a PR - the information just
needs to be there, and then can be manually applied if necessary after
restoring from a backup.

> > 3. Where files are attached in web submissions, they appear in mail as
> > MIME attachments, compressed if large, not as base64-encoded junk needing
> > manual decoding (as at present).
> 
> Bugzilla does not send attachments on emails.  Bugzillas have a tendency
> to have communities of people form around them because it is so easy to
> help out.  Hence this would result in large amounts of useless
> attachments being sent out.  I suppose it is a reasonable RFE to send
> attachments in certain situations, and I will file that, but I believe
> if you used a Bugzilla you would not consider this a major issue.

I do want to receive the attachments by mail on the lists.  I'm far more
likely to look at a bug report when it is sent (rather than months later
is at all) if everything necessary is there in front of me when reading
mail rather than needing to look in a separate program for the
information.

> > 4. All existing URLs for individual PRs get redirected to the
> > corresponding URLs in the new system.
> 
> Nothing to do with Bugzilla, I imagine.

No, it's just a web server configuration issue necessary if a change is
made.

> > 6. No login required for browsing of bugs or submitting new bugs.
> 
> Bugzilla does not require a login for searching or browsing of bugs,
> however obviously confidential bugs will not appear.  If you try to do

For GCC we decided some time ago to disallow confidential bugs.  I presume
that Bugzilla can be configured to disallow anyone from creating
confidential bugs or making bugs confidential as well?

> Bugzilla does require a login for submitting bugs.  You need to create
> an account.  I consider this a good thing.  Email addresses need to be
> verified by bug tracking systems in the same way as mailing lists (to
> prevent abuse).

How are the rules determined for what accounts have what access to change
what in the system?  Ordinary users should be able to add attachments to
their own bug reports (they complain about not being able to do it with
GNATSweb) but should not be able to delete anything (bug reports should
keep a full log - by all means mark an attachment as mistaken, but it
should still be available from the system).  Developers approved by the
Steering Committee (whether for CVS write access, or for bug tracking
system only access - we have a few of the latter, you suggest that using
Bugzilla would encourage more volunteers for this) should be able to
change bug report priorities, close bug reports, etc., but still not to
remove information.  (Conceptually, I think of a bug report as having a
history, like a file in CVS, and it shouldn't be possible to rewrite or
delete that history.  I think the ability to make most uses of "cvs admin"
was disabled for most users on sourceware to prevent people from doing so
by mistake or misunderstanding for files in CVS.)

Similarly, there should be constraints on what values of some other fields
users not approved by the SC can set - for example, they shouldn't be able
to set anything used to determine which bugs need fixing in a particular
release (priority "high", in the existing GNATS system).

I take it Bugzilla will work properly with Lynx (which more than one of us
currently use with GNATSweb)?  In some ways I'd also prefer to be able to
work with basic HTTP user authentication rather than needing to enable
cookies when manipulating bugs, but this isn't essential.

The existing GNATSweb system does not require logins to submit bugs;  
occasionally spam does get in (though probably through the gcc-gnats email
interface) but it can be changed to category "spam" and those bug reports
closed - it isn't a significant problem.  However, I only ever use the
gccbug script to submit bug reports, and have no real interest in the web
submission system - but since some people do use web submission (whether
through preferring it or through having an older version of GCC without
gccbug, or through ignoring the instructions saying gccbug is preferred),
it is necessary to consider them.

> > 7. Preferably possible to do all administration by mail (as with debbugs).
> > (If more security than debbugs is wanted, plaintext passwords (as at
> > present) or digital signatures can be used to authenticate changes.)
> 
> I have never heard anyone ask for this before, I don't think Bugzilla
> admins consider it necessary.  Administration isn't and shouldn't be a
> regular enough process to need to learn cumbersome administration
> languages.  Bugzilla has a fairly easy web administration interface.

By administration I mean such things as closing mistaken bug reports or
requesting feedback on those with insufficient information - so it should
be possible just to reply to a bug report on the list and put in the
appropriate magic to make the change as well as sending the explanation,
rather than needing another program.

It would also be desirable if CVS commits could close bug reports.  At
present, there's a limited system where the log_accum script can send the
commit message to a single PR whose number appears in the log message -
but not close the PR, and not send the message to more than one.

> You should perhaps consider moving your system elsewhere if your system
> administrators are not responsive.  But I think more to the point is
> asking why this is such a major issue here.  Corruption is generally not
> a major issue for Bugzillas, and even when it is it is rarely serious.

Are there multiple administrators on the FSF machines familiar with
Bugzilla and the underlying database?  The FSF machines and sourceware
(the current system) are the only options available (politically, if
anything the FSF machines would be better, since RMS would probably prefer
things to be hosted there).  Without necessary sysadmin support on
whichever system (the existing ones are at least familiar with GNATS, but
I don't know about any other system), changes however desirable are a
non-starter.

The problems then were kernel problems.  It's impossible to guarantee that
kernel and hardware problems won't occur.

What would need to be done to make it possible for anonymous users to
obtain a consistent copy of the whole database?  At present, the whole
GNATS database is available by anonymous rsync - and this is a good thing,
both because it encourages multiple distributed backups and because it
ought to be possible for people to get everything they need to clone the
project if necessary.  However, nothing special is done (I think) to
ensure that the result of rsyncing the database represents a consistent
database state - and with a proper database this may be more significant.

> I don't pretend we're perfect.  We have a long way to go.  If you do
> decide to move to Bugzilla you will likely lose some things you're used
> to, but I think you will gain much more.  In particular we have had

The list was based on what I've found in practice to be deficient in GNATS
and what I've observed to work for GCC (such as the mailing list (or for
some people, news gateway) - not web, not IRC - orientation).  Other
features (in Bugzilla but not GNATS) are not listed since I hadn't noticed
their absence as a significant problem (although stronger features could
probably be useful for release management - marking which releases a bug
is present in, and which release branches it should be fixed for, keeping
track of what level of review had been made of a bug by whom, keeping
track of lists of platforms on which a bug does / does not occur, ...).  
Probably some features present in GNATS (whether or not also in Bugzilla)
were not noted because they are just taken for granted.

-- 
Joseph S. Myers
jsm28@cam.ac.uk