new batch of replies (B)

[Tom Lord]

Replies in this message:

	Joseph S. Myers: Notes on Our Change Acceptance Procedures
	Joseph S. Myers: Comments About Mainline Server Requirements 


================================================================

* Joseph S. Myers: Notes on Our Change Acceptance Procedures

Joseph's account is very interesting and useful.  In reply, I have
written down a proposal for a particular kind of process automation
that I think can help quite a bit, and that is a natural fit with
arch.  

arch was designed with the kind of process automation described here
kept firmly in mind.  There are many little variations possible, I've
tried to keep it simple by describing just a few.

The goal of the automation described here is to much more effectively
distribute the work currently performed by maintainers, lightening the
load on the maintainer team.


       There are about 100 people applying patches to the mainline
       (half maintainers of some of the code who can apply some
       patches without review, half needing review for all nonobvious
       patches).  Having the release manager manually handle the
       patches from all 100 people is not a sensible scalable solution
       for GCC; the expectation is that anyone producing a reasonable
       number of good patches will get write access which reduces the
       reviewers' effort (to needing only to review the patch, not
       apply it) and means that the version control logs clearly show
       which user was responsible for a patch by who checked it in
       (the case of someone else, named in the log message, being
       responsible, being the exceptional case).  Note that the 50 or
       so maintainers all do some patch review; it's only at a late
       stage on the actual release branches that the review is
       concentrated in the release manager.

Arch handles multiple writers to a single repository quite well --
better than CVS in terms of both lock-handling and options for
updating.

At the same time, distributed repositories can make the lives of these
50 maintainers easier in at least two ways:

	1) By giving the non-privileged developers their own (remote)
           branches of mainline, and thus the ability to `star-merge'
	   and so forth as their work proceeds in parallel with 
	   mainline work.   This should help (if people use it) avoid
	   the problem people have described of reintroducing old bugs
	   from a long-lived branch back onto mainline.

	2) By giving non-privileged developers their own (remote)
           branches of mainline on which to "stage" their pending
           patches during review.  Those staging areas will be in a
           star topology relationship to mainline, so it will be
           easier to raise the bar: to reasonably ask that owners
           of pending patches themselves resolve any conflicts that
	   are created between initial submission and final patch
           acceptance.

Regarding (2), what I have in mind is a two-phase, semi-automated
review/acceptance procedure.

In the first phase, the "heavy review phase", a changeset is submitted
for review.  It goes into the first-phase pool until a maintainer has
a chance to review it.  During that period of time, mainline may
evolve, introducing conflicts.  The reviewer can test it (if that's
desired) against its true base revision, temporarily ignoring the
intervening mainline changes.  Here: better bug-tracking automation is
also desirable, so that a maintainer can quickly check for regressions
relative to the appropriate base revision rather than against HEAD.

The second phase begins if the change is accepted.  Its merge with
mainline can be _scheduled_: it can be assigned a window of time, and
an order relative to other accepted patches, and the author of the
change notified.

At any given time, then, there is a short queue of pending, accepted
(2nd phase) patches.  Here's the rule: if you are the author of an
accepted patch, then when your window comes up, you have to have
prepared a new version of the change that is fully up-to-date with
HEAD at that time.  Because you were able to see what preceded you in
the queue well before your window, you will have had plenty of time to
merge with HEAD, plus all those other 2nd phase patches.  You can even
do this merging work _before_ the intervening 2nd phase patches reach
HEAD.

Then comes the "light review phase".  In this phase, a maintainer
looks at what new changes you've made, and confirms that it's just
merging work.  The maintainer can run automatic tests again, this time
checking for regressions against HEAD.   The goal here is that the
cost in maintainer time of this "light review" is distinctly less than
the cost of the maintainer resolving conflicts himself -- the cost of
resolving those "last minute conflicts" has been passed back to the
(non-maintainer) changeset author -- the work load has been more
effectively distributed, and the burden on maintainers lightened.

A problem here occurs if a changeset assigned a position in the 2nd
phase queue fails to make it to HEAD.  Work eagerly performed by other
non-maintainer authors, whose changes follow the failed change in the
queue, has to be revisited, and that is disruptive.  I hope that
social factors can be used to minimize such occurrences, but
technology can help here, as well:

For example: some queue orderings of 2nd phase changesets may be
better than others.  Clusters of non-overlapping changes can be put
ahead of all later changes that may conflict with them.  Doing that
may minimize the impact of the occasional change that fails to make
it through the 2nd phase.  Additionally, ordering the queue this way
maximizes the time-prior-to-merge window for non-maintainer authors
whose changes are known to have conflicts that need to be fixed before
the window.

Arch can shine as a foundation for the infrastructure for submission
queues like this in at least two ways worth mentioning: (1) Since it
is a changeset based system, it is quite natural and straightforward
to write "patchutils"-type tools for tasks such as sorting the
2nd-phase queue.  Second, distributed repositories and excellent
merging support are essential tools for the non-maintainer authors in
this scenario.


	The Linux model of one person controlling everything going into the
	mainline is exceptional; 

It certainly is.  Automated queues as described above, operated by
multiple maintainers, is one of process automations I've had in the
back of my mind for some time now.  It's nice to read your description
of the current process in this context.  (Additionally, I think a
similar system of queues would help Linus.)

Here's some other variations, since I suspect that maintainers may
worry about having to put their _own_ patches through such a queue
rather than directly applying them.  One idea is a duty-cycle:
introduce gaps in the merge-window schedule, during which maintainers
can make direct modifications.  Another is simply an override: a
maintainer with an urgent change can apply it directly, then push back
all windows on the 2nd phase queue to give patch authors a chance to
react.




================================================================

* Joseph S. Myers: Comments About Mainline Server Requirements 


       For GCC there clearly needs to be some server that has the
       mainline of development we advertise on our web pages for
       users, from which release branches are made, which has some
       vague notions of the machine being securely maintained, having
       adequate bandwidth, having some backup procedure, having
       maintainers for the server keeping it up reliably, having a
       reasonable expectation that the development lines in there will
       still be available in 20 years' time when current developers
       have lost interest.  (gcc.gnu.org presents a remarkably good
       impression of this to the outside world, considering how it
       operates purely by volunteer effort.)

I would add: that central server should also collect mirrors of
important remote repositories -- so that their history is also
archived for a long time, and cataloged.

One particular strength of `arch' in this area is the simplicity of
the server.  Being, in essense, passive file systems, repositories are
cheap to create and maintain.


      There may be many other servers - private and public - but some
      server provides the line of development that gets branched into
      new releases, and inevitably multiple people may write to that
      line.  (I'm also presuming - see
      <http://gcc.gnu.org/ml/gcc/2002-12/msg00436.html> - that all the
      developments in any third party repository that get discussed on
      the lists should be mirrored into this main one to give some
      hope of long term survival and availability.  In developing GCC
      with list archives and version control we are simultaneously
      acting as curators of the history of GCC development, which
      means attempting to preserve that history for posterity (a
      period beyond the involvement of any one individual).)

Very true.  I like to borrow terminology from librarians and say that
revision control has duties both for "archival" and for "cataloging".
By wanting to make sure discussed branches are mirrored, you're
talking about another important issue: "accessibility".

When I've worked with a developer who had his own, remote arch
repository, I did that work via a mirror.   Whenever he had new
changes to consider for merging, I updated my local mirror of his
repository (giving me, then, two local repositories: mine and his) and
then worked against those.   Thus, I (still) have a complete record of
his work, and enjoyed better performance than had I had to access his
branches over the net.