Zach's list of requirements
Tom Lord
lord@emf.net
Mon Dec 9 09:45:00 GMT 2002
0. Must be at least as reliable and at least as portable as CVS. GCC
is a very large development effort. We can't afford to lose
contributors because their preferred platform is shut out, nor can
we afford to lose work due to bugs, and we *especially* cannot risk
a system which has not been audited for security exposures. It
would be relatively easy to give much stronger data integrity
guarantees than CVS currently manages:
The reference implementation of Arch is strictly rooted in 1003.2 plus
some gnu tools. Those gnu tools are themselves rooted in 1003.2.
MSFT platforms are, therefore, not supported by the reference
implementation - though alternative implementations are certainly
achievable (the reference implementation is pretty small).
----------------------------------------------------------------
0a. All data stored in the repository is under an end-to-end
checksum. All data transmitted over the network is independently
checksummed (yes, redundant with TCP-layer checksums). CVS does
no checksumming at all.
Quoting my other reply:
Arch is, at present, very weak in this area. It is very
straightforward (O(500) lines of code) to add these protections to the
reference implementation. Because this is a very important issue
(IMO), I would want those 500 lines of code to be backed up by
carefully crafted design documents and testing. I've been intrigued
by some of the lessons offered by the OpenCM design in this area (file
revisions in opencm are _identified_ by their unique hashes). This is
one of the items on the agenda for my 6-engineer/1-year/$1.2M plan for
arch 1.0.
----------------------------------------------------------------
0b. Anonymous repository access is done under a user ID that has only
OS-level read privileges on the repository's files. This cannot
be done with (unpatched) CVS.
The reference implementation of arch has this property.
----------------------------------------------------------------
0c. Remote write operations on the repository intrinsically require
the use of a protocol which makes strong cryptographic integrity
and authority guarantees. CVS can be set up like this, but it's
not built into the design.
I'm not sure what "intrinsically require" means or why it is
desirable.
As with CVS, `arch' can be set up like this, but does not _require_
it.
----------------------------------------------------------------
0d. The data stored in the repository cannot be modified by
unprivileged local users except by going through the version
control system. Presently I could take 'vi' to one of the ,v
files in /cvs/gcc and break it thoroughly, or sneak something into
the file content, and leave no trace.
If you can do that with `vi', then you are not "unprivileged".
Hey, if you have root, you can write to the raw device :-)
----------------------------------------------------------------
1. Must be at least as fast as CVS for all operations, and should be
substantially faster for all operations where CVS uses a braindead
algorithm. I would venture to guess that everyone's #1 complaint
about CVS is the amount of time we waste waiting for it to complete
this or that request. To be more specific:
Ok, the arch architecture is in _excellent_ shape here.
The reference implementation is in _fair to middling_ shape here.
Once gain: 6 hackers, one year, $1.2M.
----------------------------------------------------------------
1a. Efficient network protocol. Specifically, a network protocol that,
for *all* operations, transmits a volume of data proportional --
with a small constant! -- to the size of the diff involved, *not*
the total size of all the files touched by the diff involved, as
CVS does.
arch ships the diff, with syntactic overhead, but compressed.
----------------------------------------------------------------
1b. Efficient tags and branches. It should be possible to create
either by creating *one* metadata record, rather than touching
every single file in the repository.
arch has this property.
----------------------------------------------------------------
1c. Efficient delta storage algorithm, such that checking in a change
on the tip of a branch is not orders of magnitude slower than
checking in a change on the tip of the trunk. There are several
sane ways to do this.
arch has this propery.
----------------------------------------------------------------
1d. Efficient method for extracting a logical change after the fact,
no matter how many files it touched. (Currently the easiest way
to do this is: hunt through the gcc-cvs archive until you find the
message describing the checkin you care about, then use wget on
all of the per-file diff URLs in the list and glue them all
together. Slow, painful, doesn't always work.)
arch is efficient in this regard.
----------------------------------------------------------------
2. Should support this laundry list of features, none of which is
known to CVS. Most of them would be useful independent of the
others, though there's not much point to 2b without 2a, nor 2e
without 2d.
Ok, let's see:
----------------------------------------------------------------
2a. Atomic application of a logical change that touches many files,
possibly not all in the same directory. (This is commonly known as
a "change set".) One checkin log per change set is adequate.
arch has this property.
----------------------------------------------------------------
2b. Ability to back out an entire change set just as atomically as it
went in.
The reference implementation of arch does _not_ have this property.
It is straightforwardly implemented (O(100) lines of code) -- but the
obstacle is to choose the right way to provide this functionality from
among the possibilities.
----------------------------------------------------------------
2c. Ability to rename a file, including the ability for a file to have
different names on different branches.
arch has this property.
----------------------------------------------------------------
2d. Automatically remember that a merge occurred from branch A to
branch B; later, when a second merge occurs from A to B, don't
apply those changes again.
arch has this property. There are seven cases and the reference
implementation has a bug (it covers six of the seven cases).
----------------------------------------------------------------
2e. Understand the notion of a single-delta merge, either applying
just one change from branch A to branch B, or removing just one
change formerly on branch A ("subtractive merge").
I'm not _certain_ I understand what you are asking for.
arch has `replay --exact', which I suspect is most of what you are
looking for. If that's right, then we need to add `replay --exact
--reverse' to get 100% of what you are asking for (and that is quite
straightforward).
----------------------------------------------------------------
2f. Perform conflict resolution by automatic formation of
microbranches.
Intriguing, but I don't completely understand what you want.
Given that you do a lot of merging, I'd like to talk with you at
greater length about conflict resolution.
(Branches in arch are easy to create and use -- but you seem to be
getting at something deeper here.)
----------------------------------------------------------------
3. Should allow a user without commit privileges to generate a change
set, making arbitrary changes to the repository (none of this "you
can edit files and generate diffs but you can't add or delete
files" nonsense), which can be applied by a user who does have
commit privileges, and when the original author does an update
he/she doesn't get spurious conflicts.
arch has this property (as far as I can understand your description).
In arch, this is accomplished via the unprivileged contributor forming
a branch in their own repository, and the privileged contributor
working againt that branch.
----------------------------------------------------------------
4. The repository's on-disk data should be stored in a highly compact
format, to the maximum extent possible and consonant with being
fast. Being fast is much more important; however, GCC's CVS
repository is ~800MB in size and compresses down to ~100MB. You
can do interesting things (like keep a copy of the entire
repository on every developer's personal hard disk, as Bitkeeper
does) with a 100MB repository that are not so practical when it's
closer to a gigabyte.
arch has a canonical, minimal repository format that is quite compact:
compressed base revisions plus compressed changesets.
arch also has a very flexible ability to take advantage of modern on
very-near-future disk sizes, smoothly trading space for time.
----------------------------------------------------------------
5. Should have the ability to generate ChangeLog files automagically
from the checkin comments. (When merging to basic-improvements I
normally spend more time fixing up the ChangeLogs than anything
else. Except maybe waiting for 'cvs tag' and 'cvs update -j...'.)
arch has this feature.
More information about the Gcc
mailing list