GCC GIT mirror
There is a read-only git mirror of the gcc svn repository available at:
git://gcc.gnu.org/git/gcc.git http://gcc.gnu.org/git/gcc.git git+ssh://gcc.gnu.org/git/gcc.git
It can be browsed at repo.or.cz and at gcc.gnu.org/git.
Getting started - Read-Only
To work with git a local clone of the public repository is required. This is close to a svn checkout, but a clone copies the repository with the complete history. Therefore commands like "git log", "git status", "git diff", "git blame" can be executed without an internet connection.
Create the clone (git <= 1.6.4 warning)
> git clone git://gcc.gnu.org/git/gcc.git
Now the repository is available in the directory "gcc". Try some commands
> cd gcc ~/gcc> git status # On branch trunk nothing to commit (working directory clean)
To fetch changes from the official svn-mirror repository into trunk
~/gcc> git pull
To extract patches for submission
# Rebase first so that the patches are relative to SVN head
~/gcc> git pull --rebase
# Show all changes on this branch that are not on the upstream branch, from earliest to latest
~/gcc> git log -p --reverse @{u}..It can be useful to pull with or without --rebase, depending on the circumstance:
- git pull without --rebase gives a nice summary of what files have been updated, and will pull into a dirty working tree, and preserves the timestamps on files that aren't affected by the pull, but does a merge if I have any local commits, which may mean doing conflict resolution twice (now and later when you rebase);
- git pull --rebase rebases my local commits on top of the current SVN head in preparation for a checkin.
On branches
Our clone already contains a set of important gcc branches.
List the available branches:
~/gcc> git branch -a * trunk remotes/origin/HEAD -> origin/trunk remotes/origin/fortran-dev remotes/origin/gc-improv remotes/origin/gcc-4_0-branch remotes/origin/gcc-4_1-branch remotes/origin/gcc-4_2-branch remotes/origin/gcc-4_3-branch remotes/origin/gcc-4_4-branch remotes/origin/gccgo remotes/origin/graphite remotes/origin/ifunc remotes/origin/lw-ipo remotes/origin/master remotes/origin/melt-branch remotes/origin/microblaze remotes/origin/profile-stdlib remotes/origin/reload-v2a remotes/origin/split remotes/origin/spu-4_5-branch remotes/origin/transactional-memory remotes/origin/trunk
To work with one of these branches we need a local branch we can edit.
~/gcc> git checkout -b gccgo origin/gccgo Checking out files: 100% (2204/2204), done. Branch gccgo set up to track remote branch gccgo from origin.
Updates are the same as for trunk.
To work with a branch that is not in the set above, you can add it with
git config --add remote.origin.fetch refs/remotes/mybranch:refs/remotes/origin/mybranch git fetch
Note that this doesn't work with subdirectory branches like redhat/gcc-4_4-branch; see below for how to deal with those branches.
~/gcc> git pull
Advanced Usage
Contents
The above works fine for browsing the repository and simple hacking, but has various issues. Here's what seems to me to be the optimal way of using git-svn with the GCC repository. Please email (jason) or grosser@ if you have any questions/issues with these instructions.
Commit upstream (git-svn)
The repository we create above is read only. To commit changes upstream, patches still have to be extracted and committed using svn. However, there is a way to commit changes directly out of git: git-svn
Commit on trunk
We start with the read only repository created above.
Add git-svn support to be able to commit
# Use this exact SVN url for gcc.git and git-svn to work together. # Use the correct user name below, or there may be complaints # about ``remove the -q option from 'ssh'' ~/gcc> git svn init -Ttrunk --prefix=origin/ svn+ssh://username@gcc.gnu.org/svn/gcc # Update index, fetch changes and rebase trunk onto the SVN trunk ~/gcc> git svn rebase # Or, to update all the remote branches: ~/gcc> git svn fetch # Ignore the same files that SVN does. ~/gcc> git svn show-ignore >> .git/info/exclude
Disable non-fast-forward pulls to prevent git pull from losing changes that are in SVN but not the git mirror
git config remote.origin.fetch 'refs/heads/*:refs/remotes/origin/*'
Integrate the latest upstream changes in the local repository
~/gcc> git pull
See the changes we are going to commit
~/gcc> git log -p @{u}..
OR
~/gcc> git diff @{u}Commit our local changes into the official gcc subversion repository
# rebase first so our changes are relative to the current SVN head ~/gcc> git svn rebase ~/gcc> git svn dcommit
Working with branches
A single branch
To work on a branch available in origin/* just check it out:
~/gcc> git checkout -b gccgo origin/gccgo
This means "create a new local branch gccgo which tracks the remote branch origin/gccgo". Then you can switch between branches with
~/gcc> git checkout trunk ~/gcc> git checkout gccgo
If you want to use git svn with this branch, you need to set that up as well:
~/gcc> git config --add svn-remote.svn.fetch branches/gccgo:refs/remotes/origin/gccgo
"git svn dcommit" commits automatically to the svn branch that the local branch you are working on was created from.
Adding a branch to origin/*
If you want to add a new branch to origin/*, and you have shell access to gcc.gnu.org, then you can add a new reference in /git/gcc.git/.git with
$ cd /git/gcc.git/.git $ git symbolic-ref refs/heads/branch refs/remotes/branch
Fetching all branches
If you want a complete copy of the git mirror, you can tell git to fetch the branches under remotes/ as well:
~/gcc> git config --add remote.origin.fetch refs/remotes/*:refs/remotes/origin/* ~/gcc> git remote update
Fetching all branches increases the size of the .git directory by about 30% over just trunk and release branches. At least it does if you fetch everything at the beginning; a full fetch after a smaller fetch/clone may be larger because the pack you end up with isn't as well compressed.
The various release branches
Alternately, you might want to only fetch all the release branches:
~/gcc> for f in 2_95 3_0 3_1 3_2 3_3 3_4 ; do git config --add remote.origin.fetch refs/remotes/gcc-$f-branch:refs/remotes/origin/gcc-$f-branch; done
~/gcc> for f in 1_00 1_1; do git config --add remote.origin.fetch refs/remotes/egcs_${f}_branch:refs/remotes/origin/egcs_${f}_branch; done
~/gcc> git remote update
Subdirectory branches
If you want a branch that lives in a subdirectory of branches, such as redhat/gcc-4_4-branch, things are a bit more complicated due to a bug/shortcoming in git svn: it thinks the redhat directory is a branch by itself, rather than a directory containing multiple branches. There are multiple ways to address this. One way is to ignore the git mirror and use git svn fetch to fetch the branch from SVN directly:
~/gcc> git config --add svn-remote.svn.fetch branches/redhat/gcc-4_4-branch:refs/remotes/origin/redhat/gcc-4_4-branch ~/gcc> git svn fetch
But this can take hours for the initial fetch. Alternately, you can get the whole redhat directory from the git mirror:
~/gcc> git config --add remote.origin.fetch refs/remotes/redhat:refs/remotes/origin/redhat ~/gcc> git fetch ~/gcc> git branch redhat origin/redhat
With the second option, a checkout has all the redhat branches in subdirectories, rather than being able to check out a single branch at toplevel. If you only want one, you can use the git 1.7 sparse checkout function to limit what gets checked out:
~/gcc> git config core.sparseCheckout true ~/gcc> echo "gcc-4_4-branch/" > .git/info/sparse-checkout ~/gcc> git checkout redhat
Now only the files that match the patterns in .git/info/sparse-checkout will actually be present in the working tree. Remember to remove .git/info/sparse-checkout if you want to switch back to a normal branch. Or just do this in a separate working directory, as described below.
A third option works like the first without the slow initial fetch, but requires some git magic to rewrite the branch:
# Pull a local copy of the whole redhat directory from git ~/gcc> git fetch origin refs/remotes/redhat:redhat-gcc-4_4-branch # Set up git-svn to track the subdirectory properly ~/gcc> git config --add svn-remote.svn.fetch branches/redhat/gcc-4_4-branch:refs/remotes/origin/redhat/gcc-4_4-branch # Rewrite redhat-gcc-4_4-branch to look like it was imported from the subdirectory ~/gcc> git filter-branch --subdirectory-filter gcc-4_4-branch --msg-filter 'sed -e "s,branches/redhat@,branches/redhat/gcc-4_4-branch@,"' redhat-gcc-4_4-branch # Move the rewritten branch to where git-svn expects it to be ~/gcc> mkdir -p .git/refs/remotes/origin/redhat ~/gcc> mv .git/refs/heads/redhat-gcc-4_4-branch .git/refs/remotes/origin/redhat/gcc-4_4-branch # And now we can check out a local copy to work in. ~/gcc> git checkout -b redhat/gcc-4_4-branch origin/redhat/gcc-4_4-branch
A branch created this way won't be updated by fetch from the git mirror unless you repeat this process, but git-svn will update it just fine.
Working on multiple branches at once
To make a separate working directory for hacking on another branch at the same time, you can just create a new working tree. First make sure the branch is in your git repository, as above. Then make a local tracking branch, if you haven't already done that with checkout -b:
~/gcc> git branch foo-branch origin/foo-branch
Then use the git-new-workdir script from the git sources to create a working directory for that branch:
> git clone git://git.kernel.org/pub/scm/git/git.git > ./git/contrib/workdir/git-new-workdir ~/gcc gcc-foo foo-branch > cd gcc-foo
Now you have a directory gcc-foo which contains a checkout of your foo-branch. This directory uses the same git repository as your main directory, it just has a different branch checked out.
Note that using git cherry-pick to copy changes between SVN branches brings along svn metadata, but git-svn is clever enough not to be confused by this.
Merging SVN branches
To merge trunk into a development branch:
$ git checkout my_branch $ git svn rebase $ git merge --squash origin/trunk [ ... Fix merge conflicts ... ] $ git commit -m'Merge from blah blah'
NOTE: A normal (non-squashed) git merge works very badly with rebase, including git svn rebase; the rebase replaces the merge commit with a single copied commit for each of the commits brought in by the merge. If you then use git svn dcommit, git-svn will commit them separately, causing a lot of SVN traffic. Additionally, if the commits have references to bugzilla PRs, they will be used to spam bugzilla, which will eventually force you to send a message along the lines of http://gcc.gnu.org/ml/gcc/2011-02/msg00047.html. See also http://gcc.gnu.org/ml/gcc/2011-02/msg00096.html and other messages.
You have been warned. Use --squash!
Recent versions of git have support for svn:mergeinfo, but it sounds like this is still fragile, so using --squash seems like a more reliable solution.
Once that's done, your branch should look like this:
$ git status # On branch my_branch # Your branch is ahead of 'origin/my_branch' by 1 commit. # nothing to commit (working directory clean) $ git svn dcommit --dry-run Committing to svn+ssh://gcc.gnu.org/svn/gcc/branches/pph ... diff-tree e63f... e63f... $ git log -1 commit e63f... Author: My Name <my@address> Date: Thu Feb 3 12:55:55 2011 -0500 Merge from blah blah
Test the merge and then commit with
$ git svn dcommit
Git-only branches
To allow others to view your changes online you can push your local branches to the Git mirror. To set up your git repository for this, do
$ git config remote.origin.pushurl ssh://gcc.gnu.org/git/gcc.git $ git config remote.origin.push mybranch:myname/mybranch
You are now set up to push your local branch "mybranch" to "myname/mybranch" on gcc.gnu.org. Whenever you want to do this, just
$ git push
If you want to push another local branch, you can
$ git push origin otherbranch:myname/otherbranch
Please keep personal branches in a subdirectory based on your gcc.gnu.org username. Non-subdirectory branches can be used for collaborative development.
The git repository is configured to deny branch deletion requests; if you really need to delete a branch, you can email jason about it.
Usage hints for git
git-merge-changelog
Before long you'll get frustrated with git's handling of ChangeLog merges, which is just as bad as SVN's. But there's a fix for that!
git clone git://git.savannah.gnu.org/gnulib.git cd gnulib # alternately, install gnulib from a package manager with something like # 'sudo apt-get install gnulib' and just run /usr/bin/gnulib-tool ./gnulib-tool --create-testdir --dir=/tmp/testdir123 git-merge-changelog cd /tmp/testdir123 ./configure make make install git config --global merge.merge-changelog.name "GNU-style ChangeLog merge driver" git config --global merge.merge-changelog.driver "/usr/local/bin/git-merge-changelog %O %A %B" echo "ChangeLog merge=merge-changelog" >> ~/gcc-git/.git/info/attributes
NOTE: git-merge-changelog used to be extremely slow for cherry-picking changes from trunk to release branches, but this was fixed with a change on 2009-07-02.
Update bash function
Here's my generic "update this source tree" bash function. When I want to rebase, I do that manually either with 'git svn rebase' or 'git rebase -i trunk'
up() { if [ -d CVS ]; then cvs -q update "$@";
elif [ -d .svn ]; then svn up "$@";
elif gitd=`git rev-parse --git-dir 2>/dev/null`; then
if [ -f $gitd/objects/info/alternates ]; then
for d in `cat $gitd/objects/info/alternates`; do
(cd $d; git fetch)
done
fi
if [ -d $gitd/patches ] && `stg top 2>/dev/null`; then
stg pull
else
git pull
fi
fi; }
rebase -i: Squashing multiple local commits into a single SVN commit
git rebase -i is a very useful tool for organizing local changes for svn dcommit.
If you have been making changes to your local tree, addressing review feedback, making minor fixes, etc. your local tree will have many commits. So, when you finally push the patch to SVN with 'git svn dcommit', you will be pushing all the local commits separately, which may not be what you want; a good rule of thumb is to have one commit for each patch sent to gcc-patches. To commit all your changes as a single SVN transaction, you need to squash all your local patches together. To do this:
$ git svn fetch -p
$ git rebase -i @{u}The -i switch causes git rebase to ask you which patches you want to pick or squash. Reorganize the lines so that all the related changes are grouped together, and within a group of changes that you want to combine, replace 'pick' with 'squash' or 's' (or 'fixup' or 'f' if you don't need to adjust the commit message) on all lines except the first one. Then save the file and let 'git rebase' finish.
After this, the change(s) in your local git branch should look more like the patch(es) you want to send to gcc-patches, which you can then extract separately with git log -p for emailing to gcc-patches; see the 'git lgp' alias in the next section. Now, when you run 'git svn dcommit', you will push these patches to SVN rather than your local edit history.
While you are working, if you already know that you want your new commit to be squashed with a previous commit, you can use commit --squash or --fixup to specify that; rebase -i --autosquash will then reorganize the patches automatically.
To apply some but not all of your current commits to SVN, use git rebase -i to move the ones you want to apply to the top of the list, select "edit" for the last one you want to apply, and do git svn dcommit before git rebase --continue.
Removing accidentally committed hunks from a commit
If you look at your local commit log and notice that there's a hunk in one of your commits that doesn't belong with it, it's reasonably easy to edit it out. If the commit is not HEAD, you can use rebase -i 'edit' to go to that commit for editing. Once the commit in question is at HEAD, I do
# Move HEAD to the previous commit, leaving the commit I'm editing in the index. git reset --soft HEAD^ # Go through all the hunks interactively to choose what to remove from the index git reset -p # Re-commit everything that's left in the index, using the same commit log as before git commit -C ORIG_HEAD
At this point the commit is edited and the hunks you removed are only in the working tree. Now you can give them their own commit or discard them.
Useful aliases
Here are some aliases I'm finding useful (to add to your ~/.gitconfig). Note that these only work with the advanced setup due to differences in the branch.$BRANCH.merge config.
[alias]
st = status
ci = commit
br = branch
co = checkout
cp = cherry-pick
sr = svn rebase
sci = svn dcommit
dir = rev-parse --git-dir
# The current branch.
cbr = "!f(){ expr $( ( git symbolic-ref -q HEAD || cat $(git dir)/rebase-merge/head-name ) 2>/dev/null ) : 'refs/heads/\\(.*\\)'; }; f"
# The branch being tracked by the current branch.
track = "!f() { if p=$(git rev-parse --symbolic-full-name '@{u}' 2>/dev/null); then echo origin/${p##*/}; else git svn info|sed -n 's,^URL.*gcc/\\(branches/\\)\\?\\(.*\\),origin/\\2,p'; fi; }; f"
# Show all the local commits on this branch.
lg = "!git log -p `git track`.."
# Write all the local commits to ~/patch, filtering out modifications to ChangeLog files
lgp = "!git log -p `git track`.. | filterdiff -x '*/ChangeLog' | sed -e '/^diff.*ChangeLog/{N;d}' > ~/patch"
# Show all the local changes on this branch as one big diff.
df = "!git diff $(git merge-base $(git track) HEAD)"
dfc = diff --cached
# Reorganize the local commits on this branch.
rb = "!git rebase -i `git track`"
rc = rebase --continue
# 'git rmerge mybranch' to reintegrate a temporary branch onto the top of the current branch
rmerge = "!f(){ cur=`git cbr`; git rebase $cur $1; git rebase $1 $cur; }; f"
# Squash everything since the specified commit (or HEAD^) into that commit
sq = "!f(){ git reset --soft ${@:-HEAD^} && git commit --amend -C HEAD; }; f"
# Pop everything since the specified commit, but leave it in the index
rs = "!f(){ git reset --soft ${@:-HEAD^}; }; f"
# Choose patch hunks to remove from the index (e.g. after doing git rs); after removing some hunks I then do a commit -C ORIG_HEAD
rp = reset -p
# Show the commit corresponding to an SVN revision
ssh = "!f(){ git show $(git svn find-rev r$1); }; f"
Pushing to a testing box
I tend to do development on my laptop, and then send patches off to a hefty compile server to do full regression testing. Before git, I did this by rsyncing diffs, but git simplifies the process. First, add to your gcc-git/.git/config:
[remote "testbox"]
url = testbox.foo.bar:gcc-gitThen to push your current changes to the testbox, just "git push testbox +trunk". Note that you'll then need to "git co -f" or "git reset --hard" on the testbox to update the working copy from the git repository. This could be another alias:
tup = "!f(){ cur=`git cbr`; git push $1 +$cur && ssh $1 \"cd src/$cur; git co -f\"; }; f"So then you would write "git tup testbox" (assuming that "ssh testbox" also works). The alias could then proceed to run tests directly, but that is left as an exercise for the reader (as my setup involves a bunch of scripts).
Emacs
GNU Emacs 23 has decent git support in the vc package. The main attraction is C-x v g (vc-annotate), a handy interface to git blame that lets you step back to the revision before the line you're looking at with the 'a' key. It still needs some work; it hasn't been converted to run asynchronously, and the 'D' command doesn't seem to work properly, but it's still handy.
For editing git-related temporary files in emacs, I set my EDITOR environment variable to "emacsclient -t" and added the following to my .emacs:
(defun maybe-truncate ()
(interactive)
(cond ((string-match "/git-rebase-todo\\>" buffer-file-name)
(toggle-truncate-lines))))
(add-hook 'find-file-hook 'maybe-truncate)
Old Information
Below here is information that no longer describes my use of git, but that other people might find interesting.
Stacked Git
I used to find Stacked Git useful for hacking on GCC, since it makes it easy to pop off patches that I'm still working on so I can check the one I've just finished testing into SVN. But lately I've stopped using it and just use git branches directly.
With stg, if I want to remove a commit from my tree temporarily and come back to it later, I do
stg uncommit mypatchname [myotherpatchname...] stg pop -a
and then when I want to work on it again,
stg push mypatchname [myotherpatchname...] stg commit
This way stg only knows about unapplied patches, so it won't get confused when I do a 'git pull'.
And I periodically look at the list of suspended patches with
stg series
Making Patches
I currently make patches for gcc-patches using my 'git lgp' alias, below.
However, git always produces unified diffs, while the GCC project policy is to prefer context diffs. I don't bother with this, but if you really want to make a context diff, you can create a wrapper script somewhere:
cat > ~/bin/git_diff_wrapper <<EOF #! /bin/sh exec diff -p -L "$1" "$2" "$5" | cat EOF chmod +x ~/bin/git_diff_wrapper
and when you want to make a context diff for submitting a patch, do
GIT_EXTERNAL_DIFF=git_diff_wrapper git diff trunk
Further Reading
An introduction to git-svn for Subversion/SVK users and deserters
Using git without feeling stupid (part 1), by Paolo Bonzini