GCC GIT mirror

There is a read-only git mirror of the gcc svn repository available at:

http://gcc.gnu.org/git/gcc.git
git://gcc.gnu.org/git/gcc.git
git+ssh://gcc.gnu.org/git/gcc.git

It can be browsed at repo.or.cz.

Getting started - Read-Only

To work with git a local clone of the public repository is required. This is close to a svn checkout, however a clone copies the repository with the complete history. Therefore commands like "git log", "git status", "git diff", "git blame" can be executed very fast and without internet connect.

Create the clone (git <= 1.6.4 warning)

> git clone git://gcc.gnu.org/git/gcc.git

Now the repository is available in the directory "gcc". Try some commands

> cd gcc
~/gcc> git status
# On branch trunk
nothing to commit (working directory clean)

To fetch changes from the official svn-mirror repository into trunk

~/gcc> git pull

To extract patches for submission

# Rebase first so that the patches are relative to SVN head
~/gcc> git pull --rebase
# Show all changes on this branch that are not on the upstream branch, from earliest to latest
~/gcc> git log -p --reverse @{u}..

It can be useful to pull with or without --rebase, depending on the circumstance:

On branches

Our repository already contains a set of important gcc branches

List the available branches

~/gcc> git branch -a
* trunk
  remotes/origin/HEAD -> origin/trunk
  remotes/origin/fortran-dev
  remotes/origin/gc-improv
  remotes/origin/gcc-4_0-branch
  remotes/origin/gcc-4_1-branch
  remotes/origin/gcc-4_2-branch
  remotes/origin/gcc-4_3-branch
  remotes/origin/gcc-4_4-branch
  remotes/origin/gccgo
  remotes/origin/graphite
  remotes/origin/ifunc
  remotes/origin/lw-ipo
  remotes/origin/master
  remotes/origin/melt-branch
  remotes/origin/microblaze
  remotes/origin/profile-stdlib
  remotes/origin/reload-v2a
  remotes/origin/split
  remotes/origin/spu-4_5-branch
  remotes/origin/transactional-memory
  remotes/origin/trunk

To work with one of these branches we need a local branch we can edit.

~/gcc> git checkout -b gccgo origin/gccgo
Checking out files: 100% (2204/2204), done.
Branch gccgo set up to track remote branch gccgo from origin.

Updates are the same as for trunk.

To work with a branch that is not in the set above, you can add it with

git config --add remote.origin.fetch refs/remotes/mybranch:refs/remotes/origin/mybranch
git fetch

Note that this doesn't work with subdirectory branches like redhat/gcc-4_4-branch; see below for how to deal with those branches.

~/gcc> git pull

Advanced Usage

The above works fine for browsing the repository and simple hacking, but has various issues. Here's what seems to me to be the optimal way of using git-svn with the GCC repository. Please email (jason) or grosser@ if you have any questions/issues with these instructions.

Commit upstream (git-svn)

The repository we create above is read only. To commit changes upstream patches still have to be extracted and committed using svn. However there is a way to commit changes directly out of git: git-svn

Commit on trunk

We start based with the read only repository create above.

Add git-svn support to be able to commit

# Use this exact SVN url for gcc.git and git-svn to work together.
# Adding an SSH username to the url is ok: svn+ssh://username@gcc.gnu.org/svn/gcc
~/gcc> git svn init -Ttrunk --prefix=origin/ svn+ssh://gcc.gnu.org/svn/gcc

# Update index, fetch changes and rebase trunk onto the SVN trunk
~/gcc> git svn rebase

# Or, to update all the remote branches:
~/gcc> git svn fetch

# Ignore the same files that SVN does.
~/gcc> git svn show-ignore >> .git/info/exclude

Disable non-fast-forward pulls to prevent git pull from losing changes that are in SVN but not the git mirror

git config remote.origin.fetch 'refs/heads/*:refs/remotes/origin/*'

Integrate the latest upstream changes in the local repository

~/gcc> git pull

See the changes we are going to commit

~/gcc> git log -p @{u}..
OR
~/gcc> git diff @{u}

Commit our local changes into the official gcc subversion repository

# rebase first so our changes are relative to the current SVN head
~/gcc> git svn rebase
~/gcc> git svn dcommit

Working with branches

A single branch

To work on a branch available in origin/* just check it out:

~/gcc> git checkout -b gccgo origin/gccgo

then you can switch between branches with

~/gcc> git checkout trunk
~/gcc> git checkout gccgo

If you want to use git svn with this branch, you need to set that up:

~/gcc> git config --add svn-remote.svn.fetch branches/gccgo:refs/remotes/origin/gccgo

"git svn dcommit" commits automatically to the svn branch that the local branch you are working on was created from.

All branches

~/gcc> git config --add remote.origin.fetch refs/remotes/*:refs/remotes/origin/*
~/gcc> git remote update

Pulling all branches increases the size of the .git directory by about 30% over just trunk and release branches. At least it does if you fetch everything at the beginning; a full fetch after a smaller fetch/clone may be larger because the pack you end up with isn't as well compressed.

The various release branches

~/gcc> for f in 2_95 3_0 3_1 3_2 3_3 3_4 ; do git config --add remote.origin.fetch refs/remotes/gcc-$f-branch:refs/remotes/origin/gcc-$f-branch; done
~/gcc> for f in 1_00 1_1; do git config --add remote.origin.fetch refs/remotes/egcs_${f}_branch:refs/remotes/origin/egcs_${f}_branch; done
~/gcc> git remote update

Subdirectory branches

If you want a branch that lives in a subdirectory of branches, such as redhat/gcc-4_4-branch, things are a bit more complicated due to a bug/shortcoming in git svn. You have two choices. Either use git svn fetch to fetch the branch from SVN:

~/gcc> git config --add svn-remote.svn.fetch branches/redhat/gcc-4_4-branch:refs/remotes/origin/redhat/gcc-4_4-branch
~/gcc> git svn fetch

or get the whole redhat directory from the git mirror:

~/gcc> git config --add remote.origin.fetch refs/remotes/redhat:refs/remotes/origin/redhat
~/gcc> git fetch

With the second option, a checkout has all the redhat branches in subdirectories, rather than being able to check out a single branch at toplevel. If you only want one, you can use the git 1.7 sparse checkout function to limit what gets checked out, but it will still end up in a subdirectory.

Working on multiple branches at once

To make a separate working directory for hacking on another branch at the same time, you can just create a new working tree. First make sure the branch is in your git repository, as above. Then make a local tracking branch, if you haven't already done that with checkout -b:

~/gcc> git branch foo-branch origin/foo-branch

Then use the git-new-workdir script from the git sources to create a working directory for that branch:

> git clone git://git.kernel.org/pub/scm/git/git.git
> ./git/contrib/workdir/git-new-workdir ~/gcc gcc-foo foo-branch
> cd gcc-foo

Now you have a directory gcc-foo which contains a checkout of your foo-branch. This directory uses the same git repository as your main directory, it just has a different branch checked out.

Note that using git cherry-pick to copy changes between SVN branches brings along svn metadata which could confuse git-svn into thinking you're on a different SVN branch, so be sure to use cherry-pick -e to edit the commit message (or change it with git commit --amend afterwards).

Merging SVN branches

To merge trunk into a development branch:

$ git checkout my_branch
$ git svn rebase
$ git merge --squash origin/trunk
[ ... Fix merge conflicts ... ]
$ git commit -m'Merge from blah blah'

NOTE: It is important to squash the merge into a single commit. If you leave all the original commits in, git will commit them separately, causing a lot of SVN traffic. Additionally, if the commits have references to bugzilla PRs, they will be used to spam bugzilla, which will eventually force you to send a message along the lines of http://gcc.gnu.org/ml/gcc/2011-02/msg00047.html.

You have been warned. Use --squash!

Once that's done, your branch should look like this:

$ git status
# On branch my_branch
# Your branch is ahead of 'origin/my_branch' by 1 commit.
#
nothing to commit (working directory clean)

$ git svn dcommit --dry-run
Committing to svn+ssh://gcc.gnu.org/svn/gcc/branches/pph ...
diff-tree e63f... e63f...

$ git log -1
commit e63f...
Author: My Name  <my@address>
Date:   Thu Feb 3 12:55:55 2011 -0500

   Merge from blah blah

Test the merge and then commit with

$ git svn dcommit

Make your changes available online

To allow others to view your changes online or to pull your changes in their repositories you can make (parts) of your git repository available on services like repo.or.cz or gitorious.org.

Already available gcc forks at repo.or.cz

Let's start with creating a fork of the gcc git mirror. Go to the "Fork" page, select "push" mode and add project name, password, email and captcha. For this example the fork is called "fancyfork".

Now you have to signup as user here with login, email, and ssh public key. If you "edit" your new repository you can add the new user as committer.

Add our online repository to the local one

~/gcc> git remote add repo-or-cz ssh://repo.or.cz/srv/git/official-gcc/fancyfork.git

To make the local branch "mylocalbranch" available, push the branch to repo-or-cz

~/gcc> git push repo-or-cz mylocalbranch

New changes to already published branches can be pushed by just using

~/gcc> git push repo-or-cz

Others can now watch your activities online, add your fork as another remote repository to their local repository or just clone your fork.

Usage hints for git

git-merge-changelog

Before long you'll get frustrated with git's handling of ChangeLog merges, which is just as bad as SVN's. But there's a fix for that!

git clone git://git.savannah.gnu.org/gnulib.git
cd gnulib
# alternately, install gnulib from a package manager with something like
# 'sudo apt-get install gnulib' and just run /usr/bin/gnulib-tool 
./gnulib-tool --create-testdir --dir=/tmp/testdir123 git-merge-changelog
cd /tmp/testdir123
./configure
make
make install
git config --global merge.merge-changelog.name "GNU-style ChangeLog merge driver"
git config --global merge.merge-changelog.driver "/usr/local/bin/git-merge-changelog %O %A %B"
echo "ChangeLog   merge=merge-changelog" >> ~/gcc-git/.git/info/attributes

NOTE: git-merge-changelog used to be extremely slow for cherry-picking changes from trunk to release branches, but this was fixed with a change on 2009-07-02.

Update bash function

Here's my generic "update this source tree" bash function. When I want to rebase, I do that manually either with 'git svn rebase' or 'git rebase -i trunk'

up() { if [ -d CVS ]; then cvs -q update "$@";
       elif [ -d .svn ]; then svn up "$@";
       elif gitd=`git rev-parse --git-dir 2>/dev/null`; then
           if [ -f $gitd/objects/info/alternates ]; then
               for d in `cat $gitd/objects/info/alternates`; do
                   (cd $d; git fetch)
               done
           fi
           if [ -d $gitd/patches ] && `stg top 2>/dev/null`; then
               stg pull
           else
               git pull
           fi
       fi; }

rebase -i: Squashing multiple local commits into a single SVN commit

git rebase -i is a very useful tool for organizing local changes for svn dcommit.

If you have been making changes to your local tree, addressing review feedback, making minor fixes, etc. your local tree will have many commits. So, when you finally push the patch to SVN with 'git svn dcommit', you will be pushing all the local commits separately, which may not be what you want; a good rule of thumb is to have one commit for each patch sent to gcc-patches. To commit all your changes as a single SVN transaction, you need to squash all your local patches together. To do this:

$ git svn fetch -p
$ git rebase -i @{u}

The -i switch causes git rebase to ask you which patches you want to pick or squash. Reorganize the lines so that all the related changes are grouped together, and within a group of changes that you want to combine, replace 'pick' with 'squash' or 's' (or 'fixup' or 'f' if you don't need to adjust the commit message) on all lines except the first one. Then save the file and let 'git rebase' finish.

After this, the change(s) in your local git branch should look more like the patch(es) you want to send to gcc-patches, which you can then extract separately with git log -p for emailing to gcc-patches; see the 'git lgp' alias in the next section. Now, when you run 'git svn dcommit', you will push these patches to SVN rather than your local edit history.

To apply some but not all of your current commits to SVN, use git rebase -i to move the ones you want to apply to the top of the list, select "edit" for the last one you want to apply, and do git svn dcommit before git rebase --continue.

Useful aliases

Here are some aliases I'm finding useful (to add to your ~/.gitconfig). Note that these only work with the advanced setup due to differences in the branch.$BRANCH.merge config.

[alias]
        st = status
        ci = commit
        br = branch
        co = checkout
        sr = svn rebase
        sci = svn dcommit
        # The current branch.
        cbr = "!f(){ expr $( ( git symbolic-ref -q HEAD || cat $(git dir)/rebase-merge/head-name ) 2>/dev/null ) : 'refs/heads/\\(.*\\)'; }; f"
        # The branch being tracked by the current branch.
        track = "!f() { if p=$(git rev-parse --symbolic-full-name '@{u}' 2>/dev/null); then echo ${p##*/}; else git svn info|sed -n 's,^URL.*gcc/\\(branches/\\)\\?\\(.*\\),\\2,p'; fi; }; f"
        # Show all the local commits on this branch.
        lg = "!git log -p `git track`.."
        # Write all the local commits to ~/patch, filtering out modifications to ChangeLog files
        lgp = "!git log -p `git track`.. | filterdiff -x '*/ChangeLog' | sed -e '/^diff.*ChangeLog/{N;d}' > ~/patch"
        # Show all the local changes on this branch as one big diff.
        df = "!git diff `git track`"
        dfc = diff --cached
        # Reorganize the local commits on this branch.
        rb = "!git rebase -i `git track`"
        rc = rebase --continue
        # 'git rmerge mybranch' to reintegrate a temporary branch onto the top of the current branch
        rmerge = "!f(){ cur=`git cbr`; git rebase $cur $1; git rebase $1 $cur; }; f"

Pushing to a testing box

I tend to do development on my laptop, and then send patches off to a hefty compile server to do full regression testing. Before git, I did this by rsyncing diffs, but git simplifies the process. First, add to your gcc-git/.git/config:

[remote "testbox"]
        url = testbox.foo.bar:gcc-git

Then to push your current changes to the testbox, just "git push testbox +trunk". Note that you'll then need to "git co -f" or "git reset --hard" on the testbox to update the working copy from the git repository. This could be another alias:

        tup = "!f(){ cur=`git cbr`; git push $1 +$cur && ssh $1 \"cd src/$cur; git co -f\"; }; f"

So then you would write "git tup testbox" (assuming that "ssh testbox" also works). The alias could then proceed to run tests directly, but that is left as an exercise for the reader (as my setup involves a bunch of scripts).

Emacs

GNU Emacs 23 has decent git support in the vc package. The main attraction is C-x v g (vc-annotate), a handy interface to git blame that lets you step back to the revision before the line you're looking at with the 'a' key. It still needs some work; it hasn't been converted to run asynchronously, and the 'D' command doesn't seem to work properly, but it's still handy.

For editing git-related temporary files in emacs, I set my EDITOR environment variable to "emacsclient -t" and added the following to my .emacs:

(defun maybe-truncate ()
  (interactive)
  (cond ((string-match "/git-rebase-todo\\>" buffer-file-name)
         (toggle-truncate-lines))))
(add-hook 'find-file-hook 'maybe-truncate)

Old Information

Below here is information that no longer describes my use of git, but that other people might find interesting.

Stacked Git

I used to find Stacked Git useful for hacking on GCC, since it makes it easy to pop off patches that I'm still working on so I can check the one I've just finished testing into SVN. But lately I've stopped using it and just use git branches directly.

With stg, if I want to remove a commit from my tree temporarily and come back to it later, I do

stg uncommit mypatchname [myotherpatchname...]
stg pop -a

and then when I want to work on it again,

stg push mypatchname [myotherpatchname...]
stg commit

This way stg only knows about unapplied patches, so it won't get confused when I do a 'git pull'.

And I periodically look at the list of suspended patches with

stg series

Making Patches

I currently make patches for gcc-patches using my 'git lgp' alias, below.

However, git always produces unified diffs, while the GCC project policy is to prefer context diffs. I don't bother with this, but if you really want to make a context diff, you can create a wrapper script somewhere:

cat > ~/bin/git_diff_wrapper <<EOF
#! /bin/sh
exec diff -p -L "$1" "$2" "$5" | cat
EOF
chmod +x ~/bin/git_diff_wrapper

and when you want to make a context diff for submitting a patch, do

GIT_EXTERNAL_DIFF=git_diff_wrapper git diff trunk

Further Reading

An introduction to git-svn for Subversion/SVK users and deserters

git-svn workflow

Using git without feeling stupid (part 1), by Paolo Bonzini

Using git without feeling stupid (part 2), by Paolo Bonzini

http://git-scm.com/documentation/