Differences between revisions 28 and 29
Revision 28 as of 2009-06-15 21:24:09
Size: 10881
Editor: JasonMerrill
Comment: improve branch subdir procedure
Revision 29 as of 2009-06-16 20:28:20
Size: 8377
Editor: JasonMerrill
Comment: overhaul
Deletions are marked like this. Additions are marked like this.
Line 12: Line 12:
=== Get the source code of trunk ===
{{{
% mkdir gcc
% cd gcc
% git init
Initialized empty Git repository in /usr/home/Alice/gcc/.git/
% git remote add origin git+ssh://alice@gcc.gnu.org/git/gcc.git
% git fetch
remote: Counting objects: 870677, done.
% git branch -a
  origin/master
  origin/pre-globals-git
  origin/restrict-git
== Simple (Read-Only) Usage ==
Line 26: Line 14:
remote: Compressing objects: 100% (201036/201036), done.
Receiving objects: 100% (870677/870677), 296.70 MiB | 808 KiB/s, done.
Resolving deltas: 100% (706094/706094), done.
From git+ssh://alice@gcc.gnu.org/git/gcc
 * [new branch] master -> origin/master
 * [new branch] pre-globals-git -> origin/pre-globals-git
 * [new branch] restrict-git -> origin/restrict-git
}}}
=== Get svn branches ===
The git repository we cloned does not contain all branches. To do e.g. some graphite development we have to get the graphite branch.
If you don't have commit access to the GCC SVN respository, and you are happy with getting the default set of trunk and 4.x branches, you can just do
Line 38: Line 17:
% git config --add remote.origin.fetch '+refs/remotes/graphite:refs/remotes/origin/graphite'
% git fetch
remote: Counting objects: 86896, done.
remote: Compressing objects: 100% (41232/41232), done.
remote: Total 76919 (delta 49977), reused 59333 (delta 34708)
Receiving objects: 100% (76919/76919), 72.07 MiB | 695 KiB/s, done.
Resolving deltas: 100% (49977/49977), completed with 5347 local objects.
From git+ssh://alice@gcc.gnu.org/git/gcc
 * [new branch] graphite -> origin/graphite
% git branch -a
  origin/graphite
  origin/master
  origin/pre-globals-git
  origin/restrict-git
git clone git://gcc.gnu.org/git/gcc.git
git config branch.master.rebase true
Line 53: Line 20:
To get all branches use:
and then
Line 56: Line 24:
% git config --add remote.origin.fetch '+refs/remotes/*:refs/remotes/origin/*'
% git fetch
git pull
Line 59: Line 26:
=== Update repository from git ===
To get new commits just use "git fetch" or "git pull". The git mirror is updated at least every 30 minutes so you should get new code really fast.
Line 62: Line 27:
=== Using git-svn ===
Until now we did git only development. But there are two reasons to get git-svn in the game:

 * get new commits directly from the svn repository (no delay)
 * commit from our git repository (forget about svn completely)

!! I just figured out how to make it work for one branch

==== Initialize git svn ====
If you have cloned a repos from other developers, there is a quick way for you to rebuilding the git-svn metadata. For a detailed description of git-svn, you can read this post: [[http://utsl.gen.nz/talks/git-svn/intro.html|An introduction to git-svn for Subversion/SVK users and deserters]]

Here your working directory is your cloned repos:
any time you want to update your tree. You will probably also want to set up git-merge-changelog, as described later. To extract patches for submission, use
Line 76: Line 30:
# Initialize git-svn locally [1]
% git svn init svn+ssh://alice@gcc.gnu.org/svn/gcc/branches/graphite
git format-patch trunk
}}}
Line 79: Line 33:
# Pull the latest changes from Subversion [2]
% cp .git/refs/remotes/origin/graphite .git/refs/remotes/git-svn
== Advanced Usage ==
Line 82: Line 35:
# Update git from svn
% git svn fetch
Rebuilding .git/svn/git-svn/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4 ...
r115015 = b948ba9b92a300e4ad87fee28f9c530504b2d835
...
Done rebuilding .git/svn/git-svn/.rev_map.138bc75d-0d04-0410-961f-82ee72b054a4
The above works fine for browsing the repository and simple hacking, but doesn't really mesh well with the SVN model. Here's what seems to me to be the optimal way of using git-svn with the GCC repository. Please email me (jason) if you have any questions/issues with these instructions.
Line 89: Line 37:
}}}
1. '''IMPORTANT''': If the repos you cloned from initialize using git svn init svn+ssh://alice@gcc.gnu.org/svn/gcc/branches/graphite You must use ''exactly the same'' except the username. This means you CAN'T use svn:// or http:// instead of svn+ssh://. If you want to know details, see 3 after you reading 2.

2. This line of code

{{{
cp .git/refs/remotes/origin/graphite .git/refs/remotes/git-svn
}}}
will copy your HEAD's SHA number to file git-svn. SHA is a 40 digits number which looks like b948ba9b92a300e4ad87fee28f9c530504b2d835. So you can just simple run git log and then copy the 40 digits number to the file git-svn. The simple reason in 1 is that any different thing in a commit will cause the different SHA.

3. The reason why we do this(see 1) because every commit corresponding to a SHA number, which looks like 78ad11bf2f61b35e1cb32a978ab546d198be8a2e. If two commit has the same SHA, then they are exactly the same. This will including timestamps/commiter/patch, as well as what we mentioned in 1. If you use svn+ssh://, then it will corresponding a line like

{{{
git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/branches/graphite@145404 138bc75d-0d04-0410-961f-82ee72b054a4
}}}
 in the log, while svn:// will corresponding to

{{{
git-svn-id: svn://gcc.gnu.org/svn/gcc/branches/graphite@145404 138bc75d-0d04-0410-961f-82ee72b054a4
}}}
4. If you didn't clone a git repos from a developer, you can rebuild your own with the following instructions , feel free to use http:// svn:// or svn+ssh:// that

anything you can easily access. But this might cause you about 50 hours(for me) to rebuild the metadata.

{{{
% git svn init svn+ssh://alice@gcc.gnu.org/svn/gcc/branches/graphite
% git svn fetch}}}
For a detailed workflow of this usage, you can refer to this post:

[[http://notes.jimlindley.com/2008/3/25/git-svn-that-works-for-me|git-svn workflow]]

==== Update git svn ====
{{{
% git svn fetch
}}}
==== Commit to svn branch ====
{{{
% git svn dcommit
}}}
!! Check documentation to not commit a lot of strange stuff to the gcc repository.

== Alternative git-svn procedure (Jason Merrill) ==
Here's what seems to me to be the optimal way of using git-svn with the GCC repository:
=== Setup ===
Line 138: Line 44:
# Need to use this exact SVN url for gcc.git and git-svn to work together. # Need to use this exact SVN url for gcc.git and git-svn to work together.  Adding an SSH username to the url is ok.
Line 144: Line 50:
for f in 1_00 1_1; do git config --add remote.origin.fetch refs/remotes/egcs_"$f"_branch:refs/remotes/egcs_"$f"_branch; done for f in 1_00 1_1; do git config --add remote.origin.fetch refs/remotes/egcs_${f}_branch:refs/remotes/egcs_${f}_branch; done
Line 149: Line 55:
# Setup 'git pull'.
git config branch.master.remote origin
git config branch.master.merge refs/remotes/trunk
git pull
# Tell future pulls to rebase in order to work better with SVN.
git config branch.master.rebase true
# Actually pull down the stuff.
git fetch
# Tell pulls in tracking branches to rebase in order to work better with SVN.
git config branch.autosetuprebase remote
# Create master branch tracking SVN trunk
git checkout -b master trunk
# Ignore the same files that SVN does.
Line 159: Line 66:
If you want a branch that lives in a subdirectory of branches, such as redhat/gcc-4_4-branch, you can't get it from the gcc.git mirror for some reason, so leave out the remote.origin.fetch line for that branch, and do a git svn fetch to fetch all our branches from SVN. If you're going to do that, you probably want to do If you want a branch that lives in a subdirectory of branches, such as redhat/gcc-4_4-branch, you can't get it from the gcc.git mirror for some reason, so leave out the remote.origin.fetch line for that branch (and don't use the wildcard fetch commented out above), and use git svn fetch to fetch the branch from SVN:
Line 167: Line 74:
so you don't pull the whole world down from SVN with the fetch. clearing branches and tags across the fetch so it doesn't pull the whole world down from SVN (and take forever).
Line 169: Line 76:
Pull changes from the git mirror as desired using

{{{
git pull
}}}
or from the SVN repository (e.g. if you need a change that isn't in the git mirror yet) using

{{{
git svn rebase
}}}

To push all of your local commits to the SVN server, use

{{{
git svn dcommit
}}}
=== git-merge-changelog ===
Line 199: Line 91:
To make a local clone for hacking on another branch at the same time, you can do
=== Usage ===

Pull changes from the git mirror as desired using
Line 202: Line 97:
mkdir gcc44-git
cd gcc44-git
git pull
}}}
or from the SVN repository (e.g. if you need a change that isn't in the git mirror yet, or are working on a branch that isn't in the git mirror) using

{{{
git svn rebase
}}}
You probably don't want to use git svn fetch other than for the initial import of a branch, as shown above.

To look at your local changes, use
{{{
git log -p trunk..
}}}
To push all of them to the SVN server, use

{{{
git svn dcommit
}}}

=== Working with other branches ===
To switch your current working directory to a different branch, first set up a local tracking branch:

{{{
git checkout -b gcc44 gcc-4_4-branch
}}}
Then you can switch between branches with
{{{
git checkout master
git checkout gcc44
}}}

To make a separate working directory for hacking on another branch at the same time, you can do

{{{
mkdir lto
cd lto
Line 205: Line 134:
# Tell git not to re-fetch shared objects in the trunk repository # Tell git not to re-fetch shared objects in your main repository
Line 210: Line 139:
git config --add remote.origin.fetch refs/remotes/gcc-4_4-branch:refs/remotes/gcc-4_4-branch
git config branch.master.remote origin
git config branch.master.merge refs/remotes/gcc-4_4-branch
git pull
git config branch.master.rebase true
# If the first fetch says "remote: counting objects", stop it, update your main repository and try again.
# You can skip this first fetch if the branch you want is in your main repository.
git fetch
git config --add remote.origin.fetch refs/remotes/lto:refs/remotes/lto
git fetch
git config branch.autosetuprebase remote
git checkout -b master lto
Line 218: Line 149:
And then do pull/rebase/dcommit as with your main repository; this repository should be significantly smaller than the main one as it only contains the changes on the branch itself; it uses your main repository for anything on the trunk. In fact, if your main repository includes gcc-4_4-branch, the .git directory in the new repository will be essentially empty. If you pull in this repository sooner than in the main one, I think you'll get a few objects here, but I would expect them to be cleaned up when this repository gets repacked.  You can use this repository without doing anything in your trunk repository as long as you don't delete the trunk repository. And then do pull/rebase/dcommit as with your main repository. This repository should be significantly smaller than the main one as it only contains the changes on the branch itself; it uses your main repository for anything on the trunk. In fact, if you do this for a branch that's already in your main repository, the .git directory in the new repository will be less than 10MB. If you pull in this repository sooner than in the main one, you'll get a few duplicate objects here, but you can clean them up with 'git gc'. You can use this repository without doing anything in your trunk repository as long as you don't delete the trunk repository or remove branches that this one is using.
Line 220: Line 151:
Note that you shouldn't use git cherry-pick to copy changes between SVN branches, as that brings along svn metadata which could confuse git-svn into thinking you're on a different SVN branch. Better I think to use git show/git apply, or do the merge in SVN directly (which also handles copying testcases better). The double fetch is to work around an apparent shortcoming of the "alternates" functionality (at least with 1.6.2.2): if the alternate repository doesn't have the objects at the heads of *all* the branches you are fetching with the initial fetch, git will download everything. So if your main repository is even one commit behind the head, git will re-download the whole shebang all over again. After the first fetch git is smarter about only downloading deltas, so the second fetch will only download the differences between trunk and the branch.
Line 222: Line 153:
Note that you shouldn't use git cherry-pick to copy changes between SVN branches, as that brings along svn metadata which could confuse git-svn into thinking you're on a different SVN branch. Better I think to use git show/git apply, or do the merge in SVN directly (which also handles new testcases better).

=== Stacked Git ===
Line 224: Line 158:
Update to current git head:
Line 225: Line 160:
# Update to current git head:
Line 228: Line 162:
Commit patch FOO to SVN:
Line 229: Line 164:
# Commit patch FOO to SVN: # Pop all current patches
Line 231: Line 166:
# Make sure I'm up-to-date with SVN head
Line 232: Line 168:
# Push back the patch I want to commit to SVN
Line 233: Line 170:
# And commit it so that it's a normal git commit rather than an stg patch
Line 234: Line 172:
# Finally, push it out to SVN
Line 235: Line 174:
# And reapply my other patches to my working tree
Line 237: Line 177:

== Further Reading ==

[[http://utsl.gen.nz/talks/git-svn/intro.html|An introduction to git-svn for Subversion/SVK users and deserters]]

[[http://notes.jimlindley.com/2008/3/25/git-svn-that-works-for-me|git-svn workflow]]

GCC GIT mirror

For the gcc svn repository exists also an read only git mirror. So you can use git for your daily gcc development.

The git repository is available at:

http://gcc.gnu.org/git/gcc.git
git://gcc.gnu.org/git/gcc.git
git+ssh://gcc.gnu.org/git/gcc.git

Simple (Read-Only) Usage

If you don't have commit access to the GCC SVN respository, and you are happy with getting the default set of trunk and 4.x branches, you can just do

git clone git://gcc.gnu.org/git/gcc.git
git config branch.master.rebase true

and then

git pull

any time you want to update your tree. You will probably also want to set up git-merge-changelog, as described later. To extract patches for submission, use

git format-patch trunk

Advanced Usage

The above works fine for browsing the repository and simple hacking, but doesn't really mesh well with the SVN model. Here's what seems to me to be the optimal way of using git-svn with the GCC repository. Please email me (jason) if you have any questions/issues with these instructions.

Setup

mkdir gcc-git
cd gcc-git
git init
git remote add origin git://gcc.gnu.org/git/gcc.git
# Need to use this exact SVN url for gcc.git and git-svn to work together.  Adding an SSH username to the url is ok.
git svn init -s svn+ssh://gcc.gnu.org/svn/gcc
# Get the trunk.
git config remote.origin.fetch refs/remotes/trunk:refs/remotes/trunk
# Also get the various release branches so we can refer to them as needed.
for f in 2_95 3_0 3_1 3_2 3_3 3_4 4_0 4_1 4_2 4_3 4_4; do git config --add remote.origin.fetch refs/remotes/gcc-$f-branch:refs/remotes/gcc-$f-branch; done
for f in 1_00 1_1; do git config --add remote.origin.fetch refs/remotes/egcs_${f}_branch:refs/remotes/egcs_${f}_branch; done
# Add more branches if you want, e.g.
# git config --add remote.origin.fetch refs/remotes/lto:refs/remotes/lto
# If you want all branches, do
# git config remote.origin.fetch refs/remotes/*:refs/remotes/*
# Actually pull down the stuff.
git fetch
# Tell pulls in tracking branches to rebase in order to work better with SVN.
git config branch.autosetuprebase remote
# Create master branch tracking SVN trunk
git checkout -b master trunk
# Ignore the same files that SVN does.
git svn show-ignore >> .git/info/exclude

and you're all set to start hacking. Note that the first remote.origin.fetch line doesn't use --add because we aren't interested in origin's heads, just its remotes that track SVN. The uncommented lines pull down the SVN trunk and release branches; look at the commented lines if you want other branches. Pulling all branches increases the size of the .git directory by about 30% over just trunk and release branches.

If you want a branch that lives in a subdirectory of branches, such as redhat/gcc-4_4-branch, you can't get it from the gcc.git mirror for some reason, so leave out the remote.origin.fetch line for that branch (and don't use the wildcard fetch commented out above), and use git svn fetch to fetch the branch from SVN:

git config --unset svn-remote.svn.branches
git config --unset svn-remote.svn.tags
git config --add svn-remote.svn.fetch branches/redhat/gcc-4_4-branch:refs/remotes/redhat/gcc-4_4-branch
git svn fetch
git svn init -s svn+ssh://gcc.gnu.org/svn/gcc

clearing branches and tags across the fetch so it doesn't pull the whole world down from SVN (and take forever).

git-merge-changelog

Before long you'll get frustrated with git's handling of ChangeLog merges, which is just as bad as SVN's. But there's a fix for that!

git clone git://git.savannah.gnu.org/gnulib.git
cd gnulib
./gnulib-tool --create-testdir --dir=/tmp/testdir123 git-merge-changelog
cd /tmp/testdir123
./configure
make
make install
git config --global merge.merge-changelog.name "GNU-style ChangeLog merge driver"
git config --global merge.merge-changelog.driver "/usr/local/bin/git-merge-changelog %O %A %B"
echo "ChangeLog   merge=merge-changelog" >> ~/gcc-git/.git/info/attributes

Usage

Pull changes from the git mirror as desired using

git pull

or from the SVN repository (e.g. if you need a change that isn't in the git mirror yet, or are working on a branch that isn't in the git mirror) using

git svn rebase

You probably don't want to use git svn fetch other than for the initial import of a branch, as shown above.

To look at your local changes, use

git log -p trunk..

To push all of them to the SVN server, use

git svn dcommit

Working with other branches

To switch your current working directory to a different branch, first set up a local tracking branch:

git checkout -b gcc44 gcc-4_4-branch

Then you can switch between branches with

git checkout master
git checkout gcc44

To make a separate working directory for hacking on another branch at the same time, you can do

mkdir lto
cd lto
git init
# Tell git not to re-fetch shared objects in your main repository
echo ~/gcc-git/.git/objects >> .git/objects/info/alternates
git remote add origin git://gcc.gnu.org/git/gcc.git
git svn init -s svn+ssh://gcc.gnu.org/svn/gcc
git config remote.origin.fetch refs/remotes/trunk:refs/remotes/trunk
# If the first fetch says "remote: counting objects", stop it, update your main repository and try again.
# You can skip this first fetch if the branch you want is in your main repository.
git fetch
git config --add remote.origin.fetch refs/remotes/lto:refs/remotes/lto
git fetch
git config branch.autosetuprebase remote
git checkout -b master lto
cp ~/gcc-git/.git/info/{attributes,exclude} .git/info/

And then do pull/rebase/dcommit as with your main repository. This repository should be significantly smaller than the main one as it only contains the changes on the branch itself; it uses your main repository for anything on the trunk. In fact, if you do this for a branch that's already in your main repository, the .git directory in the new repository will be less than 10MB. If you pull in this repository sooner than in the main one, you'll get a few duplicate objects here, but you can clean them up with 'git gc'. You can use this repository without doing anything in your trunk repository as long as you don't delete the trunk repository or remove branches that this one is using.

The double fetch is to work around an apparent shortcoming of the "alternates" functionality (at least with 1.6.2.2): if the alternate repository doesn't have the objects at the heads of *all* the branches you are fetching with the initial fetch, git will download everything. So if your main repository is even one commit behind the head, git will re-download the whole shebang all over again. After the first fetch git is smarter about only downloading deltas, so the second fetch will only download the differences between trunk and the branch.

Note that you shouldn't use git cherry-pick to copy changes between SVN branches, as that brings along svn metadata which could confuse git-svn into thinking you're on a different SVN branch. Better I think to use git show/git apply, or do the merge in SVN directly (which also handles new testcases better).

Stacked Git

I'm also finding Stacked Git useful for hacking on GCC, since it makes it easy to pop off patches that I'm still working on so I can apply the one I've just finished testing. See the StGit webpage for information on general usage; for using it with git-svn what I find works best is to do

Update to current git head:

stg pull

Commit patch FOO to SVN:

# Pop all current patches
stg pop -a
# Make sure I'm up-to-date with SVN head
git svn rebase
# Push back the patch I want to commit to SVN
stg push FOO
# And commit it so that it's a normal git commit rather than an stg patch
stg commit
# Finally, push it out to SVN
git svn dcommit
# And reapply my other patches to my working tree
stg push -am

Further Reading

An introduction to git-svn for Subversion/SVK users and deserters

git-svn workflow

None: GitMirror (last edited 2017-09-15 17:54:29 by JasonMerrill)