[prev in list] [next in list] [prev in thread] [next in thread] 

List:       mercurial
Subject:    Re: Tracking a remote Subversion repository
From:       Giorgos Keramidas <keramida () ceid ! upatras ! gr>
Date:       2006-09-29 21:03:53
Message-ID: 20060929210353.GA86076 () gothmog ! pc
[Download RAW message or body]

On 2006-09-21 22:01, Paul Moore <p.f.moore@gmail.com> wrote:
> Hi. I'm looking for some advice here on how to best use Mercurial to
> manage my personal modifications to an existing project which uses
> Subversion.
> 
> To be specific, I want to check out the sources for Python. I then
> want to use that checkout to create a local Mercurial repository,
> where I'll develop some changes. I can't just use my subversion
> checkout, as I want to work disconnected, and other neat stuff that
> Mercurial makes easy.
> 
> OK, I now make changes in my Hg repository. Periodically, I'd like to
> update from Subversion, and merge that into my Hg repository. Finally,
> at some point, I'll want to create a patch file from my Hg repository,
> comprising the changes that need to be applied to the svn repository
> to incorporate my change.
> 
> I'm not bothered about retaining any svn history within hg, nor am I
> worried about pushing my changes incrementally, if that helps.
> 
> With a bit of testing, it looks like I could maybe just make my svn
> working directory a hg repository as well, by something like
> 
>    hg init my_repo
>    svn co http://svn.example.com/projects/whatever my_repo
>    cd my_repo
>    hg add
>    hg commit -m "Initial commit of revision XXXXX"
> 
> ... time passes
> 
>    svn up
>    hg add # Add any new files from svn
>    hg commit -m "Update to revision YYYYY"
> 
> (I should probably keep the .svn directories out of hg version
> control, though - I don't know how best to do that).
> 
> I'm still a bit hazy on good working practices here, though. Can
> anyone offer me any suggestions?

I use a similar 'trick' to work locally with Mercurial, and a central
Perforce repository at work.  The steps you outlined above are almost
correct, but some details need a bit of tweaking and there are some
things that you should be careful about[1].

This is not Subversion specific, but you can certainly adapt the steps
outlined below to work with Subversion too.

Copies of the Source Tree
*************************

I keep the following copies of the source tree locally:

  * A local, clean checkout of a Perforce branch in `~/p4/foo'.

  * A clean copy of the `foo' Perforce branch, imported in a local
    Mercurial repository at `~/hg/foo-import'.

  * A working branch for my own stuff in `~/hg/foo'.

Importing from Perforce into Mercurial
**************************************

To resync my Mercurial branches with changes from Perforce, I use the
following steps:

 * Resync the Perforce checkout in `~/p4/foo'

   + First, I look at the latest changes in Perforce:

         host:~$ cd ~/p4/foo
         host:~/p4/foo$ p4 changes -m5 ...

     This will show the last 5 changesets committed into the `foo'
     Perforce branch.  I take a note of the most recent change number
     here, and then sync up to this change.  Let's assume, for this
     example, that this change number is 100.

   + Then, I sync up to a specific Perforce change number:

         host:~/p4/foo$ p4 sync ...@100

 * I remove everything from the `foo-import' Mercurial branch:

         host:~/p4/foo$ cd ~/hg/foo-import
         host:~/hg/foo-import$ rm -fr *

 * I copy over the `~/p4/foo' branch:

         host:~/hg/foo-import$ cd ~/p4/foo
         host:~/p4/foo$ find . | cpio -p -dmu ~/hg/foo-import

 * I commit all the changes into the `foo-import' Mercurial branch:

         host:~/p4/foo$ cd ~/hg/foo-import
         host:~/hg/foo-import$ hg commit -A -m 'import @ 100'

   Keeping the last imported Perforce change number in the commit log
   has been extremely useful for me, so you may want to do something
   similar.  When something goes wrong, you know the range of changes in
   the Perforce branch where things started going wrong :)

 * I remove all files from `foo-import' and checkout a clean copy[3].

After this is done, the Mercurial `foo-import' branch has a clean copy
of the `foo' Perforce branch, as it was after change 100 went in[2].

Note that this does *NOT* preserve the Perforce history!!!  I usually
don't care too much about this, as I do frequent updates of the relevant
Mercurial repos.  If you think having the history is important, then a
conversion script of some sort should be used.

Pulling & Merging Changes into My Working Branch
************************************************

When everything has been imported without problems to the `foo-import'
Mercurial branch, I can clone it, push changes to `foo', pull changes
from `foo-import' into `foo', etc. as usual.

Some amount of merging may be required, especially if changes from
Perforce happen to touch the same files that you have modified in
Mercurial's `foo' branch, but that's ok and it works seamlessly with the
usual Mercurial commands for merging multiple repository heads.

Note that if you notice that something is broken in one of the merges,
you can always enable the Mercurial Queues extension[4], which will let
you use the `hg strip' command.  Then you can merge and split again the
same heads with series of commands like:

    host:~/hg/foo$ hg merge
    host:~/hg/foo$ hg strip tip

Pushing Changes Back Into Perforce
**********************************

When I'm done working in Mercurial, I usually generate a single `patch'
with the changes for a particular Perforce feature/bugfix/change and
then I have to push this back into Perforce.  Once you have a single
patch file with the changes, you can apply the patch to the Perforce
checkout as shown below:

    host:~$ cd ~/p4/foo
    host:~/p4/foo$ gpatch -p1 < ~/patchfile 2>&1 | tee ~/patchfile.log

The GNU patch version (hence the `gpatch' executable name above) will
print the path names of the files it has modified, so you can save a lot
of time by using this list of filenames when committing the changeset
into Perforce.

The normal process of committing a large changeset into Perforce would
be something like:

    host:~/p4/foo$ p4 diff -sd ... | p4 -x - delete
    host:~/p4/foo$ find . \! -type d | p4 -x - add 2>&1 | fgrep -v ' existing'
    host:~/p4/foo$ p4 diff -se ... | p4 -x - edit

Doing this for very large branches, including thousands of files,
hundreds of directories over a slow link is an absolute *KILLER* though,
so you can leverage the output of GNU patch(1), to speed things up.

First extract the list of filenames that are touched by patch(1), by
editing the `~/patchfile.log' file.  Once you have a list of files, say
in `~/files.lst', you can do something like:

    host:~/p4/foo$ p4 -x ~/files.lst diff -sd | p4 -x - delete
    host:~/p4/foo$ p4 -x ~/files.lst add
    host:~/p4/foo$ p4 -x ~/files.lst diff -se | p4 -x - edit

    NOTE: Something similar can be done with the output of the
    `svn stat' command and pipes to `svn add' or `svn rm'.
    There is no need for an `svn edit' command, in Subversion.

Then, you are ready to `p4 submit ...' in `~/p4/foo' and commit all the
modifications as a single changeset.

Leveraging Mercurial Queues
***************************

Instead of having to remember manually which changesets to 'merge' in
order to produce a series of `patches' to the `foo' Perforce branch,
I tend to use the Mercurial Queues extension.

With this extension, I can give custom "names" to the patches I have to
prepare for Perforce.  For example, if I have two patches that change
the syslog support and add some debugging statements to the `foo'
branch, I might do:

    host:~/hg/foo$ rm -fr * && hg up -C
    host:~/hg/foo$ hg qinit -c
    host:~/hg/foo$ hg qnew -m 'Syslog updates' syslog
    host:~/hg/foo$ hg qnew -m 'More debugging' debug

Then I can qpush/qpop the patches, modify them as I wish, using the
features of the MQ extension.  When I'm done, I usually pop all the
patches and use the files from .hg/patches/ directly as GNU patch(1)
input to apply the changes to the Perforce branch.

This is much easier than trying to track down which changesets to 'hg
export' and the order that they have to be applied on the Perforce
checkouts.

Notes
*****

[1] Mercurial doesn't track symlinks, while Perforce and Subversion do
    this.

    You should be careful when committing to the Perforce repository to
    avoid accidental removal of symlinks, i.e. because you copied over
    stuff from a Mercurial branch and used the `p4 diff -sd ...' command
    of Perforce.

[2] Minus any symlinks again.

[3] So that any stale symlinks copied over from the `foo' Perforce
    branch are removed.

[4] By adding to your `~/.hgrc' file:

        [extensions]
        hgext.mq =
_______________________________________________
Mercurial mailing list
Mercurial@selenic.com
http://selenic.com/mailman/listinfo/mercurial
[prev in list] [next in list] [prev in thread] [next in thread]