Bazaar

Import from a svn repository with non standard layout

Asked by Adrien Saladin on 2010-01-27

Hi,

I would like to convert a svn repository to multiple bzr branches. However the svn repo does not have exacly the classic {trunk, branches, tags} layout.

The first point is that the root of the project is a subdirectory of the repository.
Second, the branches/ directory was subdivided into several parts. For example each user has its own "private" directory for his branches. In each user directory, some directory are branches (svn cp) of trunk, other directory are branches of branches.

(see figure below)

repository/
    +- ProjectFoo/
          +-trunk/ # normal trunk
          |
          +-branches/
          | +- User1/
          | | +- feature1/ # trunk branch for feature1
          | | |
          | | +- feature2/ # trunk branch for feature2
          | | |
          | | +- feature2b/ # feature 2b is a svn cp of feature2 (a branch of the branch)
          | |
          | |+- User2/
          | |
          | |+- feature3/ # some branches do not belong
          | # to any user but are directly in the
          | # branches/ directory
          |
          |
          |+- tags/

I'm looking for a way to extract each individual branch into a bzr branch, keeping the history from the first revision of trunk when possible. For example, say that feature1/ is a "svn cp" of trunk from revision 100. I would like to keep in the bzr branch the history from revisions 1 to 100, as well as additional features added in revisions 102 to the latest one (here rev 101 is just the svn cp command. I don't know if this must be translated into bzr concepts).

I have tried with the svn2bzr tool but I did't find a correct set of options to do what I want. The best I could obtain was a partial history of feature2, from revision 101 to the latest one. So revision 1 in the new bzr branch was the state of trunk at rev 100, all 99 previous commits information were lost.

I'm now trying to better with bzr fast-export/fast-import. I probably typed incorrect commands since this generates a bzr internal error.

Can someone give me some hints about the correct commands for fast-export-from-svn fast-filter and fast-import to
import feature2 and feature2b directories, OR any working method to correctly convert out svn history to bzr branches.

Thanks for your time.

Question information

Language:
English Edit question
Status:
Solved
For:
Bazaar Edit question
Assignee:
No assignee Edit question
Solved by:
Adrien Saladin
Solved:
2010-01-29
Last query:
2010-01-29
Last reply:
Adrien Saladin (adrien-saladin) said : #1

Hello again,

Just somme more information about the bzr internal error. This happens when I try to do a bzr fast-import of my fast-exported svn repository. I have tried several options for the fast-export command, like changing paths for trunk and/or branches.

I can't tell if it's really a bzr bug since I don't know if I'm correctly using fast-export-from-svn and fast-import.

Thank you.

$ bzr fast-import ptools.fi ptoolsbzr
01:10:23 Collecting statistics ...
01:10:23 Starting import of 255 commits ...
ABORT: exception occurred processing commit @211
bzr: ERROR: exceptions.IndexError: list index out of range

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/__init__.py", line 384, in run
    params, verbose, user_map=user_map)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/__init__.py", line 111, in _run
    return proc.process(p.iter_commands)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/processor.py", line 95, in process
    self._process(command_iter)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/processors/generic_processor.py", line 280, in _process
    processor.ImportProcessor._process(self, command_iter)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/processor.py", line 117, in _process
    handler(self, cmd)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/processors/generic_processor.py", line 490, in commit_handler
    handler.process()
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/processor.py", line 200, in process
    self.pre_process_files()
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/bzr_commit_handler.py", line 578, in pre_process_files
    super(InventoryDeltaCommitHandler, self).pre_process_files()
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/bzr_commit_handler.py", line 91, in pre_process_files
    self.revision = self.build_revision()
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/bzr_commit_handler.py", line 223, in build_revision
    self.branch_ref)
  File "/usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport/branch_mapper.py", line 39, in git_to_bzr
    name_no_ext = self._git_to_bzr_name(parts[-1])
IndexError: list index out of range

bzr 2.0.2 on python 2.5.4 (Linux-2.6.30-2-amd64-x86_64-with-debian-squeeze-sid)
arguments: ['/usr/bin/bzr', 'fast-import', 'ptools.fi', 'ptoolsbzr']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'fr_FR.UTF-8'
plugins:
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [2.0.1]
  explorer /home/adrien/.bazaar/plugins/explorer [0.10.0dev]
  fastimport /usr/lib/python2.5/site-packages/bzrlib/plugins/fastimport [0.9.0dev]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [2.0.2]
  netrc_credential_store /usr/lib/python2.5/site-packages/bzrlib/plugins/netrc_credential_store [2.0.2]
  qbzr /home/adrien/.bazaar/plugins/qbzr [0.17.0dev]
  rebase /usr/lib/python2.5/site-packages/bzrlib/plugins/rebase [0.5.4]
  search /usr/lib/python2.5/site-packages/bzrlib/plugins/search [1.7.0dev]
  svn /usr/lib/python2.5/site-packages/bzrlib/plugins/svn [1.0.0]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

Ian Clatworthy (ian-clatworthy) said : #2

Hi Adrien,

It looks like a bzr-fastimport bug. I'll take a look.

Ian Clatworthy (ian-clatworthy) said : #3

Adrien,

Can you try with the latest bzr-fastimport trunk, .i.e. rev 271 or later. (That code in branch_mapper.py no longer exists.)

Adrien Saladin (adrien-saladin) said : #4

I have updated bzr-fastimport to rev 271. Since the error message is different I guess bzr is using this new version instead of my linux distribution version.

However I still have an error. The error message is clearer and does not trigger an exception:

$ bzr fast-import ptools.fi ptools.bzr
13:03:41 Collecting statistics ...
13:03:41 Starting import of 255 commits ...
ABORT: exception occurred processing commit @211
bzr: ERROR: An inconsistent delta was supplied involving u'/SConstruct', 'sconstruct-20100128120341-wcaza3s25gxghi98-3'
reason: New path is not consistent with parent path.

Ian Clatworthy (ian-clatworthy) said : #5

OK. Let's convert this to an bug on bzr-fastimport. Could you try running with -Derror and recording the full exception in the bug report please?

Adrien Saladin (adrien-saladin) said : #6

Hi Ian,

I pasted the log below. Please tell me if you need an access to the svn repository.
Thanks,

$ bzr fast-import ptools.fi ptools.bzr -Derror
09:06:47 Collecting statistics ...
09:06:47 Starting import of 255 commits ...
ABORT: exception occurred processing commit @211
bzr: ERROR: bzrlib.errors.InconsistentDelta: An inconsistent delta was supplied involving u'/SConstruct', 'sconstruct-20100129080647-cx5239q3luoyvyxb-3'
reason: New path is not consistent with parent path.

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/adrien/.bazaar/plugins/fastimport/__init__.py", line 384, in run
    params, verbose, user_map=user_map)
  File "/home/adrien/.bazaar/plugins/fastimport/__init__.py", line 111, in _run
    return proc.process(p.iter_commands)
  File "/home/adrien/.bazaar/plugins/fastimport/processor.py", line 95, in process
    self._process(command_iter)
  File "/home/adrien/.bazaar/plugins/fastimport/processors/generic_processor.py", line 280, in _process
    processor.ImportProcessor._process(self, command_iter)
  File "/home/adrien/.bazaar/plugins/fastimport/processor.py", line 117, in _process
    handler(self, cmd)
  File "/home/adrien/.bazaar/plugins/fastimport/processors/generic_processor.py", line 490, in commit_handler
    handler.process()
  File "/home/adrien/.bazaar/plugins/fastimport/processor.py", line 208, in process
    self.post_process_files()
  File "/home/adrien/.bazaar/plugins/fastimport/bzr_commit_handler.py", line 607, in post_process_files
    self._get_inventories)
  File "/home/adrien/.bazaar/plugins/fastimport/revision_store.py", line 401, in load_using_delta
    builder.finish_inventory()
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 210, in finish_inventory
    self.parents)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 1067, in add_inventory_by_delta
    basis_inv.apply_delta(delta)
  File "/usr/lib/python2.5/site-packages/bzrlib/inventory.py", line 1193, in apply_delta
    "New path is not consistent with parent path.")
InconsistentDelta: An inconsistent delta was supplied involving u'/SConstruct', 'sconstruct-20100129080647-cx5239q3luoyvyxb-3'
reason: New path is not consistent with parent path.

Adrien Saladin (adrien-saladin) said : #7

Ok, I solved my problem of importing from svn with "bzr svn-import".
(thanks to mzz for the suggestion).

I will fill a separated bug report for bzr-fastimport.