Launchpad itself

SVN import from SourceForge fails every time

Asked by Mantas Kriaučiūnas on 2011-11-17

Hi launchpad developers,

I can't import an sourceforge svn repository of latest stable SuperTuxKart game release (0.7.3) into launchpad (https://code.launchpad.net/~stk/stk/0.7.3) since yesterday. Import fails every time with the following error msg:

2011-11-17 07:29:11 INFO Starting job.
2011-11-17 07:29:11 INFO Getting exising bzr branch from central store.
2011-11-17 07:29:11 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-11-17 07:29:12 INFO 253 bytes transferred
2011-11-17 07:29:24 INFO fetching svn revision info 1/1
2011-11-17 07:29:25 INFO Importing branch.
2011-11-17 07:29:26 INFO determining revisions to fetch 0/10213
2011-11-17 07:29:33 INFO
2011-11-17 07:29:33 INFO copying revision 0/5000
2011-11-17 07:30:33 INFO 18923357 bytes transferred | copying revision 0/5000
2011-11-17 07:31:33 INFO 9804533 bytes transferred | copying revision 15/5000
2011-11-17 07:31:46 INFO
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/scripts/code-import-worker.py", line 98, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/scripts/code-import-worker.py", line 93, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/lib/lp/codehosting/codeimport/worker.py", line 588, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/lib/lp/codehosting/codeimport/worker.py", line 774, in _doImport
    inter_branch.fetch(limit=revision_limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/branch.py", line 830, in fetch
    exclude_non_mainline=exclude_non_mainline)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/branch.py", line 842, in _fetch_revmetas
    project=self.source.project, mapping=self.source.mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/fetch.py", line 1647, in fetch
    pack_hint = self._fetch_revisions(needed, pb)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/fetch.py", line 1568, in _fetch_revisions
    use_replay=self._use_replay)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/fetch.py", line 1538, in _fetch_revisions_nochunks
    parent_revmeta)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/fetch.py", line 1478, in _fetch_revision_switch
    report_inventory_contents(reporter, parent_revnum, start_empty)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/errors.py", line 141, in convert
    return unbound(*args, **kwargs)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/fetch.py", line 1188, in report_inventory_contents
    reporter.finish()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/fetch.py", line 355, in change_prop
    self.editor.revmeta.get_previous_fileprops())
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/util.py", line 77, in keys
    self._ensure_init()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/util.py", line 29, in _ensure_init
    self.dict = self.create_fn(*self.args)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/branchprops.py", line 53, in _real_get_properties
    (_, _, props) = self.log._transport.get_dir(path, revnum)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14291/bzrplugins/optional/svn/errors.py", line 149, in convert
    raise mapped_err
bzrlib.plugins.svn.errors.DavRequestFailed: A Subversion remote access command failed: PROPFIND of '/svnroot/supertuxkart/!svn/bc/1118/trunk/supertuxkart': Could not read response body: Connection reset by peer (http://supertuxkart.svn.sourceforge.net)
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

See http://launchpadlibrarian.net/85324603/stk-stk-0.7.3.log and https://code.launchpad.net/~stk/stk/0.7.3 for more info.

I've tried both access methods - http and https, but both fails :(
Also I found more questions about same SVN import problems, see question #170639

Btw, 4 months ago SVN import of previous SuperTuxkart 0.7.2 release passed without problems, see https://code.launchpad.net/~stk/stk/0.7.2

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
Mantas Kriaučiūnas
Solved:
2011-12-10
Last query:
2011-12-10
Last reply:
2011-12-05
Jelmer Vernooij (jelmer) said : #1

Unfortunately this is SourceForge hanging up on us on Launchpad while it is doing the import.

One way to work around this would be for Launchpad to not import as many revisions at once, lowering the chance of connection issues. The most flexible solution would probably be to add a setting for imports that controls the batch size.

We can change the batch size of imports at the moment, but unfortunately this impacts all SVN imports - and would slow down new imports.

Mantas Kriaučiūnas (mantas) said : #2

Hi Jelmer,

Maybe there is a way to import just last revision of
http://supertuxkart.svn.sourceforge.net/svnroot/supertuxkart/main/branches/0.7.3/

We don't need full history of commits in this branch, we need just
last code and, if this possible, a possibility to get new revisions from
http://supertuxkart.svn.sourceforge.net/svnroot/supertuxkart/main/branches/0.7.3/

On Fri, Nov 18, 2011 at 12:50:47AM -0000, Jelmer Vernooij wrote:
> Unfortunately this is SourceForge hanging up on us on Launchpad while it
> is doing the import.
>
> One way to work around this would be for Launchpad to not import as many
> revisions at once, lowering the chance of connection issues. The most
> flexible solution would probably be to add a setting for imports that
> controls the batch size.
>
> We can change the batch size of imports at the moment, but unfortunately
> this impacts all SVN imports - and would slow down new imports.
>
> --
> Your question #179080 on Launchpad itself changed:
> https://answers.launchpad.net/launchpad/+question/179080

--
Labanaktis/Good luck,
Mantas Kriaučiūnas Jabber ID: <email address hidden> GPG ID: 43535BD5
Public organization "Open Source for Lithuania" - www.akl.lt
Geriausios biuro programos verslui ir namams - http://openoffice.lt
Prekyba naujais ir atnaujintais kompiuteriais su Linux OS - http://tinklas.eu
Naudok Baltix GNU/Linux sistemą savo kompiuteryje - http://baltix.lt

Mantas Kriaučiūnas (mantas) said : #3

Hello Jelmer and all launchpad developers,

Please tell use if there is a way to import just last revision of
http://supertuxkart.svn.sourceforge.net/svnroot/supertuxkart/main/branches/0.7.3/

We don't need full history of commits in this branch, we need just
last code and, if this possible, a possibility to get new revisions from
http://supertuxkart.svn.sourceforge.net/svnroot/supertuxkart/main/branches/0.7.3/

Please answer us, we are waiting for about a week for import of new SuperTuxKart version :(

On Fri, Nov 18, 2011 at 12:50:47AM -0000, Jelmer Vernooij wrote:
> Unfortunately this is SourceForge hanging up on us on Launchpad while it
> is doing the import.
> One way to work around this would be for Launchpad to not import as many
> revisions at once, lowering the chance of connection issues.

Mantas Kriaučiūnas (mantas) said : #4

jelmer told me in IRC #launchpad channel:
you could do the import yourself manually, and then push that to launchpad

ubuntu-baltix: jelmer, how I can import manually ?
jelmer: using bzr-svn, run "bzr init import"; "bzr pull -d import URL -r500"; "bzr pull -d import URL -r1000"; etc
ubuntu-baltix: can I import just last revision of http://supertuxkart.svn.sourceforge.net/svnroot/supertuxkart/main/branches/0.7.3/ ?
jelmer: you should be able to do that too, by just using "svn export" and then versioning that with bzr

ubuntu-baltix: if I import just last revision in this way, would it have a possibility to get new revisions from
http://supertuxkart.svn.sourceforge.net/svnroot/supertuxkart/main/branches/0.7.3/ with launchpad.net web interface?
jelmer: no, launchpad won't update the branch if you do a manual import
ubuntu-baltix: even if I import with bzr-svn commands "bzr init import"; "bzr pull -d import URL -r500"; etc ?
 jelmer: yes, launchpad doesn't import into user-managed branches

jelmer: we're going to change the batch size to 500 revisions, perhaps that will help

Julian Edwards (julian-edwards) said : #5

How did the manual import go?

Mantas Kriaučiūnas (mantas) said : #6

Julian Edwards (julian-edwards) said on 2011-12-05:
> How did the manual import go?

I repeated the import of sourceforge svn repository when you changed batch size to 500 revisions and this time SuperTuxKart 0.7.3 code was imported successfully, see https://code.launchpad.net/~stk/stk/0.7.3