code imports from sourceforge are blocked

Bug #244575 reported by Diogo Matsubara
12
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Joey Stanford

Bug Description

Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../scripts/code-import-worker.py", line 46, in ?
    script.main()
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../scripts/code-import-worker.py", line 41, in main
    import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../lib/canonical/codehosting/codeimport/worker.py", line 436, in run
    foreign_tree = self.getForeignTree()
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../lib/canonical/codehosting/codeimport/worker.py", line 362, in getForeignTree
    return self.foreign_tree_store.fetch(
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../lib/canonical/codehosting/codeimport/worker.py", line 262, in fetch
    return self.fetchFromArchive(source_details, target_path)
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../lib/canonical/codehosting/codeimport/worker.py", line 288, in fetchFromArchive
    tree.update()
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../lib/canonical/codehosting/codeimport/foreigntree.py", line 68, in update
    tree.update()
  File "/srv/importd.launchpad.net/production/launchpad/scripts/../lib/svn_oo/WorkingTree.py", line 143, in update
    client.update(self.root, ignore_externals=True)
pysvn._pysvn_2_4.ClientError: PROPFIND request failed on '/svnroot/wp-grain/theme/trunk'
PROPFIND of '/svnroot/wp-grain/theme/trunk': could not connect to server (https://wp-grain.svn.sourceforge.net)
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

Looks like our IP is blocked. This bug report is to track progress on a fix for this issue.

Tags: lp-code
Changed in launchpad-bazaar:
assignee: nobody → mwhudson
importance: Undecided → High
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Joey is in contact with SF about this.

Changed in launchpad-bazaar:
assignee: mwhudson → rinchen
Joey Stanford (joey)
Changed in launchpad-bazaar:
status: New → In Progress
Revision history for this message
Joey Stanford (joey) wrote :

Hi, We have a relationship with SF now. They are going through a datacenter move which is complicating this. They have agreed to restore service but have not given an exact date yet.

Revision history for this message
Rolf Leggewie (r0lf) wrote :

Maybe you guys should consider throttling bandwidth or something. The gnucash admin also blocked canonical because of the bandwidtch used. Maybe you can take a break every few hours and import piece by piece? Like say, import 100 revisions and pause for the amount of time that took. This would also lead to a partial import becoming available faster for the imported bzr branch.

Revision history for this message
Joey Stanford (joey) wrote :

I've asked Sourceforge about this but so far they have not replied. They are in the midst of a datacenter move so I expect it may take them some time to reply.

Revision history for this message
Michael Chang (thenewme91) wrote :

Would it make sense for us (i.e. Launchpad) to temporarily disable/stop all imports from sourceforge for the time being?

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

We seem to have access again. Probably best to leave this bug open until we know that this was intentional on their end though :)

Revision history for this message
Jonathan Lange (jml) wrote :

This looks like it's still an issue:
https://code.edge.launchpad.net/~vcs-imports/openrpg/trunk

http://launchpadlibrarian.net/16103401/openrpg-trunk-log.txt

cvs [checkout aborted]: connect to openrpg.cvs.sourceforge.net(66.35.250.83):2401 failed: Connection refused

Revision history for this message
Diogo Matsubara (matsubara) wrote :

<matsubara> Rinchen: any news about the SF blockage?
<Rinchen> yeah they emailed me yesterday
<Rinchen> 1-2 weeks

Revision history for this message
Joey Stanford (joey) wrote :

Tentatively targeting to 2.1.8. SF said they should be able to unblock us fully by the 8th. We've seen intermittent restoration of service though already.

Changed in launchpad-bazaar:
milestone: none → 2.1.8
Revision history for this message
Treeform (starplant) wrote :

Its the 9th. Do they have any idea when they could restore it - or did they just forget?

Revision history for this message
Joey Stanford (joey) wrote : Re: [Bug 244575] Re: code imports from sourceforge are blocked

I've emailed them again around 14:38 UTC today.

Revision history for this message
Joey Stanford (joey) wrote :

Jacob at SourceForge emailed me today:

"We had a delay in handling this part of the migration, so I'll need to circle back in a couple more weeks. Our migration efforts cover a lot of very different services (from a systems level); we've moved on to the next service and will be circling back to finish out SVN."

In a previously publicized note to SF users, the end of August was used as the completion date. So, it looks like 1st of September might be the next checkpoint.

Revision history for this message
Joey Stanford (joey) wrote :

Slipping to 2.1.9 per my previous comment. :-(

Changed in launchpad-bazaar:
milestone: 2.1.8 → 2.1.9
Revision history for this message
Treeform (starplant) wrote :

6 more days! We been waiting for months... some how i don't feel confident they will meet the dead line.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Actually, it's been less than a month :)

Time flies, but not _that_ fast.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Er, I can't count. /me hides.

Revision history for this message
Joey Stanford (joey) wrote :

Have sent another touch base email to SF today. Should have a
response in a day or so.

Revision history for this message
Joey Stanford (joey) wrote :

From SF:

Date: Fri, 29 Aug 2008 09:08:57 -0400

We're gearing up for a test of some additional NAS gear (which is the dependency for opening SVN access to you again). Hoping for that to wrap-up shortly after Labor Day. I'll be back in touch at that time with a more in-depth reply to the info you've provided here.

Revision history for this message
Treeform (starplant) wrote :

what about cvs? That's the beast i want.

Revision history for this message
Joey Stanford (joey) wrote :

Sourceforge has released their latest migration schedule. It appears this will linger a few more week.

MIGRATION SCHEDULE

The SourceForge.net Service Operations team is now actively working to
complete our migration from our old California datacenter to our new
Chicago datacenter. The following migration events have been scheduled:

* On Monday, 2008-09-15, SourceForge.net user account mail aliases
(@users.sourceforge.net accounts) will be migrated as per:
http://sourceforge.net/community/forum/topic.php?id=3388&page

* On Monday, 2008-09-15, Project Web service for those projects already
running in Chicago (the "bigprojects" pool) will have a brief downtime
as we move data on to new storage hardware as per:
http://sourceforge.net/community/forum/topic.php?id=3411&page&replies=1

* On Tuesday, 2008-09-16, the remainder of Project Web will be migrated
(with some downtime) as per:
http://sourceforge.net/community/forum/topic.php?id=3410&page&replies=1

* Tentatively, on Wednesday, 2008-09-17, CVS service will be migrated
(with some downtime) as per:
http://sourceforge.net/community/forum/topic.php?id=3413&page

* On Thursday, 2008-09-18, Mailing List service will be migrated (no
downtime anticipated), including Mailing List archives and Mailing List
search, as per:
http://sourceforge.net/community/forum/topic.php?id=3412&page

Revision history for this message
Joey Stanford (joey) wrote :

Given the latest schedule from Sourceforge, this won't make the release next week. I'm pushing it out again until 2.1.10. I know the SF crew is working hard to complete their move.

Changed in launchpad-bazaar:
milestone: 2.1.9 → 2.1.10
Revision history for this message
Joey Stanford (joey) wrote :

Emailed the SF crew again today.

Revision history for this message
Joey Stanford (joey) wrote :

There is a plan to fix this as of today. Next week maybe.

Revision history for this message
Joey Stanford (joey) wrote :

Today from Sourceforge: "Blocks have been removed fully, as far as I can see. Please go ahead and resume activity."

Changed in launchpad-bazaar:
status: In Progress → Fix Released
Revision history for this message
Joey Stanford (joey) wrote :

I confirmed that we can do checkouts now on our import machines however the automated process is still failing.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

It was just one import that was failing for slightly obscure reasons. The vast majority are working fine again.

Revision history for this message
Treeform (starplant) wrote :
Revision history for this message
Treeform (starplant) wrote :

Oh sorry i just noticed its failing for a different reason. I'll post that as a new bug.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.