Import bugs from Trac for Dosage

Asked by Tristan Seligmann on 2009-10-11

We would like to start using Launchpad for bug tracking for Dosage. The launchpad project already exists: https://launchpad.net/dosage. I have generated (with a little coaxing) an XML export of the current Trac database, which is available here: http://slipgate.za.net/~mithrandi/dosage.xml

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
Graham Binns Edit question
Solved by:
Gavin Panella
Solved:
2009-11-13
Last query:
2009-11-13
Last reply:
2009-11-13
Whiteboard:
Will proceed with EC2 import on 2009-10-13
Graham Binns (gmb) said : #1

Hi Tristan,

Thanks for generating the XML. The way it generally works from here is as follows:

 1. We set up a standalone LP instance and import your bugs to it so you can check them over and make sure that they look right. This is a simple way of sanity-checking the data.
 2. We do a test run of the import on staging.launchpad.net to make sure that things should go smoothly when importing to production. If possible, we ask you to check the results of this, too.
 3. We do the import on production.

If this sounds okay to you I'll proceed with step one tomorrow. Just let me know.

Regards,

Graham

Tristan Seligmann (mithrandi) said : #2

That sounds great, please go ahead.

Tristan Seligmann (mithrandi) said : #3

Any news on this?

Graham Binns (gmb) said : #4

2009/10/16 Tristan Seligmann <email address hidden>:
> Question #85438 on Launchpad Bugs changed:
> https://answers.edge.launchpad.net/malone/+question/85438
>
> Tristan Seligmann posted a new comment:
> Any news on this?

Hi Tristan,

Apologies for the delay on this; I've been off sick so I haven't been
able to attend to it. I'll have the test instance up for you by the
end of Friday, UK time.

Graham Binns (gmb) said : #5

I've emailed you privately about the test import.

Graham Binns (gmb) said : #6

Just checking that you received my latest email about performing the export from Trac using the tip of http://code.launchpad.net/trac-launchpad-migrator/trunk.

I'll be handing this import over to a colleague as I'm unavailable for the next week. They'll be in touch in due course.

Regards,

Graham

Gavin Panella (allenap) said : #7

Hi Tristan,

Graham has handed this over to me. When you're able, please send your latest bug export over to me, or reply to this question with a link to the export.

Thanks!

Gavin.

Tristan Seligmann (mithrandi) said : #8

I've sent you the latest export by email (hope I got the address right!).

Gavin Panella (allenap) said : #9

Yes, correct address :) It all looks good. I'm trying to get it imported into staging. The sys admins are extremely busy right now, preparing for the Karmic release, but I hope they'll be able to get it done today or tomorrow.

Gavin Panella (allenap) said : #10

Hi Tristan,

The Dosage bugs have been imported into staging, at:

  https://bugs.staging.launchpad.net/dosage

Please give them a quick sanity check before we load them into
production. Staging is wiped daily, so the import will not appear
tomorrow.

** I saw on http://trac.slipgate.za.net/dosage/query that there are 74
** bugs for Dosage. However, there are only 44 in the export XML you
** sent. Is this right?

Note that I've changed the bug nicknames to be of the form
"dosage-${trac-bug-id}". This means that you can set up a redirect
from your Trac installation (using Apache or whatever) to Launchpad in
such a way that users will be redirected to the correct bug in
Launchpad.

For example, if someone goes to:
  http://trac.slipgate.za.net/dosage/ticket/41

They could be redirected to (this would be your responsibility):
  https://bugs.launchpad.net/bugs/dosage-41

Which would then redirect them to:
  https://bugs.launchpad.net/dosage/+bug/654321

Obviously, bug #654321 doesn't exist yet; we won't know the
corresponding bug number in Launchpad until we've done the import into
production.

You can try the above out on staging now:
  https://bugs.staging.launchpad.net/bugs/dosage-41

Gavin.

Tristan Seligmann (mithrandi) said : #11

On Wed, Oct 28, 2009 at 6:06 PM, Gavin Panella
<email address hidden> wrote:
> The Dosage bugs have been imported into staging, at:
>
>  https://bugs.staging.launchpad.net/dosage

I'm getting an error trying to view this page, OOPS-1398S10. I was
able to work around this by bypassing this page, but I thought you
should know about it.

> ** I saw on http://trac.slipgate.za.net/dosage/query that there are 74
> ** bugs for Dosage. However, there are only 44 in the export XML you
> ** sent. Is this right?

By my count, there actually ought to be 375 bugs. It appears the
reporter is finding all of them, but skipping most of them for some
reason; I'll look into this and get back to you.

> Note that I've changed the bug nicknames to be of the form
> "dosage-${trac-bug-id}". This means that you can set up a redirect
> from your Trac installation (using Apache or whatever) to Launchpad in
> such a way that users will be redirected to the correct bug in
> Launchpad.

This is great!
--
mithrandi, i Ainil en-Balandor, a faer Ambar

Tristan Seligmann (mithrandi) said : #12

On Thu, Oct 29, 2009 at 3:58 AM, Tristan Seligmann
<email address hidden> wrote:
> By my count, there actually ought to be 375 bugs. It appears the
> reporter is finding all of them, but skipping most of them for some
> reason; I'll look into this and get back to you.

Okay, I think I've resolved this issue. The new import is located here:

http://slipgate.za.net/~mithrandi/dosage-20091029.xml
--
mithrandi, i Ainil en-Balandor, a faer Ambar

Gavin Panella (allenap) said : #13

> > https://bugs.staging.launchpad.net/dosage
>
> I'm getting an error trying to view this page, OOPS-1398S10. I was
> able to work around this by bypassing this page, but I thought you
> should know about it.

Oh, sorry about that. Thanks for letting me know.

The good news is that I think that it is a known problem, and a fix
has been prepared for it. However, it may not be in staging yet.

> Okay, I think I've resolved this issue. The new import is located here:

I've imported all the bugs into staging without error, so have another
look and I'll get it put into production later.

Out of interest, what was the issue? I'll file a bug report if it's
something we can/should fix.

Gavin.

Gavin Panella (allenap) said : #14

I forgot to mention that you can work around the OOPS you saw by logging in first (assuming it's the same issue that I'm thinking about).

Tristan Seligmann (mithrandi) said : #15

On Fri, Oct 30, 2009 at 12:40 PM, Gavin Panella
<email address hidden> wrote:
> Out of interest, what was the issue? I'll file a bug report if it's
> something we can/should fix.

The fix was to set TRAC_HOSTNAME and KNOWN_USERS in migrate.py so that
users without email addresses in trac (basically every user in this
database) can be migrated anyway.
--
mithrandi, i Ainil en-Balandor, a faer Ambar

Tristan Seligmann (mithrandi) said : #16

So, it looks like everything is fine; I've just checked to make sure there have been no changes since the last export I did, and I have now disabled the Trac environment so no further changes can be made there. Please go ahead with the final import on production.

Gavin Panella (allenap) said : #17

Hi Tristan,

We've imported 190 bugs into production. However, we've had issues with the remaining 205. The problems are not with the data, but on our side; the import script is not connecting to the master database, but to a read-only slave for some write operations. I'm working on a fix for this, and will get the remaining bugs imported as soon as possible. If necessary I will prepare a manual data fix to allow them to import.

It's fine to work with the other bugs, modify them, etc. so I hope that's of some use while we get this sorted out.

I'm really sorry for the inconvenience.

Gavin.

Best Gavin Panella (allenap) said : #18

Hi Tristan,

The remaining bugs have been imported. Again, I'm sorry for the delay, and I hope you have nothing but good experiences with Launchpad from here on :)

Gavin.

Tristan Seligmann (mithrandi) said : #19

Thanks Gavin Panella, that solved my question.