Launchpad itself

Chromium upload failure (via regular ftp, with time stamps)

Asked by Rob Savoury

Today an attempt at uploading the latest Chromium official release 122.0.6261.111 via regular ftp (rather than sftp) resulted in the same type of failure as previously reported [*] and this time here is the output with UTC timestamps:

2024-03-06T03:11:34,727366326+00:00
D: Splitting host argument out of ppa:savos.
D: Setting host argument.
Checking signature on .changes
gpg: ../chromium-browser-122.0.6261.111-0ubuntu0.18.04.1sav0_result/chromium-browser_122.0.6261.111-0ubuntu0.18.04.1sav0_source.changes: Valid signature from 2E2CC42845C9AB64
Checking signature on .dsc
gpg: ../chromium-browser-122.0.6261.111-0ubuntu0.18.04.1sav0_result/chromium-browser_122.0.6261.111-0ubuntu0.18.04.1sav0.dsc: Valid signature from 2E2CC42845C9AB64
Uploading to ppa (via ftp to ppa.launchpad.net):
  Uploading chromium-browser_122.0.6261.111-0ubuntu0.18.04.1sav0.dsc: done.
  Uploading chromium-browser_122.0.6261.111.orig.tar.xz: 3342767k/3342768k
2024-03-06T04:16:32,532599365+00:00

As can be seen the upload took a little over an hour, which is about the average I've seen for the full Chromium source tarball in recent months. Hopefully the timestamps will help to locate the failure in the server logs such that the issue with large uploads (specifically via regular ftp) in the past week or so can be resolved. Thanks.

[*] https://answers.launchpad.net/launchpad/+question/709449

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
Ines Almeida
Solved:
Last query:
Last reply:
Revision history for this message
Ines Almeida (ines-almeida) said : 		#1

Thank you for providing the timestamps! I'll look into this

Revision history for this message
Ines Almeida (ines-almeida) said : 		#2

I replicated the issue in our development environment, it seems to be due to a timeout that is currently set for 30 min.
I opened a MP to increase that timeout and will discuss with the team about it: https://code.launchpad.net/~ines-almeida/launchpad-mojo-specs/+git/launchpad-mojo-specs/+merge/461902

I'll get back to this issue once resolved

Revision history for this message
Rob Savoury (savoury1) said : 		#3

Thanks for the update. It's curious that the uploads work with sftp but take FAR longer than 30 min, but consistently work. For instance, after I submitted this report yesterday I then let the same upload run via sftp overnight. It took over 5 hours (as per the previous sftp upload I did the other day) but did not timeout, and was therefore successful. Yet looking at the merge request you linked the timeouts appear to be the same at present for both ftp and sftp (1800000 or 30 mins). So I wonder why the sftp uploads work, but the ftp ones don't?

Revision history for this message
Ines Almeida (ines-almeida) said : 		#4

That's an excellent point.

My understanding is that those timeouts are timeout for "inactivity" (maximum inactivity on the server side and on the client side), not necessarily for how long the connection is open.

ftp and sftp connections work in different ways where, for example, sftp uses 1 channel and ftp uses 2 channels. My best guess is that when uploading via sftp the one connection is never "inactive" so it doesn't timeout. While when uploading via ftp, one of the connections is inactive while the other is used to send the data, so it times out.

My understanding of ftp vs. sftp is flawed, this is just my best guess :) I'll add these details to the MP to discuss with the team, and update the MP to only increase the ftp timeout.

Thank you for pointing this out!

Revision history for this message
Rob Savoury (savoury1) said : 		#5

Got it, yes anonymous ftp of course works differently as there's no authentication going on. Your guess at an explanation as to what's happening sounds close enough to actuality, in terms of ftp vs. sftp! So it does make sense not to change the sftp timeout.

Revision history for this message
Best Ines Almeida (ines-almeida) said : 		#6

I have merged and deployed the change.
Let us know how it goes next time you deploy a large file through FTP!

Revision history for this message
Rob Savoury (savoury1) said : 		#7

Thank you and I will definitely report back when I do the next large upload through regular FTP.

Revision history for this message
Rob Savoury (savoury1) said : 		#8

Latest large upload (3.2GB) via regular FTP has worked fine! Also, it was for some reason three times faster than the usual speed, taking less than 22 minutes in total. Here is the output of my dput command with "date -uIns" at start and finish:

2024-03-17T03:09:14,617678246+00:00
D: Splitting host argument out of ppa:savos.
D: Setting host argument.
Checking signature on .changes
gpg: ../chromium-browser-122.0.6261.128-0ubuntu0.18.04.1sav0_result/chromium-browser_122.0.6261.128-0ubuntu0.18.04.1sav0_source.changes: Valid signature from 2E2CC42845C9AB64
Checking signature on .dsc
gpg: ../chromium-browser-122.0.6261.128-0ubuntu0.18.04.1sav0_result/chromium-browser_122.0.6261.128-0ubuntu0.18.04.1sav0.dsc: Valid signature from 2E2CC42845C9AB64
Uploading to ppa (via ftp to ppa.launchpad.net):
  Uploading chromium-browser_122.0.6261.128-0ubuntu0.18.04.1sav0.dsc: done.
  Uploading chromium-browser_122.0.6261.128.orig.tar.xz: done.
  Uploading chromium-browser_122.0.6261.128-0ubuntu0.18.04.1sav0.debian.tar.xz: done.
  Uploading chromium-browser_122.0.6261.128-0ubuntu0.18.04.1sav0_source.changes: done.
Successfully uploaded packages.
2024-03-17T03:30:57,277409395+00:00

Thanks to the Launchpad support team for the efforts maintaining this excellent software packaging service.

Revision history for this message
Ines Almeida (ines-almeida) said : 		#9

Happy to hear!