Comment 4 for bug 1734144

Revision history for this message
Joris van Eijden (joris-vaneijden) wrote :

Code path:
 write_multivol(backup_type, tarblock_iter, man_outfp, sig_outfp, backend)
creates a local file tdp and then calls
 put(tdp, dest_filename, vol_num)
which first gets the local file's size
 putsize = tdp.getsize()
then tells the backend to store it
 backend.put(tdp, dest_filename)
then calls this function
 validate_block(putsize, dest_filename)
which gets the remote filesize
 info = backend.query_info([dest_filename])[dest_filename]
 size = info['size']
and compares it with the local one
 if size != orig_size:
and throws a fatal error if they don't match
 code_extra = "%s %d %d" % (util.escape(dest_filename), orig_size, size)
 log.FatalError(_("File %s was corrupted during upload.") % util.ufn(dest_filename),
                log.ErrorCode.volume_wrong_size, code_extra)

It does not retry transfering the file since the retry mechanism is inside `backend.put()`.

The swift client uses the 'content-length' response header to a head request for the file as the remote size.

So there's 2 possible scenarios happening. Either the objectstore actually failed to write the file completely while still returning an http code >= 200 and < 300, or the objectstore returns an inaccurate 'content-length' header.

Both cases seem to indicate something wrong with the objectstore. Perhaps adding retries to work around that is not the most prudent way forward.