Roles of the manifest and signature files

On 10.02.2013 12:35, nils wrote:
> New question #221509 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/221509
>
> Good morning,
> I'm using duplciity to perform a backup to S3 and am now considering to use the Amazon Glacier option as this could save quite some money....
>
> I just notided however that if I'd put a rule on the S3 bucket to archive the content to Glacier this will not only affect the data itself but also the manifest and signature files.
>
> So I was wondering whether you could provide me with any information on the role of those files. Specifically I' like to understand under which circumstances duplicity will do a read of those files on the remote side (as this would not longer be possible if the data is archived on Glacier).
>

just answered it here last year
https://lists.gnu.org/archive/html/duplicity-talk/2012-12/msg00081.html

make sure your signatures stay available and you should be set. of course you wont be able to verify/restore without the volumes and their manifests.

..ede/duply.net

Are you sure that the signatures have to be available rather than the manifest? Reading the other answer makes me think that the manifest is actually the important file.

I also just started a try and get and error when doing an incremental backup saying: BackendException: Error downloading [manifestfile].

Thanks
Nils

read carefully.. citing myself from the mailing list link above

"the signature is pretty much optional. it's needed only for file listing. ...

essential is the manifest."

..ede/duply.net

Thanks for clarification. Makes more sense for me now ;-)

However, it brings me to another question: Is there any way to change the name that duplicity uses for the manifest? The background is that the glacier rules @Amazon only work with prefixes of the file names. And as the manifest and the actual archives start with the same naming, it is currently not possible to leave out the manifest when archiving to Glacier....

Thanks
Nils

On 13.02.2013 13:55, nils wrote:
> Question #221509 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/221509
>
> Status: Answered => Open
>
> nils is still having a problem:
> Thanks for clarification. Makes more sense for me now ;-)
>
> However, it brings me to another question: Is there any way to change
> the name that duplicity uses for the manifest? The background is that
> the glacier rules @Amazon only work with prefixes of the file names. And
> as the manifest and the actual archives start with the same naming, it
> is currently not possible to leave out the manifest when archiving to
> Glacier....
>

sorry no.. i am also unsure if patching duplicity regarding that would be worth the effort. obviously s3 is missing a crucial feature here.
https://forums.aws.amazon.com/thread.jspa?messageID=423435

..ede/duply.net

I understand and agree that this is a feature that should primarily be provided by S3. However, I'd doubt that Amazon has this high on their list. So I'd really appreciate such a feature as it would allow me to use Glacier and to complete my online backup (which would be too expensive on S3).

If there is anything I could do to support the development of such a feature, just let me know.

Nils

On 13.02.2013 21:25, nils wrote:
> Question #221509 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/221509
>
> Status: Answered => Solved
>
> nils confirmed that the question is solved:
> I understand and agree that this is a feature that should primarily be
> provided by S3. However, I'd doubt that Amazon has this high on their
> list. So I'd really appreciate such a feature as it would allow me to
> use Glacier and to complete my online backup (which would be too
> expensive on S3).
>
> If there is anything I could do to support the development of such a
> feature, just let me know.
>

well, you could
a. implement it yourself
b. offer a bounty
c. simply manually keep copies of the signatures somewhere else and point duplicity there for listings of past backups

..ede/duply.net

I had a "quick" look at the code and realized that duplicity is doing a lot of it logic based on the names of the files. So for me, implementing it would not work within reasonable time.

I guess I have to look for another soluting then.

Thanks for your support

On 15.02.2013 16:35, nils wrote:
> Question #221509 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/221509
>
> nils posted a new comment:
> I had a "quick" look at the code and realized that duplicity is doing a
> lot of it logic based on the names of the files. So for me, implementing
> it would not work within reasonable time.
>
> I guess I have to look for another soluting then.

yeah.. exactly my point

> Thanks for your support
>

nP .. ede/duply.net

Not sure if this is of interest for anybody here in the forum but I eventually found a workaround to get duplicity working with Glacier: http://blog.epsilontik.de/?page_id=68

Rgds
Nils