Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

No I see that it is even worse:
It asks for password ONCE, then asks for "passphrase for signing key" (ONCE).
I can provide two different values, but they both doesn't matter. I'm able to decrypt *.gpg files without entering any passwords.
Who can explain it?

On 9/15/2017 22:14, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
>
> ardabro gave more information on the question:
> No I see that it is even worse:
> It asks for password ONCE, then asks for "passphrase for signing key" (ONCE).
> I can provide two different values, but they both doesn't matter. I'm able to decrypt *.gpg files without entering any passwords.
> Who can explain it?
>

that makes no sense..

1. what is your duplicity version?

2. what is your duplicity cmd line?

3. what is the complete terminal output? (obfuscate anything deemed private), zip&attach it if big

..ede/duply.net

On 9/15/2017 22:04, ardabro wrote:
> New question #658091 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/658091
>
> Hi
> When i use --encrypt-key=######## parameter everything is OK. duplicity uses my key and doesn't ask for encryption password. But when I use it with both: --encrypt-key=######## --sign-key=####### it asks for encryption password again!
> Why is it so? I just want to sign files with gpg. What is this password for?
>
>

because signing uses your private key along w/ your passphrase to validate the message you've sent! everybody might encrypt against your public key but only you can sign w/ your private key! the magic of asymmetric encryption.

..ede/duply.net

OK
I'm Sorry, my mistake and additional confusion from gpg agent and cached keys.
Everything is ok with gpg and its keys/passwords but I still have no
idea why duplicity asks for decryption password when encryption key is used.
It asks only once (second confusion) and does not use it at all -
I'm able to decrypt with gpg key without entering any password (third confusion)

....
Main action: full
================================================================================
duplicity 0.7.11 (December 31, 2016)
Args: /usr/bin/duplicity full --name=test_backup --encrypt-key=DEADBEEF --sign-key=DEADBEEF /home/ard/temp/duplicity_test/src file:///home/ard/temp/duplicity_test/dst
Linux t430-deb 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) x86_64
/usr/bin/python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118]
================================================================================
....
Last full backup date: none
PASSPHRASE variable not set, asking user.
GnuPG passphrase for decryption: # "12345" - doesn't matter
PASSPHRASE variable not set, asking user.
GnuPG passphrase for signing key: # correct - my real key-wrapping password is reqiured
....
--------------[ Backup Statistics ]--------------
StartTime 1505562881.09 (Sat Sep 16 13:54:41 2017)
....
TotalDestinationSizeChange 912 (912 bytes)
Errors 0
-------------------------------------------------

>cd /home/ard/temp/duplicity_test/dst
>ls -1
duplicity-full.20170916T115427Z.manifest.gpg
duplicity-full.20170916T115427Z.vol1.difftar.gpg
duplicity-full-signatures.20170916T115427Z.sigtar.gpg

>gpg --output xxx --decrypt duplicity-full-signatures.20170916T115427Z.sigtar.gpg
Please enter the passphrase to unlock the OpenPGP secret key:
"ard <email address hidden>"
2048-bit RSA key, ID ****************,
created 2017-07-29 (main key ID ****************).

Passphrase:
gpg: encrypted with 2048-bit RSA key, ID ****************, created 2017-07-29
      "ard <email address hidden>"
gpg: Signature made Sat 16 Sep 2017 13:54:41 CEST
gpg: using RSA key ****************************************
gpg: Good signature from "ard <email address hidden>" [ultimate]

It asks for wrapping password for my keys; this is ok;
and does NOT ask for this confusing duplicity password ("12345")
I think it is a bug.

ardabro,

your command line looks like
> Args: /usr/bin/duplicity full --name=test_backup --encrypt-key=DEADBEEF --sign-key=DEADBEEF /home/ard/temp/duplicity_test/src file:///home/ard/temp/duplicity_test/dst

so you
1. sign
&
2. encrypt

when signing and encryption key are identical, the decryption passphrase is (re)used for signing.

even when not signing, duplicity will ask the passphrase as a precaution, it _might_ need to decrypt files for the repository (on resume, archive folder sync).

> I'm able to decrypt with gpg key without entering any password (third confusion)

that cannot be right, unless your private key is not protected by a passphrase.

please read up on how gpg and asymmetrical encryption work in general. the duplicity answer section is not the proper place to educate you in this regard.

have fun ..ede/duply.net

On 9/16/2017 14:23, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
>
> Status: Answered => Open
>
> ardabro is still having a problem:
> OK
> I'm Sorry, my mistake and additional confusion from gpg agent and cached keys.
> Everything is ok with gpg and its keys/passwords but I still have no
> idea why duplicity asks for decryption password when encryption key is used.
> It asks only once (second confusion) and does not use it at all -
> I'm able to decrypt with gpg key without entering any password (third confusion)
>
> ....
> Main action: full
> ================================================================================
> duplicity 0.7.11 (December 31, 2016)
> Args: /usr/bin/duplicity full --name=test_backup --encrypt-key=DEADBEEF --sign-key=DEADBEEF /home/ard/temp/duplicity_test/src file:///home/ard/temp/duplicity_test/dst
> Linux t430-deb 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) x86_64
> /usr/bin/python 2.7.13 (default, Jan 19 2017, 14:48:08)
> [GCC 6.3.0 20170118]
> ================================================================================
> ....
> Last full backup date: none
> PASSPHRASE variable not set, asking user.
> GnuPG passphrase for decryption: # "12345" - doesn't matter
> PASSPHRASE variable not set, asking user.
> GnuPG passphrase for signing key: # correct - my real key-wrapping password is reqiured
> ....
> --------------[ Backup Statistics ]--------------
> StartTime 1505562881.09 (Sat Sep 16 13:54:41 2017)
> ....
> TotalDestinationSizeChange 912 (912 bytes)
> Errors 0
> -------------------------------------------------
>
>> cd /home/ard/temp/duplicity_test/dst
>> ls -1
> duplicity-full.20170916T115427Z.manifest.gpg
> duplicity-full.20170916T115427Z.vol1.difftar.gpg
> duplicity-full-signatures.20170916T115427Z.sigtar.gpg
>
>> gpg --output xxx --decrypt duplicity-full-signatures.20170916T115427Z.sigtar.gpg
> Please enter the passphrase to unlock the OpenPGP secret key:
> "ard <email address hidden>"
> 2048-bit RSA key, ID ****************,
> created 2017-07-29 (main key ID ****************).
>
> Passphrase:
> gpg: encrypted with 2048-bit RSA key, ID ****************, created 2017-07-29
> "ard <email address hidden>"
> gpg: Signature made Sat 16 Sep 2017 13:54:41 CEST
> gpg: using RSA key ****************************************
> gpg: Good signature from "ard <email address hidden>" [ultimate]
>
> It asks for wrapping password for my keys; this is ok;
> and does NOT ask for this confusing duplicity password ("12345")
> I think it is a bug.
>

Sorry, maybe my English is too bad. Also I see I introduced a bit mess in this line:
"I'm able to decrypt with gpg key without entering any password (third confusion)"
This is false. It asks for password that secures my GPG key(s), and ONLY for this one.
Actually it asks everytime because I have gpg keys caching disabled.
So, when decrypting, the behaviour is 100% CORRECT for scenario with gpg keys.
The problem occurs when backup is created.

Once again the whole case (PLEASE, READ CAREFULLY):

1) run duplicity with optios as previously
2) provide ___ANY_RANDOM_UNNECESSARY_USELESS_PASSWORD__ after "GnuPG passphrase for decryption:" (only once, no confirmation required!)
3) provide __REAL__ password that wraps my BOTH gpg keys (it is actually the same key pair) after "GnuPG passphrase for signing key:"
4) backup is created.
5) run gpg --output xxx --decrypt duplicity-full*.gpg OR duplicity restore file://dst trg
6) provide __REAL__ password that wraps my both keys (the same one as used in p3)
7) file is decrypted or backup restored WITHOUT using this ___ANY_RANDOM_UNNECESSARY_USELESS_PASSWORD__ used in p2 !

The above scenario is 100% reproductible!

P.S.
In p2) duplicity asks the for the password with exactly the same way as it does it twice when I encrypt with password directly (no keys).
But this time it doesn't use entered value at all (why should it use it when a gpg-key is used for encryption?)

On 9/17/2017 18:09, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
>
> Status: Answered => Open
>
> ardabro is still having a problem:
> Sorry, maybe my English is too bad. Also I see I introduced a bit mess in this line:
> "I'm able to decrypt with gpg key without entering any password (third confusion)"
> This is false. It asks for password that secures my GPG key(s), and ONLY for this one.
> Actually it asks everytime because I have gpg keys caching disabled.
> So, when decrypting, the behaviour is 100% CORRECT for scenario with gpg keys.
> The problem occurs when backup is created.
>
> Once again the whole case (PLEASE, READ CAREFULLY):
>
> 1) run duplicity with optios as previously
> 2) provide ___ANY_RANDOM_UNNECESSARY_USELESS_PASSWORD__ after "GnuPG passphrase for decryption:" (only once, no confirmation required!)
> 3) provide __REAL__ password that wraps my BOTH gpg keys (it is actually the same key pair) after "GnuPG passphrase for signing key:"
> 4) backup is created.
> 5) run gpg --output xxx --decrypt duplicity-full*.gpg OR duplicity restore file://dst trg
> 6) provide __REAL__ password that wraps my both keys (the same one as used in p3)
> 7) file is decrypted or backup restored WITHOUT using this ___ANY_RANDOM_UNNECESSARY_USELESS_PASSWORD__ used in p2 !
>
> The above scenario is 100% reproductible!
>
> P.S.
> In p2) duplicity asks the for the password with exactly the same way as it does it twice when I encrypt with password directly (no keys).
> But this time it doesn't use entered value at all (why should it use it when a gpg-key is used for encryption?)
>

as i said. encryption passphrase during backup (p1-4) is always requested but only _needed_ when the backup resumes or the archive needs to be synced, because only in these cases decryption is needed during backup.

..ede/duply.net

Does it mean, that duplicity will ask me for this "unnecessary" password when I resume interrupted process?
Does it mean that during a process of backing-up some of my files are encrypted symmetrically for a while?
If it has to ask me for a password when resuming - why cannot ask me for password to my private key?
What the advantage of using temporary password over encrypting everything asymmetrically from the beginning to the end?

Sorry, but I don't grasp this idea. It looks strange.

On 9/18/2017 20:53, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
>
> Status: Answered => Open
>
> ardabro is still having a problem:
> Does it mean, that duplicity will ask me for this "unnecessary" password when I resume interrupted process?

no, it will ask you for the proper passphrase or it won't proceed, because the old encryption does not match the new encryption.

> Does it mean that during a process of backing-up some of my files are encrypted symmetrically for a while?

no.

> If it has to ask me for a password when resuming - why cannot ask me for password to my private key?

it does.

> What the advantage of using temporary password over encrypting everything asymmetrically from the beginning to the end?

there is no temp passphrase. just because you can give junk when it is not neeed does not mean the junk is used.

>
> Sorry, but I don't grasp this idea. It looks strange.

better now? ..ede/duply.net

PS: Ken?

> as i said. encryption passphrase during backup (p1-4) is always requested but only _needed_ when the backup
> resumes or the archive needs to be synced, because only in these cases decryption is needed during backup.

>> Does it mean, that duplicity will ask me for this "unnecessary" password when I resume interrupted process?

> no, it will ask you for the proper passphrase or it won't proceed, because the old encryption does not match
> the new encryption.

Sorry, but I't seems to be false.
I started backup. Gave encryption password=="12345". I iterrupted the process after 2nd volume.
Then I restarted with exactly the same commandline. It realized that the process was interrupted.
It asked for encruption password and I gave "qwerty" this time. Everything went perfectly OK.

A piece of log from 2nd run:
Local and Remote metadata are synchronized, no sync needed.
Last full backup left a partial set, restarting.
Last full backup date: Wed Sep 20 22:01:48 2017
GnuPG passphrase for decryption:
GnuPG passphrase for signing key:
RESTART: Volumes 2 to 2 failed to upload before termination.
         Restarting backup at volume 2.
Restarting after volume 1, file debian9.iso, block 1595
...

I still don't know when this password is necessary, why it asks only once (what if I make a typo?) and why it needs to verify anything with additional password when I order it to enrypt with my gpg key.

On 9/20/2017 22:23, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
>
> Status: Answered => Open
>
> ardabro is still having a problem:
>> as i said. encryption passphrase during backup (p1-4) is always requested but only _needed_ when the backup
>> resumes or the archive needs to be synced, because only in these cases decryption is needed during backup.
>
>>> Does it mean, that duplicity will ask me for this "unnecessary"
> password when I resume interrupted process?
>
>> no, it will ask you for the proper passphrase or it won't proceed, because the old encryption does not match
>> the new encryption.
>
> Sorry, but I't seems to be false.
> I started backup. Gave encryption password=="12345". I iterrupted the process after 2nd volume.
> Then I restarted with exactly the same commandline. It realized that the process was interrupted.
> It asked for encruption password and I gave "qwerty" this time. Everything went perfectly OK.
>
> A piece of log from 2nd run:
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup left a partial set, restarting.
> Last full backup date: Wed Sep 20 22:01:48 2017
> GnuPG passphrase for decryption:
> GnuPG passphrase for signing key:
> RESTART: Volumes 2 to 2 failed to upload before termination.
> Restarting backup at volume 2.
> Restarting after volume 1, file debian9.iso, block 1595
> ...
>
> I still don't know when this password is necessary, why it asks only
> once (what if I make a typo?) and why it needs to verify anything with
> additional password when I order it to enrypt with my gpg key.
>

this source code comment explains it.
  http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/bin/duplicity#L347

why the error is not raised is beyond me.

another thing you can try is doing an encrypted backup, then deleting the corresponding local archive dir (usually under ~/.cache/duplicity/).
on the next run it should try to synchronize the local w/ the remote and see the local is missing and download/decrypt some files from the remote to the local archive dir.
give the wrong passphrase this should fail.

what is your gpg version? make sure to kill the gpg-agent instance between runs to make sure that really no passphrases are cached.

..ede/duply.net

Ok, Thanks
This is good starting point. I didn't plan to debug it but I see I have no option ;)

BTW: I have password caching disabled in gpg-agent.conf and it works. I'm asked for password every time:
default-cache-ttl 0
max-cache-ttl 0
pinentry-program /usr/bin/pinentry-tty

My gpg version:
gpg (GnuPG) 2.1.18
libgcrypt 1.7.6-beta

If you look at: https://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/bin/duplicity#L145
you will see the assumptions that drive duplicity's get_passphrase(). There is no direct way in gpg to find out if a real passphrase is needed, thus the builtin assumptions for now.

As to why it needs a passphrase when we do incrementals, it's a combination of the assumptions above and the fact that sometimes the local cache gets deleted or munged by a user. Trying to second guess that is trying to second guess humans. Can't be done.