gpg encryption process pegs a single core

Asked by Peter Dyson on 2016-07-08

Can multiple gpg processes be run concurrently so that all cpu cores are used rather than just one?

I realise if this is one large encrypted stream then you're stuck with whatever gpg and the algorithm it's using are capable of.
Is there any chance to design duplicity so that multiple streams can be processed simultaneously to take advantage of modern multicore processors.

Thanks.

I'm using the following versions:
$ duplicity --version
duplicity 0.7.07.1
$ uname -a
mycomputer 15.5.0 Darwin Kernel Version 15.5.0: Tue Apr 19 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64 x86_64

Question information

Language:
English Edit question
Status:
Answered
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Last query:
2016-07-08
Last reply:
2016-07-08
edso (ed.so) said : #1

On 08.07.2016 03:37, Peter Dyson wrote:
> New question #296122 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/296122
>
> Can multiple gpg processes be run concurrently so that all cpu cores are used rather than just one?
>
> I realise if this is one large encrypted stream then you're stuck with whatever gpg and the algorithm it's using are capable of.
> Is there any chance to design duplicity so that multiple streams can be processed simultaneously to take advantage of modern multicore processors.
>

that's mainly a gpg issue. gpg does not support multithreading and probably never will. you can search the web about the why.

to implement what your asking in duplicity we would have to create volumes in temp space and only thereafter encrypt them in parallel. the current design however is to stream all data and cutting chunks whenever the volume size is reached. this way no extra temp space is used but yes it is not possible to parallelize as well.

..regards ede/duply.net

Marc (marc32111) said : #2

it even got worse with gpg v2: you cannot even run multiple gpg v2 instances in parallel because they all lock the gpg-agent which is now doing all the work........ maybe we should look for an alternative when doing mass encryption.

Can you help with this problem?

Provide an answer of your own, or ask Peter Dyson for more information if necessary.

To post a message you must log in.