[UBUNTU 20.04] OpenCryptoki >= 3.13 with upgraded EP11 host library - Dilithium support not available

Default Comment by Bridge

Default Comment by Bridge

Default Comment by Bridge

debdiff for kinetic
from 3.17.0+dfsg+20220202.b40982e-0ubuntu1 to 3.17.0+dfsg+20220202.b40982e-0ubuntu2

The attachment "debdiff kinetic" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

debdiff for kinetic

Default Comment by Bridge

debdiff for jammy

debdiff for impish

Re-uploading the debdiffs due to some typos (c&p issues).

Test builds for all affected opencryptoki version are available here (for all major architectures):
https://launchpad.net/~fheimes/+archive/ubuntu/lp1973296

Uploaded to Kinetic, I'll review and upload the others tomorrow.

Hello bugproxy, or anyone else affected,

Accepted opencryptoki into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/opencryptoki/3.16.0+dfsg-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello bugproxy, or anyone else affected,

Accepted opencryptoki into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/opencryptoki/3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello bugproxy, or anyone else affected,

Accepted opencryptoki into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/opencryptoki/3.13.0+dfsg-0ubuntu5.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Accepted, but please adjust your Test Plan to actually test the previous missing *functionality* of opencryptoki+EP11+Dilithium from a user's perspective. This means something like "verify that a user can use their token to authenticate against X" rather than "verify that the library supports X".

Just checking that it's detected and _might_ work is insufficient, IMHO. Because if it turns out that a further patch is needed to make it actually work for users, we shouldn't be pushing updates multiple times to unaffected users because we didn't test the stack all the way through.

@Robie, I agree that just checking the detected and offered functionality is probably a bit shallow.
Even if it worked before (but no longer, if the latest EP11 host library is used, that's how I understand it), it makes sense the run a more thorough test involving Dilithium.

I think it could either be a manual test using a token with Dilithium, or maybe running the dedicated 'dilithium_tests' from testcases/crypto available upstream with an upstream package.
(Btw. these testcases cannot be easily automated or even scripted in our case, since the external ep11 package is needed here, and a lot of upfront token/slot preparations, default PINs setup, hw etc. - but it's possible with some manual effort - did it in the past for a different testcase ...)

I leave this here in case of any further discussions (also with IBM).

And sorry for the delay in the verification (due to pub. holidays and the complex nature of this).

------- Comment From <email address hidden> 2022-06-07 10:25 EDT-------
Successfully tested this on 20.04 using https://launchpad.net/ubuntu/+source/opencryptoki/3.13.0+dfsg-0ubuntu5.2 from -proposed.

With original package (3.13.0+dfsg-0ubuntu5.1) and EP11 host library of 3.0.1, it does show support for the CKM_IBM_DILITHIUM mechanism. It also shows a firmware version of '3.1' wit pkcsconf -t for the EP11 token. Although '3.1' is actually wrong, this is still what is expected with the original opencryptoki package version (which has a version query bug leading to this incorrectness).

Once upgraded to the new version of the EP11 host library, it now shows a firmware version of '3.0' and CKM_IBM_DILITHIUM is no longer available. This is the error that this BZ is supposed to fix.

Next I upgraded opencryptoki to 3.13.0+dfsg-0ubuntu5.2 from -proposed. With that the firmware version shown by pkcsconf is back to '3.1' and CKM_IBM_DILITHIUM is available again.

So this confirms that the opencryptoki/3.13.0+dfsg-0ubuntu5.2 fixes the problem.

Changed keyword verification-needed-focal to verification-done-focal.

Many thanks for the (API) verification!

Since this verifies that the CKM_IBM_DILITHIUM mechanism is now again available via the API,
would it also be possible to test Dilithium itself (like requested in LP comment #20: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1973296/comments/20)?
This would be good to avoid any further or follow-on surprises with Dilithium.

I think it should be ok to have an EP11 token ready to use and then running 'testcases/crypto/dilithium_tests'.

However, the testcases folder is not part of the Ubuntu package, but I have it - taken from the 3.13 upstream package - compiled on/for focal on one of my local systems, and could could share it (to avoid too much hassle).
I just cannot do this particular test by myself on our system, since I've currently no free adapter that I can move into 'EP11-Coproc' mode (the few adapters we have are heavily shared across running LPARs) and we also don't have a TKE to do the initial config (master pw). (I also believe that the IBM 'csulcca' package only allows to set master pws in case of adapters in cca mode, but not in EP11 mode - if I've read the docs correctly ...)

manually compiled upstream v3.13 on focal, incl. testcases

------- Comment From <email address hidden> 2022-06-08 09:38 EDT-------
I did not mention this explicitly in my previous comment, but of course I also did run the dilithu?um test ('testcases/crypto/dilithium_tests') during my tests to check that it really works.

Perfect - thx a lot, Ingo!

> So this confirms that the opencryptoki/3.13.0+dfsg-0ubuntu5.2 fixes the problem.

> ...I also did run the dilithu?um test ('testcases/crypto/dilithium_tests') during my tests to check that it really works.

Great - thanks!

Could you please do similar testing for 3.16.0+dfsg-0ubuntu1.1 in Impish and for 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 in Jammy, confirming that these are actually the package versions you tested?

------- Comment From <email address hidden> 2022-06-20 06:54 EDT-------
@Canonical: answer is currently pending / delayed due to vacation. Will update next week.

We are still waiting to hear about the testing for Impish and Jammy per comment #27.

------- Comment From <email address hidden> 2022-07-12 07:30 EDT-------
@Canonical: sorry for the delay. Our SME is still out due to medical conditions. With his backup being ill as well, our prediction is that we cannot start working on the verification before next week (7/18).

------- Comment From <email address hidden> 2022-07-12 10:26 EDT-------
not sure what exactly to verify here ...
however, I did an installation of ubuntu 21.10, made sure I have
all packages updated to latest version and then installed opencryptoki
root@t35lp54:~# dpkg -l | grep opencryptoki
ii libopencryptoki0:s390x 3.16.0+dfsg-0ubuntu1 s390x PKCS#11 implementation (library)
ii opencryptoki 3.16.0+dfsg-0ubuntu1 s390x PKCS#11 implementation (daemon)
and the ep11 library
root@t35lp54:~# dpkg -l | grep ep11
ii libep11 3.0.1-1 s390x IBM Z Enterprise PKCS #11 Support Program

Then configured opencryptoki and tried to find the dilitium mechanism but it isn't there. A simple pkcsconf -c 3 -m does not show CKM_IBM_DILITHIUM or similar.

I did this on a z15 with exact one CEX7 card in EP11 mode.

So what did i do wrong here ?

------- Comment From <email address hidden> 2022-07-12 10:49 EDT-------
same as above but I used a newer EP11 lib:
root@t35lp54:~# dpkg -l | grep ep11
ii libep11 3.1.0-5 s390x IBM Z Enterprise PKCS #11 Support Program

but same result - DILITHIUM is not listed as a valid mechanism.

Hi Harald, thx for taking a look at this.
So you still have the old package in use, which is '3.16.0+dfsg-0ubuntu1',
but the patched package is '3.16.0+dfsg-0ubuntu1.1' (please notice the trailing ".1")which is as of today still in "-proposed".
(It will actually be moved from -proposed to -updates if the verification is successful.)

Hereare all the version in the different pockets (impish-proposed / jammy-proposed):
$ rmadison opencryptoki --arch=s390x
opencryptoki | 3.16.0+dfsg-0ubuntu1 | impish/universe | s390x
opencryptoki | 3.16.0+dfsg-0ubuntu1.1 | impish-proposed/universe | s390x
opencryptoki | 3.17.0+dfsg+20220202.b40982e-0ubuntu1 | jammy/universe | s390x
opencryptoki | 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 | jammy-proposed/universe | s390x
opencryptoki | 3.17.0+dfsg+20220202.b40982e-0ubuntu2 | kinetic/universe | s390x

Since yo are on impish/21.10 you would need to install '3.16.0+dfsg-0ubuntu1.1'.
And '3.17.0+dfsg+20220202.b40982e-0ubuntu1.1' on jammy/22.04.

Here is how to install packages from "-proposed":
$ sudo apt -y -q install software-properties-common   # 'software-properties-common' is usually already installed on newer Ubuntu releases
$ sudo add-apt-repository -y "deb http://us.ports.ubuntu.com/ubuntu-ports/ $(lsb_release -sc)-proposed restricted main universe"   # to activate -proposed for the pockets 'the' the package belongs to (here activated for main and universe)
$ sudo apt update   # update archive index, should nowadays be automatically triggered by the previous command
$ sudo apt install opencryptoki # to update the package to the latest version available
( or if you want to be more careful install by explicitly pointing to the right version:
$ sudo apt install opencryptoki=3.16.0+dfsg-0ubuntu1.1 )

'apt cache-policy opencryptoki' gives an idea about which package is available, which is installed and where did it come from.

And it's great to test the API so that Dilithium is listed, but the request was to make sure that it also really works, like: https://bugs.launchpad.net/ubuntu/+source/opencryptoki/+bug/1973296/comments/25

------- Comment From <email address hidden> 2022-07-13 02:29 EDT-------
Ok, did as described and now I have hopefully the 'right' ock:

root@t35lp54:~# dpkg -l | grep opencryptoki
ii libopencryptoki0:s390x 3.16.0+dfsg-0ubuntu1.1 s390x PKCS#11 implementation (library)
ii opencryptoki

Rebooted, ldconfiged but still same result. pkcsconf -c 3 -m does not list anything related to DILITHIUM. Please note that ep11info clearly lists CKM_IBM_DILITHIUM.

And of course I understood your request to test not only the listing of the Dilithium mechanism but also test it. But when ock does not even list the mechanism there is no way to run a successful test, or ?

Hmm, there is either s/t still missing in the opencryptoki version of impish (maybe more pre-req commits than only b40982e1 ?), or this is caused by the EP11 package (what I don't believe).
We picked up the patches like requested.

Well, would you mind giving it a try on jammy/22.04 (in this case of course with version '3.17.0+dfsg+20220202.b40982e-0ubuntu1.1' - and an updated EP11 package due to openssl 3)?
Since jammy's opencryptoki version is much never and it's even more important to check if it works on LTS (and since impish - as non-LTS - reaches it's EOL soon anyway).

And yes, checking the API by listing the Dilithium mechanism is surely the 1st step that needs to work - and if that fails trying to test the functionality is pointless. It wasn't meant disparaging, I just mentioned it to be sure, since sometimes not all comments are synched between BZ and LP (and there are already a lot)...

------- Comment From <email address hidden> 2022-07-13 03:38 EDT-------
Arg, I had two EP11 slots ... forget my previous comment.
I see the export CKM_IBM_DILITHIUM mechanism now. So let's test ...

------- Comment From <email address hidden> 2022-07-13 04:33 EDT-------
Ok, test done.
The dilitium test runs successful on
Ubuntu 21.10
with these packages:
ii libopencryptoki0:s390x 3.16.0+dfsg-0ubuntu1.1 s390x PKCS#11 implementation (library)
ii opencryptoki 3.16.0+dfsg-0ubuntu1.1 s390x PKCS#11 implementation (daemon)
ii libep11 3.1.0-5 s390x IBM Z Enterprise PKCS #11 Support Program

------- Comment From <email address hidden> 2022-07-13 06:00 EDT-------
And now I am about to test for 22.04.
I updated to 22.04 and installed the proposed ock stuff:
dpkg -l | grep opencryptoki
ii libopencryptoki0:s390x 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 s390x PKCS#11 implementation (library)
ii opencryptoki 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 s390x PKCS#11 implementation (daemon)

but already a pkcsconf -t fails
# pkcsconf -t
Error initializing the PKCS11 library: 0x5 (CKR_GENERAL_ERROR)

when traces are active I see:

07/13/2022 11:51:59 752 [usr/lib/common/trace.c:210 api] INFO: **** OCK Trace level 4 activated for OCK version 3.17.0 ****
07/13/2022 11:51:59 752 [usr/lib/api/api_interface.c:2875 api] INFO: C_Initialize
07/13/2022 11:51:59 752 [usr/lib/api/policy.c:1666 api] ERROR: Failed to open /etc/opencryptoki/strength.conf: No such file or directory
07/13/2022 11:51:59 752 [usr/lib/api/api_interface.c:3092 api] ERROR: Policy loading failed! rc=0x5
07/13/2022 11:51:59 752 [usr/lib/api/api_interface.c:1656 api] INFO: C_Finalize
07/13/2022 11:51:59 752 [usr/lib/api/api_interface.c:1658 api] ERROR: API not initialized

looks like here is this policy stuff active but not all the required files have been packaged.

------- Comment From <email address hidden> 2022-07-13 06:09 EDT-------
This failure is gone when I copy the file strength-example.conf from opencryptoki/doc to /etc/strength.conf and chmod 640 strength.conf and chown root.pkcsconf strength.conf

------- Comment From <email address hidden> 2022-07-13 06:28 EDT-------
When the above fix is done I can successfully run the dilitium tests on
Ubuntu 22.04 LTS
with these packages:
dpkg -l | grep opencryptoki
ii libopencryptoki0:s390x 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 s390x PKCS#11 implementation (library)
ii opencryptoki 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1 s390x PKCS#11 implementation (daemon)

Great! Many thx for testing these add. two cases - much appreciated!

A "strength.conf" file is neither incl. in the opencryptoki packages (other than as strength-example.conf) nor copied over or renamed.
I just double-checked it with the non-proposed packages in focal, impish and jammy, as well with the -proposed packages for these releases - so that's just like it is and no regression.

Since it's just a sample (that may not fit for all use cases), one needs to copy it manually, like mentioned in the header of strength-example.conf:
"
# Move/copy to /etc/opencryptoki/strength.cfg to use it with opencryptoki.
# Then chown it to root:pkcs11 and chmod it to 0640.
"

(
It's btw. a similar approach that we follow like with '/usr/share/doc/openssl-ibmca/examples/openssl.cnf.sample' in package 'openssl-ibmca' that one needs to copy, too.

I remember that we once discussed to change this, but the discussion went nowhere.)

With that, let me thank you again, and I consider this as successfully verified for focal, impish and jammy.

(A potential change on how to handle 'strength.conf' in a different way could be discussed as part of a separate ticket ...)

This bug was fixed in the package opencryptoki - 3.16.0+dfsg-0ubuntu1.1

---------------
opencryptoki (3.16.0+dfsg-0ubuntu1.1) impish; urgency=medium

  * d/p/b40982e1-EP11-Dilithium-Specify-OID-of-key-strength-at-key-ge.patch
    and d/p/6759faed-EP11-Fix-host-library-version-query.patch to
    fix unavailability of Dilithium support in OpenCryptoki >= 3.13
    with upgraded EP11 host library
    Thanks to Ingo Franzki (LP: #1973296)

 -- Frank Heimes <email address hidden> Tue, 17 May 2022 19:31:04 +0200

The verification of the Stable Release Update for opencryptoki has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package opencryptoki - 3.13.0+dfsg-0ubuntu5.2

---------------
opencryptoki (3.13.0+dfsg-0ubuntu5.2) focal; urgency=medium

  * d/p/b40982e1-EP11-Dilithium-Specify-OID-of-key-strength-at-key-ge.patch
    and d/p/6759faed-EP11-Fix-host-library-version-query.patch to
    fix unavailability of Dilithium support in OpenCryptoki >= 3.13
    with upgraded EP11 host library
    Thanks to Ingo Franzki (LP: #1973296)

 -- Frank Heimes <email address hidden> Wed, 18 May 2022 10:38:07 +0200

This bug was fixed in the package opencryptoki - 3.17.0+dfsg+20220202.b40982e-0ubuntu1.1

---------------
opencryptoki (3.17.0+dfsg+20220202.b40982e-0ubuntu1.1) jammy; urgency=medium

  * d/p/6759faed-EP11-Fix-host-library-version-query.patch
    fix unavailability of Dilithium support in OpenCryptoki >= 3.13
    with upgraded EP11 host library
    Thanks to Ingo Franzki (LP: #1973296)

 -- Frank Heimes <email address hidden> Mon, 16 May 2022 13:24:15 +0200

------- Comment From <email address hidden> 2022-07-18 09:08 EDT-------
Is there anything I still need to do for this BZ?

Nothing needed anymore on this, since Harald was so kind doing the jammy verification meanwhile (but thx for checking).

This bug is in status Fix Released, hence it's closed as done.

(There is a side conversation about how to handle the strength.conf file, but - depending on the outcome - requires a new LP bug anyway.)