The Diffie-Hellman algorithm "modp6144" is not permitted in FIPS mode of Ubuntu strongswan, right?

Asked by Pei Pei Xiao

I have a Ubuntu server, and install the strongswan package with FIPS mode enabled.

1. check the ubuntu version
# cat /proc/version
Linux version 5.4.0-1072-fips (buildd@lcy02-amd64-020) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #81-Ubuntu SMP Wed Jan 25 11:07:24 UTC 2023

2. check the strongswan ipsec version
# ipsec version
Linux strongSwan U5.8.2/K5.4.0-1072-fips
University of Applied Sciences Rapperswil, Switzerland
See 'ipsec --copyright' for copyright information.

3. enable fips in ubuntu server
# sudo ua status
SERVICE ENTITLED STATUS DESCRIPTION
esm-apps yes enabled Expanded Security Maintenance for Applications
esm-infra yes enabled Expanded Security Maintenance for Infrastructure
fips-updates yes enabled NIST-certified core packages with priority security updates
livepatch yes enabled Canonical Livepatch service
usg yes disabled Security compliance and audit tools

NOTICES
A change has been detected in your contract.
Please run `sudo pro refresh`.

Enable services with: pro enable <service>

     Account: sky9090******
Subscription: Ubuntu Pro - free personal subscription

4. when I set the IKE Diffie-Hellman group to `modp6144`, the connection failed to be established.
conn all
       type=tunnel
       auto=start
       esp=aes256-sha256!
       ike=aes256-sha256-modp6144!
       left=%any
       ......
       ......
       leftauth=psk
       rightauth=psk
       keyexchange=ikev2
       lifetime=10800s
       ikelifetime=36000s
       dpddelay=30s
       dpdaction=restart
       dpdtimeout=120s

5. check the ipsec status, and see the connection is not up
# ipsec statusall
Status of IKE charon daemon (strongSwan 5.8.2, Linux 5.4.0-1072-fips, x86_64):
  uptime: 9 minutes, since Feb 13 06:06:00 2023
  malloc: sbrk 2969600, mmap 0, used 835264, free 2134336
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 0
  loaded plugins: charon test-vectors ldap pkcs11 tpm mgf1 rdrand nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf curve25519 agent chapoly ntru curl attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock unity counters
Listening IP addresses:
  10.240.4.5
Connections:
Security Associations (0 up, 0 connecting):
  none

6. the following message was found in the log, and it says the algorithm `modp6144` is not permitted in FIPS mode.
Feb 13 06:06:00 ppei-vsi-001 charon: 05[CFG] algorithm 'modp6144' not permitted in fips mode
Feb 13 06:06:00 ppei-vsi-001 charon: 05[CFG] skipped invalid proposal string: aes256-sha512-modp6144

7. but I checked some NIST documentation and the DH algorithm `modp6144` is listed in the Approved Algorithms provided by the bound OpenSSL module. Therefore, based on my test results, I am not sure of the reason why the DH algorithm `modp6144` is not permitted in FIPS mode of Ubuntu strongswan.
https://csrc.nist.gov/CSRC/media/projects/cryptographic-module-validation-program/documents/security-policies/140sp3648.pdf (Table 9 – Approved Algorithms provided by the bound OpenSSL module)
https://csrc.nist.gov/CSRC/media/projects/cryptographic-module-validation-program/documents/security-policies/140sp4046.pdf (Table 8 – Approved and Allowed Algorithms provided by the bound OpenSSL module)

I also reported the issue on strongswan github: https://github.com/strongswan/strongswan/discussions/1537

Question information

Language:
English Edit question
Status:
Open
For:
Ubuntu strongswan Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:

Can you help with this problem?

Provide an answer of your own, or ask Pei Pei Xiao for more information if necessary.

To post a message you must log in.