Ubuntu
freeipa package

PyAsn1Error when trying to renew certs

Asked by Shaav

For some reason certmonger failed to renew my certs earlier this month. I've gone through everything I was able to find online about rolling back the date to renew the certificates manually to no avail and I've been trying suggestions from <email address hidden>, also nothing that has helped yet.

On freeipa-users, they think that probably the certificate is being issued properly but then throwing an error while being parsed by pyasn1 based on the errors in the apache log (appended)

Based on that, the last thing that I tried was trying to rollback the pyasn1 version, but still got the same error.

I'm on 18.04 with 4.7.0~pre1+git20180411-2ubuntu2; it's the only server.
python-pyasn1: 0.4.2-3
python-pyasn1-modules: 0.2.1-0.2

I was going to bite-the-bullet and try upgrading to 20.04, until I saw that the server isn't *in* 20.04.

Any thoughts or suggestions would be greatly appreciated.

Apache:
-----
[Thu Oct 08 00:02:02.421838 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] ipa: ERROR: non-public: PyAsn1Error: <TagSet object at 0x7ff98039fc90 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7ff98039f8d0 tagSet <TagSet object at 0x7ff99bed4290 tags 0:0:4> encoding iso-8859-1>
[Thu Oct 08 00:02:02.421902 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] Traceback (most recent call last):
[Thu Oct 08 00:02:02.421914 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipaserver/rpcserver.py", line 367, in wsgi_execute
[Thu Oct 08 00:02:02.421925 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] result = command(*args, **options)
[Thu Oct 08 00:02:02.421935 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 450, in __call__
[Thu Oct 08 00:02:02.421972 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] return self.__do_call(*args, **options)
[Thu Oct 08 00:02:02.421989 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 478, in __do_call
[Thu Oct 08 00:02:02.422005 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] ret = self.run(*args, **options)
[Thu Oct 08 00:02:02.422021 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 800, in run
[Thu Oct 08 00:02:02.422034 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] return self.execute(*args, **options)
[Thu Oct 08 00:02:02.422048 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/cert.py", line 884, in execute
[Thu Oct 08 00:02:02.422062 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] self.obj._parse(result, all)
[Thu Oct 08 00:02:02.422072 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/cert.py", line 493, in _parse
[Thu Oct 08 00:02:02.422082 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] cert.san_general_names)
[Thu Oct 08 00:02:02.422092 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipalib/x509.py", line 318, in san_general_names
[Thu Oct 08 00:02:02.422102 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] gns = self.__pyasn1_get_san_general_names()
[Thu Oct 08 00:02:02.422112 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/ipalib/x509.py", line 350, in __pyasn1_get_san_general_names
[Thu Oct 08 00:02:02.422123 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] ext['extnValue'], asn1Spec=univ.OctetString())[0]
[Thu Oct 08 00:02:02.422133 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] File "/usr/lib/python2.7/dist-packages/pyasn1/codec/ber/decoder.py", line 1318, in __call__
[Thu Oct 08 00:02:02.422143 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] '%s not in asn1Spec: %r' % (tagSet, asn1Spec)
[Thu Oct 08 00:02:02.422153 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] PyAsn1Error: <TagSet object at 0x7ff98039fc90 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7ff98039f8d0 tagSet <TagSet object at 0x7ff99bed4290 tags 0:0:4> encoding iso-8859-1>
[Thu Oct 08 00:02:02.422713 2020] [wsgi:error] [pid 7261] [remote 10.1.5.4:58624] ipa: INFO: [xmlserver] <email address hidden>: cert_request(u'MIIDozCCAosCAQAwNDEVMBMGA1UECgwMU0lNUExZV1MuQ09NMRswGQYDVQQDExJpcGEwMS5zaW1wbHl3cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDaVNd4cdKXfUZk1lwc++sU64iYNoLn7kuN2JWYrt0smsAJrAbKBDIwsnTwmlM16xg/ioibnweTU3+0tYvTftQh3gZMy46hCzdOgyUsjsFvmJS2QklyBM2SPspaIuXJojR87D+AmfsFKAC9EO4+ZjnTRoa32UvjTNCGJwFLn7TAM26iSrWagWza717tTJHwX2Js90hR1RxEdU1TFo/3Thj3r1oBeLJYxoyh7IQeMrKYahmVAAch2KnkAgkDAzb4XNKMxOqoF1tV+pPzk9m1iGRud8lf4QmjIrAxdHM7igXTSAL6ALrD/5w+gw+RNjJmeEb2JyUAd+VJv7s/q1ZKSpQdAgMBAAGgggEoMCsGCSqGSIb3DQEJFDEeHhwAMgAwADEAOAAxADAAMgAxADAAOAAzADcAMgA0MIH4BgkqhkiG9w0BCQ4xgeowgecwfwYDVR0RAQEABHUwc6AwBgorBgEEAYI3FAIDoCIMIGtyYnRndC9TSU1QTFlXUy5DT01AU0lNUExZV1MuQ09NoD8GBisGAQUCAqA1MDOgDhsMU0lNUExZV1MuQ09NoSEwH6ADAgEBoRgwFhsGa3JidGd0GwxTSU1QTFlXUy5DT00wDAYDVR0TAQH/BAIwADAgBgNVHQ4BAQAEFgQUwwx6Pted7FRZ4JUOLne9svpuVCwwNAYJKwYBBAGCNxQCAQEABCQeIgBLAEQAQwBzAF8AUABLAEkATgBJAFQAXwBDAGUAcgB0AHMwDQYJKoZIhvcNAQELBQADggEBAH6kQREhM1h+Plpzqcn80+UO/HtExe+JQiXewyIc4CEBSvZFb7nC7bF0aAGgzV4lJQyInbBNCRJHz7J2BUctrMimdnZsL56iz3e/HHOpcAMagmlco5rpxVnvBbSSzrYrH5NQa+8FdbjLT50LP3g3MEjegIdjDG/n9+Mh6vlEhi6dAzLeRk60pqW8m4FdWYd9mjDmEm3uaC/v1sUwjKNq8XdGuu+ZICw3nTPA3/1vDAE5CB0m5g6lN1jGth8f/eLHm9DEAVUOw5b+1xYoGCwkmG8/l2Z2MgIwIxQrcKggzZV/gzOeETzF62tSjABCDZV1rIWUNdNSAfSlgpbO1krylw0=', profile_id=u'KDCs_PKINIT_Certs', <email address hidden>', add=True, version=u'2.51'): InternalError
-----

getcert list:
-----
Number of certificates and requests being tracked: 9.
Request ID '20181021083324':
 status: MONITORING
 stuck: no
 key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
 certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=IPA RA,O=MYREALM.COM
 expires: 2022-09-02 02:33:38 MDT
 key usage: digitalSignature,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command: /usr/lib/ipa/certmonger/renew_ra_cert_pre
 post-save command: /usr/lib/ipa/certmonger/renew_ra_cert
 track: yes
 auto-renew: yes
Request ID '20181021083404':
 status: MONITORING
 stuck: no
 key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
 certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=localhost
 expires: 2022-09-05 12:15:19 MDT
 key usage: digitalSignature,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command: /usr/lib/ipa/certmonger/stop_pkicad
 post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
 track: yes
 auto-renew: yes
Request ID '20181021083405':
 status: NEED_CSR_GEN_TOKEN
 stuck: yes
 key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
 certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=localhost
 expires: 2020-10-13 12:14:21 MDT
 key usage: digitalSignature,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command: /usr/lib/ipa/certmonger/stop_pkicad
 post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
 track: yes
 auto-renew: yes
Request ID '20181021083406':
 status: NEED_CSR_GEN_TOKEN
 stuck: yes
 key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
 certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=localhost
 expires: 2020-10-13 12:15:01 MDT
 key usage: digitalSignature,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command: /usr/lib/ipa/certmonger/stop_pkicad
 post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
 track: yes
 auto-renew: yes
Request ID '20181021083407':
 status: NEED_CSR_GEN_TOKEN
 stuck: yes
 key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
 certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=localhost
 expires: 2020-10-10 02:34:28 MDT
 key usage: digitalSignature,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command: /usr/lib/ipa/certmonger/stop_pkicad
 post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
 track: yes
 auto-renew: yes
Request ID '20181021083408':
 status: NEED_CSR_GEN_TOKEN
 stuck: yes
 key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
 certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=localhost
 expires: 2020-10-13 12:14:29 MDT
 key usage: digitalSignature,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command: /usr/lib/ipa/certmonger/stop_pkicad
 post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
 track: yes
 auto-renew: yes
Request ID '20181021083613':
 status: CA_UNREACHABLE
 ca-error: Server at https://ipa01.mydomain.com/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).
 stuck: no
 key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MYREALM-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MYREALM-COM/pwdfile.txt'
 certificate: type=NSSDB,location='/etc/dirsrv/slapd-MYREALM-COM',nickname='Server-Cert',token='NSS Certificate DB'
 CA: IPA
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=ipa01.mydomain.com,O=MYREALM.COM
 expires: 2020-10-21 02:36:13 MDT
 dns: ipa01.mydomain.com
 principal name: <email address hidden>
 key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command:
 post-save command: /usr/lib/ipa/certmonger/restart_dirsrv MYREALM-COM
 track: yes
 auto-renew: yes
Request ID '20181021083714':
 status: NEED_CSR_GEN_PIN
 stuck: yes
 key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA'
 certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
 CA: IPA
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=ipa01.mydomain.com,O=MYREALM.COM
 expires: 2020-10-21 02:37:17 MDT
 dns: ipa01.mydomain.com
 principal name: <email address hidden>
 key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-kp-clientAuth
 pre-save command:
 post-save command: /usr/lib/ipa/certmonger/restart_httpd
 track: yes
 auto-renew: yes
Request ID '20181021083724':
 status: CA_UNREACHABLE
 ca-error: Server at https://ipa01.mydomain.com/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).
 stuck: no
 key pair storage: type=FILE,location='/var/lib/krb5kdc/kdc.key'
 certificate: type=FILE,location='/var/lib/krb5kdc/kdc.crt'
 CA: IPA
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=ipa01.mydomain.com,O=MYREALM.COM
 expires: 2020-10-21 02:37:25 MDT
 principal name: <email address hidden>
 key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
 eku: id-kp-serverAuth,id-pkinit-KPKdc
 pre-save command:
 post-save command: /usr/lib/ipa/certmonger/renew_kdc_cert
 track: yes
 auto-renew: yes

Question information

Language:
English Edit question
Status:
Expired
For:
Ubuntu freeipa Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Bernard Stafford (bernard010) said : 		#1

Correct - It is manual server install only [option 3] - https://ubuntu.com/download/server
https://ubuntu.com/server/docs/security-certificates
http://manpages.ubuntu.com/manpages/bionic/man8/update-ca-certificates.8.html
Certificates must have a .crt extension in order to be included by update-ca-certificates.
 *An expired certificate was removed from ca-certificates. -> https://ubuntu.com/security/notices/USN-4377-1
Update instructions: The problem can be corrected by updating your system to the following package versions.

Revision history for this message
Shaav (shaav) said : 		#3

Thank you for replying.

I installed the .deb package here: https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+build/19389901

which I think is what you were saying to do.

I restarted ipa, apache and certmonger, still with the date rolled back to before the certificates expired.

I still get the same error though—did I miss something?

Revision history for this message
Bernard Stafford (bernard010) said : 		#4

Package: python- pyasn1: 0.4.2-3 / No Bug Reports
Package: python-pyasn1-modules: 0.2.1-0.2 / No Bug Reports
Package: 4.7.0~pre1+git20180411-2ubuntu2 / 3 - Bug Reports - Not relevant to your problem.

Revision history for this message
Bernard Stafford (bernard010) said : 		#5

https://launchpad.net/ubuntu/+source/ca-certificates/20190110~18.04.1

Revision history for this message
Shaav (shaav) said : 		#6

Right: From https://launchpad.net/ubuntu/+source/ca-certificates/20190110~18.04.1

I followed the builds link to here: https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+build/19389901

And downloaded and installed the .deb. It hasn't changed anything.

Since my original question, I saw https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1772450—while it's isn't directly related, it caused me to take a look in the webui which I seldom use. I did get the error in the bug report and modified plugins/dogtag.py to get around it.

The relevant bit is that in the certificates list, it is clear that new certs are being issued everytime certmonger tries—I now have >50 of the same two certs (two are created each time certmonger is restarted). If I try to view any of those, I get the identical PyASN1 error both on screen and in the log.

Inferring from the logs and getcert list, I believe they are the certs in:
/var/lib/krb5kdc and
/etc/dirsrv/slapd-MYREALM-COM/Server-Cert

Revision history for this message
Bernard Stafford (bernard010) said : 		#7

Go ahead and file a Bug Report .

Revision history for this message
Bernard Stafford (bernard010) said : 		#8

Did you see this? https://bugs.launchpad.net/ubuntu/+source/ca-certificates/+bug/1881533

Revision history for this message
Shaav (shaav) said : 		#9

done: https://bugs.launchpad.net/ubuntu/+source/freeipa/+bug/1902458

And yeah... says that it doesn't seem to affect 18.04?

Nonetheless I ran "update-ca-certificates -f -v" and tried again; no change.

Revision history for this message
Bernard Stafford (bernard010) said : 		#10

Thank You I will do more research to find an answer. I will leave this question in open status.

Revision history for this message
Bernard Stafford (bernard010) said : 		#11

You do realize that https://ipa01.mydomain.com/ipa/xml is not online.

Revision history for this message
Bernard Stafford (bernard010) said : 		#12

You can create your own self-signed certificate. https://ubuntu.com/server/docs/security-certificates#generating-a-csr

Revision history for this message
Bernard Stafford (bernard010) said : 		#13

Request ID '20181021083613':
 status: CA_UNREACHABLE
 ca-error: Server at https://ipa01.mydomain.com/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).
 stuck: no

According to this change servers. The server is offline.

Revision history for this message
Bernard Stafford (bernard010) said : 		#14

certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
 CA: dogtag-ipa-ca-renew-agent
 issuer: CN=Certificate Authority,O=MYREALM.COM
 subject: CN=localhost
https://www.dogtagpki.org/wiki/NSS_Database
Moved to Github:
https://github.com/dogtagpki/pki/wiki/NSS-Database
Just more information.

Revision history for this message
Shaav (shaav) said : 		#15

> You do realize that https://ipa01.mydomain.com/ipa/xml is not online.

I'm not exactly sure what you mean...

mydomain.com is not the actual domain—I replaced it because I didn't really want it published in a public forum. Also the real domain is only accessible on the internal network.

Internally, it absolutely is accessible.

Revision history for this message
Shaav (shaav) said : 		#16

> You can create your own self-signed certificate. https://ubuntu.com/server/docs/security-certificates#generating-a-csr

Indeed—the thing I don't know is if I generate my own certificate, will I be able to use it in IPA? Like if I just created a new cert with the right parameters, can I just copy it into the right location (ex. /var/lib/krb5bdc/kdc.key/crt and have it work? Somehow that seems like too much to hope for...

Revision history for this message
Bernard Stafford (bernard010) said : 		#17

I found this: Offline System Certificate Renewal - Offline system certificate renewal process.
 http://manpages.ubuntu.com/manpages/bionic/man8/pki-server-cert.8.html

Revision history for this message
Bernard Stafford (bernard010) said : 		#18

Could it be Certmonger ?
Did you try Troubleshooting Certmonger ?
https://floblanc.wordpress.com/2016/12/19/troubleshooting-certmonger-issues-with-freeipa/

Revision history for this message
Shaav (shaav) said : 		#19

> Did you try Troubleshooting Certmonger ?

Yup, absolutely. That was one of the first pages that I found.

Revision history for this message
Launchpad Janitor (janitor) said : 		#20

This question was expired because it remained in the 'Open' state without activity for the last 15 days.