Temperror after updating to python3.5

Asked by steve@swid.co.uk

Hi,
After moving from python 2.7 to 3.5 and from pydns to py3dns all spfcheck fail with spfcheck: pyspf result: "['Temperror', 'SPF Temporary Error: DNS No working name servers discovered', 'mailfrom']".
OS is macOS 10.12 sierra.
Python35 installed via macports

root# pip list
authres (0.800)
pip (8.1.2)
py3dns (3.1.0)
pypolicyd-spf (1.3.2)
pyspf (2.0.11)
readline (6.2.4.1)
setuptools (28.1.0)
virtualenv (15.0.3)

Typical result of grep spf /var/log/mail.log (server names and ipaddresses munged:

Oct 3 23:01:49 mac3 policyd-spf[27404]: Starting
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "request=smtpd_access_policy"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "protocol_state=RCPT"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "protocol_name=ESMTP"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "client_address=aa.bb.cc.ddd"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "client_name=mailgate.invalid.co.uk"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "client_port=41974"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "reverse_client_name=mailgate.invalid.co.uk"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "helo_name=mailgate.invalid.co.uk"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "<email address hidden>"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "<email address hidden>"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "recipient_count=0"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "queue_id="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "instance=6b09.57f2d54d.5f6af.0"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "size=2785"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "etrn_domain="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "stress="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "sasl_method="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "sasl_username="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "sasl_sender="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "ccert_subject="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "ccert_issuer="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "ccert_fingerprint="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "ccert_pubkey_fingerprint="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "encryption_protocol=TLSv1.2"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "encryption_cipher=AECDH-AES256-SHA"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "encryption_keysize=256"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: "policy_context="
Oct 3 23:01:49 mac3 policyd-spf[27404]: Read line: ""
Oct 3 23:01:49 mac3 policyd-spf[27404]: Found the end of entry
Oct 3 23:01:49 mac3 policyd-spf[27404]: Config: {'PermError_reject': 'False', 'Mail_From_reject': 'False', 'Lookup_Time': 20, 'TempError_Defer': 'False', 'Authserv_Id': 'mac3.example.com', 'HELO_reject': 'False', 'Void_Limit': 2, 'Reject_Not_Pass_Domains': '', 'debugLevel': 5, 'Header_Type': 'AR', 'defaultSeedOnly': 1, 'skip_addresses': '127.0.0.0/8,::ffff:127.0.0.0/104,::1'}
Oct 3 23:01:49 mac3 policyd-spf[27404]: Cached data for this instance: []
Oct 3 23:01:49 mac3 policyd-spf[27404]: spfcheck: pyspf result: "['Temperror', 'SPF Temporary Error: DNS No working name servers discovered', 'helo']"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Temperror; identity=helo; client-ip=aa.bb.cc.ddd; helo=mailgate.invalid.co.uk; <email address hidden>; <email address hidden>
Oct 3 23:01:49 mac3 policyd-spf[27404]: Header type: AR; Authres ID (for AR): mac3.example.com
Oct 3 23:01:49 mac3 policyd-spf[27404]: spfcheck: pyspf result: "['Temperror', 'SPF Temporary Error: DNS No working name servers discovered', 'mailfrom']"
Oct 3 23:01:49 mac3 policyd-spf[27404]: Temperror; identity=mailfrom; client-ip=aa.bb.cc.ddd; helo=mailgate.invalid.co.uk; <email address hidden>; <email address hidden>
Oct 3 23:01:49 mac3 policyd-spf[27404]: Header type: AR; Authres ID (for AR): mac3.example.com
Oct 3 23:01:49 mac3 policyd-spf[27404]: Action: prepend: Text: Authentication-Results: mac3.example.com; spf=temperror (SPF Temporary Error: DNS No working name servers discovered) smtp.mailfrom=example.com (client-ip=aa.bb.cc.ddd; helo=mailgate.invalid.co.uk; <email address hidden>; <email address hidden>)

Tests:
root# python3 spf.py _spf.google.com
v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all

root# /opt/local/bin/python3
import DNS
DNS.DiscoverNameServers()
print (DNS.defaults['server'])
['127.0.0.1', '217.169.20.21', '217.169.20.22', '::1', '2001:8b0::2020', '2001:8b0::2021', '127.0.0.1', '217.169.20.21', '217.169.20.22', '::1', '2001:8b0::2020', '2001:8b0::2021']
>>>
req = DNS.DnsRequest('launchpad.net', qtype='TXT', timeout=20)
resp = req.req()
resp.show()
; <<>> PDG.py 1.0 <<>> launchpad.net TXT
;; options: recurs
;; got answer:
;; ->>HEADER<<- opcode 0, status NOERROR, id 21032
;; flags: qr rd ra; Ques: 1, Ans: 1, Auth: 4, Addit: 0
;; QUESTIONS:
;; launchpad.net, type = TXT, class = IN

;; ANSWERS:
launchpad.net 600 TXT [b'google-site-verification=ua10zhseVKf6We9evg5KeBV4vGEncOVjavFYK-qGZAE']

;; AUTHORITY RECORDS:
launchpad.net 3600 NS ns4.p27.dynect.net
launchpad.net 3600 NS ns1.p27.dynect.net
launchpad.net 3600 NS ns2.p27.dynect.net
launchpad.net 3600 NS ns3.p27.dynect.net

;; ADDITIONAL RECORDS:

;; Total query time: 12 msec
;; To SERVER: 127.0.0.1
;; WHEN: Mon Oct 3 23:39:48 2016
>>>

Any other useful tests I can try?

Steve

Question information

Language:
English Edit question
Status:
Solved
For:
pypolicyd-spf Edit question
Assignee:
No assignee Edit question
Solved by:
Scott Kitterman
Solved:
Last query:
Last reply:
Revision history for this message
Scott Kitterman (kitterman) said :
#1

There is a newer version of pyspf available. It probably won't make a difference, but you could try upgrading that first. Unfortunately, I'm not very familiar with how Mac OS does things like name server discovery, so we may be at this for a little while.

Revision history for this message
steve@swid.co.uk (stevm) said :
#2

I've tried pyspf 2.0.12 with the same results.
DNS.DiscoverNameServers() is returning a correct list of name servers as defined in /etc/resolv.conf
I'm running unbound as a DNS server.

Revision history for this message
steve@swid.co.uk (stevm) said :
#3

If I change to pydns 3.0.4 I get different temp fail messages:

Oct 4 08:13:18 mac3 policyd-spf[16210]: spfcheck: pyspf result: "['Temperror', "SPF Temporary Error: DNS 'NoneType' object has no attribute 'header'", 'helo']"
Oct 4 08:13:18 mac3 policyd-spf[16210]: Temperror; identity=helo; client-ip=aa.bb.cc.ddd; helo=mailgate.invalid.co.uk; <email address hidden>; <email address hidden>
Oct 4 08:13:18 mac3 policyd-spf[16210]: Header type: AR; Authres ID (for AR): mac3.example.com
Oct 4 08:13:18 mac3 policyd-spf[16210]: spfcheck: pyspf result: "['Temperror', "SPF Temporary Error: DNS 'NoneType' object has no attribute 'header'", 'mailfrom']"
Oct 4 08:13:18 mac3 policyd-spf[16210]: Temperror; identity=mailfrom; client-ip=aa.bb.cc.ddd; helo=mailgate.invalid.co.uk; <email address hidden>; <email address hidden>
Oct 4 08:13:18 mac3 policyd-spf[16210]: Header type: AR; Authres ID (for AR): mac3.example.com
Oct 4 08:13:18 mac3 policyd-spf[16210]: Action: prepend: Text: Authentication-Results: mac3.example.com; spf=temperror (SPF Temporary Error: DNS 'NoneType' object has no attribute 'header') smtp.mailfrom=example.com (client-ip=aa.bb.cc.ddd; helo=mailgate.invalid.co.uk; <email address hidden>; <email address hidden>)

Revision history for this message
steve@swid.co.uk (stevm) said :
#4

I've worked around the problem for now by editing DNS/Base.py and changing darwin to darwinx so /etc/resolv.conf is used instead of scutil --dns.

Revision history for this message
steve@swid.co.uk (stevm) said :
#5

The real problem is that postfix spawns the policy-spf as user nobody and /usr/sbin/scutil is not in the search path for nobody .

Fixed in DNS/Base.py with
457c457
< scutil = os.popen('scutil --dns', 'r')
---
> scutil = os.popen('/usr/sbin/scutil --dns', 'r')

Steve

Revision history for this message
Best Scott Kitterman (kitterman) said :
#6

Thanks for trouble shooting. I'll fix that in py3dns as I'm one of the developers for it as well.

Revision history for this message
steve@swid.co.uk (stevm) said :
#7

Thanks Scott Kitterman, that solved my question.