dkim signature pass to gmail, fail to yahoo

Asked by Rob Nichols on 2012-02-22

Hello. I am new to DKIM, and might be doing something silly.

I build a message using email.mime, call as_string() on it, construct a DKIM object with the msg, and call sign. I prepend that signature onto my message, and send it. An email to a google apps account gives me "dkim=pass. An email to a yahoo account gives "dkim=permerror (bad sig)". This is using the same code, same from email and domain, etc.. I get the same results using pydkim-3 and with the newer dkimpy-5.1.

While I was still using pydkim-3, I tried different things, and managed to get both yahoo and gmail to give me a pass. I changed the sign function to omit the spaces around the colon in the "h=" list, and removed the call to the fold utility in the sign function so that the signature header was all on one long line. With those changes, yahoo and google were both happy.

Any suggestions?

Thanks,
Rob

Question information

Language:
English Edit question
Status:
Answered
For:
dkimpy Edit question
Assignee:
No assignee Edit question
Last query:
2012-02-22
Last reply:
2012-02-23

RFC 4871 says in section 3.5 concerning h=: "Folding whitespace (FWS) MAY be included on either side of the colon separator. "
ABNF:
       sig-h-tag = %x68 [FWS] "=" [FWS] hdr-name
                     0*( *FWS ":" *FWS hdr-name )
       hdr-name = field-name

This is not changed by RFC 5672. Therefore google is correct, and yahoo (ironically, since they invented it) is wrong.

In the interest of compatibility, we could have an option, or even default to removing any FWS around the ':' where possible. This is problematic in general, since long header list MUST be folded to fit within line length restrictions. So yahoo will *still* fail with long header lists. My initial reaction is to file a bug report with yahoo rather than try to accomodate them.

Or are you saying there is non folding space around the ':'? I guess I'll have to set up signing and see.

Yup, sig from test suite looks like:
DKIM-Signature: v=1; a=rsa-sha256; c=simple/relaxed; d=canonical.com; <email address hidden>;
 q=dns/txt; s=example; t=1329964779; h=From : To : Subject;
 bh=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=; b=WkKIHoIG92Bn3PyYVShbPhHxmR3OwI04zGZgxnHRyPd+EkgpGZQiCl+TpzSTRm+g35rBr+CsfEn2/yMz08AKGg==

Well, that still fits the definition of FWS:
       WSP = SP / HTAB
       LWSP = *(WSP / CRLF WSP)
       FWS = [*WSP CRLF] 1*WSP

In this case it is 1*WSP. So yahoo is still wrong. pydkim does it this way to make line breaks simple (so it doesn't have to know it can break around ':' in h=).

I confirmed that simply removing the space breaks line breaking (header names get split across lines, which is not allowed). Since colon is not allowed except as a separator, we could post process the signature to remove spaces adjacent to ':' as an option. But let's not bother until we try to get yahoo to fix their system.

Possible patch for compatibility with yahoo braindamage. Passes test suite.

--- dkim/__init__.py 2012-02-03 22:10:49 +0000
+++ dkim/__init__.py 2012-02-23 03:06:30 +0000
@@ -238,7 +238,7 @@
             j = i + 1
         pre += header[:j] + b"\r\n "
         header = header[j:]
- return pre + header
+ return (pre + header).replace(' : ',':')

A better way to do it would be to have the fold method break on either ':' or ' '.

Okay, I just repeated all of my tests based on dkimpy-5.1. The change that seems to make yahoo happy is removing the call to fold() on 444 of dkim/__init__.py. Removing the spaces around the colons didn't mater. My appologies --- I did not test those in isolation before.

I thought I'd paste the passing and failing headers below, but the word wrapping masked the interesting parts. The one that yahoo passes is a single line that is vaguely 500 chars long. The one that yahoo fails was folded --- oh, and the folding looks fine to me. Just to repeat, gmail took all four permutations - original, without space, without space and not folded, and not folded....

It's been a year or two since I read rfc822 and company, but I just checked rfc2822. Headers CAN be 998 chars long, but SHOULD be folded at 78. In that respect, this all-on-one-line header is legal, but poor. (The "b=" part of the signature is always longer than 78 chars, for example.) Of course, yahoo is also REQUIRED to understand folded dkim headers. Oh well...

I'm writing an email to yahoo now, and will submit via their feedback loop. I'll update here if they respond.

Just to be clear what fold() I'm talking about, here is the diff of the code that made yahoo happy.

--- __init__.py 2012-02-22 21:56:26.000000000 -0600
+++ __init__.py 2012-02-22 21:19:23.000000000 -0600
@@ -441,7 +441,8 @@
     # record what verify should extract
     self.include_headers = tuple(include_headers)

- sig_value = fold(b"; ".join(b"=".join(x) for x in sigfields))
+ sig_value = b"; ".join(b"=".join(x) for x in sigfields)
     dkim_header = (b'DKIM-Signature', b' ' + sig_value)
     h = hashlib.sha256()
     sig = dict(sigfields)

Can you help with this problem?

Provide an answer of your own, or ask Rob Nichols for more information if necessary.

To post a message you must log in.