dkim signature pass to gmail, fail to yahoo

Asked by Rob Nichols

Hello. I am new to DKIM, and might be doing something silly.

I build a message using email.mime, call as_string() on it, construct a DKIM object with the msg, and call sign. I prepend that signature onto my message, and send it. An email to a google apps account gives me "dkim=pass. An email to a yahoo account gives "dkim=permerror (bad sig)". This is using the same code, same from email and domain, etc.. I get the same results using pydkim-3 and with the newer dkimpy-5.1.

While I was still using pydkim-3, I tried different things, and managed to get both yahoo and gmail to give me a pass. I changed the sign function to omit the spaces around the colon in the "h=" list, and removed the call to the fold utility in the sign function so that the signature header was all on one long line. With those changes, yahoo and google were both happy.

Any suggestions?


Question information

English Edit question
dkimpy Edit question
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Stuart Gathman (stuart-gathman) said :

RFC 4871 says in section 3.5 concerning h=: "Folding whitespace (FWS) MAY be included on either side of the colon separator. "
       sig-h-tag = %x68 [FWS] "=" [FWS] hdr-name
                     0*( *FWS ":" *FWS hdr-name )
       hdr-name = field-name

This is not changed by RFC 5672. Therefore google is correct, and yahoo (ironically, since they invented it) is wrong.

In the interest of compatibility, we could have an option, or even default to removing any FWS around the ':' where possible. This is problematic in general, since long header list MUST be folded to fit within line length restrictions. So yahoo will *still* fail with long header lists. My initial reaction is to file a bug report with yahoo rather than try to accomodate them.

Revision history for this message
Stuart Gathman (stuart-gathman) said :

Or are you saying there is non folding space around the ':'? I guess I'll have to set up signing and see.

Revision history for this message
Stuart Gathman (stuart-gathman) said :

Yup, sig from test suite looks like:
DKIM-Signature: v=1; a=rsa-sha256; c=simple/relaxed;; <email address hidden>;
 q=dns/txt; s=example; t=1329964779; h=From : To : Subject;
 bh=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=; b=WkKIHoIG92Bn3PyYVShbPhHxmR3OwI04zGZgxnHRyPd+EkgpGZQiCl+TpzSTRm+g35rBr+CsfEn2/yMz08AKGg==

Well, that still fits the definition of FWS:
       WSP = SP / HTAB
       LWSP = *(WSP / CRLF WSP)
       FWS = [*WSP CRLF] 1*WSP

In this case it is 1*WSP. So yahoo is still wrong. pydkim does it this way to make line breaks simple (so it doesn't have to know it can break around ':' in h=).

Revision history for this message
Stuart Gathman (stuart-gathman) said :

I confirmed that simply removing the space breaks line breaking (header names get split across lines, which is not allowed). Since colon is not allowed except as a separator, we could post process the signature to remove spaces adjacent to ':' as an option. But let's not bother until we try to get yahoo to fix their system.

Revision history for this message
Stuart Gathman (stuart-gathman) said :

Possible patch for compatibility with yahoo braindamage. Passes test suite.

--- dkim/ 2012-02-03 22:10:49 +0000
+++ dkim/ 2012-02-23 03:06:30 +0000
@@ -238,7 +238,7 @@
             j = i + 1
         pre += header[:j] + b"\r\n "
         header = header[j:]
- return pre + header
+ return (pre + header).replace(' : ',':')

Revision history for this message
Stuart Gathman (stuart-gathman) said :

A better way to do it would be to have the fold method break on either ':' or ' '.

Revision history for this message
Rob Nichols (0x2jjt-rob-0tlbog) said :

Okay, I just repeated all of my tests based on dkimpy-5.1. The change that seems to make yahoo happy is removing the call to fold() on 444 of dkim/ Removing the spaces around the colons didn't mater. My appologies --- I did not test those in isolation before.

I thought I'd paste the passing and failing headers below, but the word wrapping masked the interesting parts. The one that yahoo passes is a single line that is vaguely 500 chars long. The one that yahoo fails was folded --- oh, and the folding looks fine to me. Just to repeat, gmail took all four permutations - original, without space, without space and not folded, and not folded....

It's been a year or two since I read rfc822 and company, but I just checked rfc2822. Headers CAN be 998 chars long, but SHOULD be folded at 78. In that respect, this all-on-one-line header is legal, but poor. (The "b=" part of the signature is always longer than 78 chars, for example.) Of course, yahoo is also REQUIRED to understand folded dkim headers. Oh well...

I'm writing an email to yahoo now, and will submit via their feedback loop. I'll update here if they respond.

Just to be clear what fold() I'm talking about, here is the diff of the code that made yahoo happy.

--- 2012-02-22 21:56:26.000000000 -0600
+++ 2012-02-22 21:19:23.000000000 -0600
@@ -441,7 +441,8 @@
     # record what verify should extract
     self.include_headers = tuple(include_headers)

- sig_value = fold(b"; ".join(b"=".join(x) for x in sigfields))
+ sig_value = b"; ".join(b"=".join(x) for x in sigfields)
     dkim_header = (b'DKIM-Signature', b' ' + sig_value)
     h = hashlib.sha256()
     sig = dict(sigfields)

Can you help with this problem?

Provide an answer of your own, or ask Rob Nichols for more information if necessary.

To post a message you must log in.