dkim signature pass to gmail, fail to yahoo

RFC 4871 says in section 3.5 concerning h=: "Folding whitespace (FWS) MAY be included on either side of the colon separator. "
ABNF:
       sig-h-tag = %x68 [FWS] "=" [FWS] hdr-name
                     0*( *FWS ":" *FWS hdr-name )
       hdr-name = field-name

This is not changed by RFC 5672. Therefore google is correct, and yahoo (ironically, since they invented it) is wrong.

In the interest of compatibility, we could have an option, or even default to removing any FWS around the ':' where possible. This is problematic in general, since long header list MUST be folded to fit within line length restrictions. So yahoo will *still* fail with long header lists. My initial reaction is to file a bug report with yahoo rather than try to accomodate them.

Or are you saying there is non folding space around the ':'? I guess I'll have to set up signing and see.

Yup, sig from test suite looks like:
DKIM-Signature: v=1; a=rsa-sha256; c=simple/relaxed; d=canonical.com; <email address hidden>;
 q=dns/txt; s=example; t=1329964779; h=From : To : Subject;
 bh=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=; b=WkKIHoIG92Bn3PyYVShbPhHxmR3OwI04zGZgxnHRyPd+EkgpGZQiCl+TpzSTRm+g35rBr+CsfEn2/yMz08AKGg==

Well, that still fits the definition of FWS:
       WSP = SP / HTAB
       LWSP = *(WSP / CRLF WSP)
       FWS = [*WSP CRLF] 1*WSP

In this case it is 1*WSP. So yahoo is still wrong. pydkim does it this way to make line breaks simple (so it doesn't have to know it can break around ':' in h=).

I confirmed that simply removing the space breaks line breaking (header names get split across lines, which is not allowed). Since colon is not allowed except as a separator, we could post process the signature to remove spaces adjacent to ':' as an option. But let's not bother until we try to get yahoo to fix their system.

Possible patch for compatibility with yahoo braindamage. Passes test suite.

--- dkim/__init__.py 2012-02-03 22:10:49 +0000
+++ dkim/__init__.py 2012-02-23 03:06:30 +0000
@@ -238,7 +238,7 @@
             j = i + 1
         pre += header[:j] + b"\r\n "
         header = header[j:]
- return pre + header
+ return (pre + header).replace(' : ',':')

A better way to do it would be to have the fold method break on either ':' or ' '.

Okay, I just repeated all of my tests based on dkimpy-5.1. The change that seems to make yahoo happy is removing the call to fold() on 444 of dkim/__init__.py. Removing the spaces around the colons didn't mater. My appologies --- I did not test those in isolation before.

I thought I'd paste the passing and failing headers below, but the word wrapping masked the interesting parts. The one that yahoo passes is a single line that is vaguely 500 chars long. The one that yahoo fails was folded --- oh, and the folding looks fine to me. Just to repeat, gmail took all four permutations - original, without space, without space and not folded, and not folded....

It's been a year or two since I read rfc822 and company, but I just checked rfc2822. Headers CAN be 998 chars long, but SHOULD be folded at 78. In that respect, this all-on-one-line header is legal, but poor. (The "b=" part of the signature is always longer than 78 chars, for example.) Of course, yahoo is also REQUIRED to understand folded dkim headers. Oh well...

I'm writing an email to yahoo now, and will submit via their feedback loop. I'll update here if they respond.

Just to be clear what fold() I'm talking about, here is the diff of the code that made yahoo happy.

--- __init__.py 2012-02-22 21:56:26.000000000 -0600
+++ __init__.py 2012-02-22 21:19:23.000000000 -0600
@@ -441,7 +441,8 @@
     # record what verify should extract
     self.include_headers = tuple(include_headers)

- sig_value = fold(b"; ".join(b"=".join(x) for x in sigfields))
+ sig_value = b"; ".join(b"=".join(x) for x in sigfields)
     dkim_header = (b'DKIM-Signature', b' ' + sig_value)
     h = hashlib.sha256()
     sig = dict(sigfields)