rfc822-parse doesn't correctly unpack headers

Asked by Stephen Nightingale

I diagnosed the problem.

This is the message emitted by the Sender end (bracketed by <BEGIN> <END>):

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  <email address hidden>; q=dns/txt; s=mailkey; t=1340661225;
  h=Date : From : To : Cc : Subject : Message-ID : MIME-Version :
  Content-Type : Content-Disposition : User-Agent;
Received: (from night@localhost)
     by had1.antd.nist.gov (8.14.4/8.14.4/Submit) id q5PLriME003627;
     Mon, 25 Jun 2012 17:53:44 -0400
Date: Mon, 25 Jun 2012 17:53:44 -0400
From: Fushimi Inari <email address hidden>
To: <email address hidden>
Cc: <email address hidden>
Subject: Short01 test
Message-ID: <email address hidden>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.20 (2009-12-10)


- I am using Sendmail 8.14.4 at the Sender (good.emailtest) and receiver
(pythentic) ends, and dkimpy 0.5.2 in the milters.
- There is a Sender ppymilter, that calculates the signature and
introduces the DKIM-Signature header at index 0,
and a Receiver ppymilter that does SPF and then verifies the DKIM signature.
- The signature is folded at the Sender end using dkim.fold(). This
splits the signature with "\r\n<space>" sequences.
- The message as delivered through Sendmail has newlines stripped.
Specifically, the "\r\n\" sequences in the DKIM-Signature are replaced
by "\r<space> ".
- This signature fails dkim.verify.
- When I explicitly replace the "\r<space>" sequences in the signature
with "\r\n" sequences, before giving the message to verify, the
signature verifies.

So I suspect there is a problem with the way that the
rfc822_parse(message) function 'unfolds' the signature.
Application of the 'lines = re.split(b"\r?\n", message)' statement would
seem to leave a set of spurious spaces in the signature. The lines
[i][0] in (<tab>, <space> ...) statement doesn't seem to patch it.
dkim.util.parse_tag_value already assumes the signature is correctly
unfolded when it isn't, and the computation fails.

I suspect the problem can be solved by fixing the way that rfc822_parse
unfolds the signature. Or by fixing the newline stripping behavior of
Sendmail. Or there's something even more arcane going on that I don't
yet know.



On 6/19/2012 4:51 PM, Stuart Gathman wrote:
> Your question #200918 on dkimpy changed:
> https://answers.launchpad.net/dkimpy/+question/200918
> Stuart Gathman posted a new comment:
> If the bodyhashes in the DKIM-Signature header field are the same, then
> it is not a header hashing issue. The bh= includes both header and body
> hash. You didn't provide any details, e.g. the DKIM-Signature header
> field in question, or the exception. Note that calling the class method
> lets you get an informative exception, and not just a boolean result. I
> suspect that there is something wrong with the DNS record you published.
> There are many tests of sign/verify symmetry in the unit tests. If it
> turns out you have an actual failure, we'll add it to the suite.

Question information

English Edit question
dkimpy Edit question
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Scott Kitterman (kitterman) said :

Sendmail stripping newlines will invalidate signatures using the simple canonicalization algorithm for headers. RFC 6376 (the current DKIM RFC) says:

3.4.1. The "simple" Header Canonicalization Algorithm

   The "simple" header canonicalization algorithm does not change header
   fields in any way. Header fields MUST be presented to the signing or
   verification algorithm exactly as they are in the message being
   signed or verified. In particular, header field names MUST NOT be
   case folded and whitespace MUST NOT be changed.

So independent of this issue, you need to figure out how to teach Sendmail now to strip the newlines or you're going to have trouble with simple.

Section 3.4.2 (relaxed header canonicalization) says in part:

   o Unfold all header field continuation lines as described in
      [RFC5322]; in particular, lines with terminators embedded in
      continued header field values (that is, CRLF sequences followed by
      WSP) MUST be interpreted without the CRLF. Implementations MUST
      NOT remove the CRLF at the end of the header field value.

Does Sendmail turn the final \r\n for the line to \r? If so, it's definitely breaking the signature.

Additionally, the RFC 5322 reference in 3.4.2 says (5322 section 2.2.3):

Unfolding is accomplished by simply removing any CRLF that is immediately followed by WSP.

If Sendmail is turning "\r\n " into "\r ", I think it's no longer a CRLF that needs unfolding.

Based on your input and my reading of the relevant RFCs, I think Sendmail is wrong here. Sendmail has a history of doing 'helpful' things that turn out to not help, so I don't find this surprising.

Can you help with this problem?

Provide an answer of your own, or ask Stephen Nightingale for more information if necessary.

To post a message you must log in.