Invalid data - SHA1 hash mismatch for file (One cause)

Asked by nacs8quad

I have been getting the error "Invalid data - SHA1 hash mismatch for file" only when run duplicity on files larger than about 7GB. I have a script that does a checksum (using hashdeep), a duplicity full backup, a duplicity verify, and then a duplicity restore. I kept have a problem with the verify working but the restore giving me the above error, with no consistency what-so-ever (it would happen to different volumes). It turns out after days of working on this and writing scripts to automate my test, the evidence points to a single possible problem: bad memory.

After the first day, I suspected bad memory and did a memtest86+ and verify indeed after a single memtest86+ pass, I received an error (which was repeatable between reboots and multiple memtest passes). I tried using the badram capability in grub2 (using the output from memtest86), but it didn't help. It turns out that the "badram" output from memtest86+ is actually formatted for 32-bit not 64-bit linux (according to what I read online).

I finally gave in and swapped some RAM from another machine into my server, verified that it did indeed pass memtest86+ with no errors, and then ran my backup script again.

I have run that backup script 8 times since I put in the new RAM and I haven't had a single failure. The bottom line is, all the evidence apparently points to the RAM being the problem. Hard to be 100% sure, but I'm pretty confident at this point.

My backup test script is here:
http://pastebin.com/dK32VcX0

Also, I'm guessing my hashdeep checksums are redundant (presumably checksums are checked when doing a restore).

Matt

Question information

Language:
English Edit question
Status:
Expired
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
edso (ed.so) said :
#1

hey Matt, thanks for sharing your experience.. ede/duply.net

Revision history for this message
nacs8quad (nacs8quad) said :
#2

No problem. I wish I found somewhere for someone to suggest checking the ram when coming across this error.

Matt

Revision history for this message
Launchpad Janitor (janitor) said :
#3

This question was expired because it remained in the 'Open' state without activity for the last 15 days.