Bad sector handling strategies.

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Slow response from me, so re-opened

Hi,

Sorry it's taken me so long to respond to this.

1) At first, 64KB is the default, but if there are subsequent read that fail consecutively, the size double, I think.
2) As far as I know, yes.
3) The reading strategies aren't written by me, I just wrote a GUI for the data recovery program ddrescue. I'd definitely be interested to hear them though, and the author of ddrescue, Antonio Diaz Diaz, might be interested too :) What do you have in mind?

Once again, sorry for the slow reply.

Hamish

Thank you Hamish, no problem with the delay. Its good that you take the time to make dd easier to use.

I am not a programmer, so i cannot really assist you with how to implement the coding side, but i do have much experience using the command line versions of ddrescue / dd_rescue and dd_rhelp.

Also i have never contacted Antonio about his perspective.

I will get to the 'strategies' in another post.

But i did want to know if it would be possible to add some extra settings that can be changed from the main GUI screen?
For example the amount of sectors to be skipped, if the user wanted the 'jumps' to be bigger ie 128k/256k/512k.......8mb.....64mb...... or any amount for that matter.
I know the command line version you can set these yourself, but on the gui version, having these selectable as a menu item would be a much faster way to change the defaults - even to be able to save a custom profile of different settings.

Hi,

Certainly I could add them, I was just waiting to see if anyone would want them XD

Hamish

On 11 Mar 2017, 11:28 +0000, Mark Edmunds <email address hidden>, wrote:
Question #478211 on DDRescue-GUI changed:
https://answers.launchpad.net/ddrescue-gui/+question/478211

Mark Edmunds posted a new comment:
Thank you Hamish, no problem with the delay. Its good that you take the
time to make dd easier to use.

I am not a programmer, so i cannot really assist you with how to
implement the coding side, but i do have much experience using the
command line versions of ddrescue / dd_rescue and dd_rhelp.

Also i have never contacted Antonio about his perspective.

I will get to the 'strategies' in another post.

But i did want to know if it would be possible to add some extra settings that can be changed from the main GUI screen?
For example the amount of sectors to be skipped, if the user wanted the 'jumps' to be bigger ie 128k/256k/512k.......8mb.....64mb...... or any amount for that matter.
I know the command line version you can set these yourself, but on the gui version, having these selectable as a menu item would be a much faster way to change the defaults - even to be able to save a custom profile of different settings.

--
You received this question notification because you are subscribed to
the question.

ok, great. Thank you again.

As well as having each ddrescue setting/switch selectable on its own, i
was also thinking a 'set' of settings could be pre-configured and
grouped into a list of selectable 'strategies'

Which is where my thoughts about recovery strategies would come in....

But first, here is just a quick bit of background to my situation
.......and why i think i have some useful input, regarding why my
different strategies could be taylor made for different data loss or
disk failure situations.

I had 3 disks full of my pictures over the years, (3 different disk
brands), so i thought i should be smart and backup the files, so i
copied all the content to 3 other new disks.................... 'Great'
i thought, now ive covered myself.

BUT made the ultimate dumb mistake of.....you guessed it...... putting 6
disks all in the same box. And the day came when we moved house, and
sure enough when i drove the van to the new place we had a crash on the
motorway, everyone was ok but all my belongings were now smashed all
over 2 lanes of the motorway - including my box of 6 disks which had
split open on impact. So each disk had been bouncing along the road for
several meters......

- of course what i should have done was put the 3 original drives in one
box on the first trip, and then come back for the 2nd set of disks on my
next trip. A classic example of learning from your mistakes.....

So, i picked the disks up, put them in another box, and stored them at
our new house for a few months while we moved in. OK so a few months
later, when everything had settled down, i started each disk up to see
how bad the damage was and started to copy files. Some disks would copy
only a few % other disks copied about 60% before locking up.

It was at this point that i noticed that the original 3 disks would copy
some files that the 'backup' 3 disks would not, and vice versa. So i got
confident that IF i was able to 'merge' the content of, (for example
'original disk1' with 'backup disk1'), then i should be able to get
'close to' 100% of the stuff back. As it would be unlikely that the same
picture would be damaged on BOTH disks.

And in the end i was right, as after merging the content of Disk1 with
Backup_Disk1, i think that out of about 14,000 files i probably lost
about 300 pictures, which was fine as i normally take a few multiple
pictures at the same time of the same scene anyway. I did this for
Disks2 and 3 with the same sort of results.... Which is great, i was
very satisfied with the result.

........The only thing thing to add, (in relevance to this thread), was
that before i started to copy and merge the files, i used ddrescue to
clone all 6 disks to 6 .img files on my computer -INCASE any of the bad
disks should fail catastrophically during the copy/merge process.
Surprisingly for me, even though the disks had been through a lot (with
many read errors), none of the 6 disks have failed at a hardware level,
and there all still readable even now.

And....... as i have now recovered all the data that i need. These bad
disks will be perfect for testing read strategies......., (and i do not
care if they do fail even more to the point where they cannot be read
anymore)....... which i know will happen at some point anyway.

So, this is why i think i have a perfect set of 'bad test-disks' for
the purpose of testing different reading strategies, as i know where the
errors and bad sectors are, and the time that it 'should' take for each
bad disk to be read 99.99%.

So that way i can compare which overall strategy works best for each
different damaged disk.

So let me know if you want me to suggest strategies, for different
scenarios.

my email is

mark0958@hotmail

if you want to ask me specific questions

On 2017-03-11 20:23, Hamish McIntyre-Bhatty wrote:
> Your question #478211 on DDRescue-GUI changed:
> https://answers.launchpad.net/ddrescue-gui/+question/478211
>
> Hamish McIntyre-Bhatty proposed the following answer:
> Hi,
>
> Certainly I could add them, I was just waiting to see if anyone would
> want them XD
>
> Hamish
>
>
> On 11 Mar 2017, 11:28 +0000, Mark Edmunds <email address hidden>, wrote:
> Question #478211 on DDRescue-GUI changed:
> https://answers.launchpad.net/ddrescue-gui/+question/478211
>
> Mark Edmunds posted a new comment:
> Thank you Hamish, no problem with the delay. Its good that you take the
> time to make dd easier to use.
>
> I am not a programmer, so i cannot really assist you with how to
> implement the coding side, but i do have much experience using the
> command line versions of ddrescue / dd_rescue and dd_rhelp.
>
> Also i have never contacted Antonio about his perspective.
>
>
> I will get to the 'strategies' in another post.
>
> But i did want to know if it would be possible to add some extra settings that can be changed from the main GUI screen?
> For example the amount of sectors to be skipped, if the user wanted the 'jumps' to be bigger ie 128k/256k/512k.......8mb.....64mb...... or any amount for that matter.
> I know the command line version you can set these yourself, but on the gui version, having these selectable as a menu item would be a much faster way to change the defaults - even to be able to save a custom profile of different settings.
>
> --
> You received this question notification because you are subscribed to
> the question.
>

Hi,

Sounds like a really good idea, I never had any properly bad disks to
test on, just some self-scratched CDs, and some rubbishy old memory
sticks. Thanks, that sounds like it would be really helpful, I shall
contact you when I have some ideas :)

Hamish

On 12/03/17 10:23, Mark Edmunds wrote:
> Question #478211 on DDRescue-GUI changed:
> https://answers.launchpad.net/ddrescue-gui/+question/478211
>
> Status: Answered => Open
>
> Mark Edmunds is still having a problem:
> ok, great. Thank you again.
>
> As well as having each ddrescue setting/switch selectable on its own, i
> was also thinking a 'set' of settings could be pre-configured and
> grouped into a list of selectable 'strategies'
>
> Which is where my thoughts about recovery strategies would come in....
>
>
> But first, here is just a quick bit of background to my situation
> .......and why i think i have some useful input, regarding why my
> different strategies could be taylor made for different data loss or
> disk failure situations.
>
>
> I had 3 disks full of my pictures over the years, (3 different disk
> brands), so i thought i should be smart and backup the files, so i
> copied all the content to 3 other new disks.................... 'Great'
> i thought, now ive covered myself.
>
>
> BUT made the ultimate dumb mistake of.....you guessed it...... putting 6
> disks all in the same box. And the day came when we moved house, and
> sure enough when i drove the van to the new place we had a crash on the
> motorway, everyone was ok but all my belongings were now smashed all
> over 2 lanes of the motorway - including my box of 6 disks which had
> split open on impact. So each disk had been bouncing along the road for
> several meters......
>
> - of course what i should have done was put the 3 original drives in one
> box on the first trip, and then come back for the 2nd set of disks on my
> next trip. A classic example of learning from your mistakes.....
>
> So, i picked the disks up, put them in another box, and stored them at
> our new house for a few months while we moved in. OK so a few months
> later, when everything had settled down, i started each disk up to see
> how bad the damage was and started to copy files. Some disks would copy
> only a few % other disks copied about 60% before locking up.
>
> It was at this point that i noticed that the original 3 disks would copy
> some files that the 'backup' 3 disks would not, and vice versa. So i got
> confident that IF i was able to 'merge' the content of, (for example
> 'original disk1' with 'backup disk1'), then i should be able to get
> 'close to' 100% of the stuff back. As it would be unlikely that the same
> picture would be damaged on BOTH disks.
>
> And in the end i was right, as after merging the content of Disk1 with
> Backup_Disk1, i think that out of about 14,000 files i probably lost
> about 300 pictures, which was fine as i normally take a few multiple
> pictures at the same time of the same scene anyway. I did this for
> Disks2 and 3 with the same sort of results.... Which is great, i was
> very satisfied with the result.
>
>
> ........The only thing thing to add, (in relevance to this thread), was
> that before i started to copy and merge the files, i used ddrescue to
> clone all 6 disks to 6 .img files on my computer -INCASE any of the bad
> disks should fail catastrophically during the copy/merge process.
> Surprisingly for me, even though the disks had been through a lot (with
> many read errors), none of the 6 disks have failed at a hardware level,
> and there all still readable even now.
>
>
> And....... as i have now recovered all the data that i need. These bad
> disks will be perfect for testing read strategies......., (and i do not
> care if they do fail even more to the point where they cannot be read
> anymore)....... which i know will happen at some point anyway.
>
>
> So, this is why i think i have a perfect set of 'bad test-disks' for
> the purpose of testing different reading strategies, as i know where the
> errors and bad sectors are, and the time that it 'should' take for each
> bad disk to be read 99.99%.
>
> So that way i can compare which overall strategy works best for each
> different damaged disk.
>
>
> So let me know if you want me to suggest strategies, for different
> scenarios.
>
> my email is
>
> mark0958@hotmail
>
> if you want to ask me specific questions
>
>
>
> On 2017-03-11 20:23, Hamish McIntyre-Bhatty wrote:
>> Your question #478211 on DDRescue-GUI changed:
>> https://answers.launchpad.net/ddrescue-gui/+question/478211
>>
>> Hamish McIntyre-Bhatty proposed the following answer:
>> Hi,
>>
>> Certainly I could add them, I was just waiting to see if anyone would
>> want them XD
>>
>> Hamish
>>
>>
>> On 11 Mar 2017, 11:28 +0000, Mark Edmunds <email address hidden>, wrote:
>> Question #478211 on DDRescue-GUI changed:
>> https://answers.launchpad.net/ddrescue-gui/+question/478211
>>
>> Mark Edmunds posted a new comment:
>> Thank you Hamish, no problem with the delay. Its good that you take the
>> time to make dd easier to use.
>>
>> I am not a programmer, so i cannot really assist you with how to
>> implement the coding side, but i do have much experience using the
>> command line versions of ddrescue / dd_rescue and dd_rhelp.
>>
>> Also i have never contacted Antonio about his perspective.
>>
>>
>> I will get to the 'strategies' in another post.
>>
>> But i did want to know if it would be possible to add some extra settings that can be changed from the main GUI screen?
>> For example the amount of sectors to be skipped, if the user wanted the 'jumps' to be bigger ie 128k/256k/512k.......8mb.....64mb...... or any amount for that matter.
>> I know the command line version you can set these yourself, but on the gui version, having these selectable as a menu item would be a much faster way to change the defaults - even to be able to save a custom profile of different settings.
>>
>> --
>> You received this question notification because you are subscribed to
>> the question.
>>
>

Thank you, for the ongoing help.

I will put forward some strategies, and explain why strategy x would be
benficial over strategy y - with a few example cases from my disks.

And maybe you could pre configure the various strategies/methods as a
separate selectable menu items

Mark

On 2017-03-26 15:28, Hamish McIntyre-Bhatty wrote:
> Your question #478211 on DDRescue-GUI changed:
> https://answers.launchpad.net/ddrescue-gui/+question/478211
>
> Status: Open => Answered
>
> Hamish McIntyre-Bhatty proposed the following answer:
> Hi,
>
> Sounds like a really good idea, I never had any properly bad disks to
> test on, just some self-scratched CDs, and some rubbishy old memory
> sticks. Thanks, that sounds like it would be really helpful, I shall
> contact you when I have some ideas :)
>
> Hamish
>
> On 12/03/17 10:23, Mark Edmunds wrote:
>> Question #478211 on DDRescue-GUI changed:
>> https://answers.launchpad.net/ddrescue-gui/+question/478211
>>
>> Status: Answered => Open
>>
>> Mark Edmunds is still having a problem:
>> ok, great. Thank you again.
>>
>> As well as having each ddrescue setting/switch selectable on its own, i
>> was also thinking a 'set' of settings could be pre-configured and
>> grouped into a list of selectable 'strategies'
>>
>> Which is where my thoughts about recovery strategies would come in....
>>
>>
>> But first, here is just a quick bit of background to my situation
>> .......and why i think i have some useful input, regarding why my
>> different strategies could be taylor made for different data loss or
>> disk failure situations.
>>
>>
>> I had 3 disks full of my pictures over the years, (3 different disk
>> brands), so i thought i should be smart and backup the files, so i
>> copied all the content to 3 other new disks.................... 'Great'
>> i thought, now ive covered myself.
>>
>>
>> BUT made the ultimate dumb mistake of.....you guessed it...... putting 6
>> disks all in the same box. And the day came when we moved house, and
>> sure enough when i drove the van to the new place we had a crash on the
>> motorway, everyone was ok but all my belongings were now smashed all
>> over 2 lanes of the motorway - including my box of 6 disks which had
>> split open on impact. So each disk had been bouncing along the road for
>> several meters......
>>
>> - of course what i should have done was put the 3 original drives in one
>> box on the first trip, and then come back for the 2nd set of disks on my
>> next trip. A classic example of learning from your mistakes.....
>>
>> So, i picked the disks up, put them in another box, and stored them at
>> our new house for a few months while we moved in. OK so a few months
>> later, when everything had settled down, i started each disk up to see
>> how bad the damage was and started to copy files. Some disks would copy
>> only a few % other disks copied about 60% before locking up.
>>
>> It was at this point that i noticed that the original 3 disks would copy
>> some files that the 'backup' 3 disks would not, and vice versa. So i got
>> confident that IF i was able to 'merge' the content of, (for example
>> 'original disk1' with 'backup disk1'), then i should be able to get
>> 'close to' 100% of the stuff back. As it would be unlikely that the same
>> picture would be damaged on BOTH disks.
>>
>> And in the end i was right, as after merging the content of Disk1 with
>> Backup_Disk1, i think that out of about 14,000 files i probably lost
>> about 300 pictures, which was fine as i normally take a few multiple
>> pictures at the same time of the same scene anyway. I did this for
>> Disks2 and 3 with the same sort of results.... Which is great, i was
>> very satisfied with the result.
>>
>>
>> ........The only thing thing to add, (in relevance to this thread), was
>> that before i started to copy and merge the files, i used ddrescue to
>> clone all 6 disks to 6 .img files on my computer -INCASE any of the bad
>> disks should fail catastrophically during the copy/merge process.
>> Surprisingly for me, even though the disks had been through a lot (with
>> many read errors), none of the 6 disks have failed at a hardware level,
>> and there all still readable even now.
>>
>>
>> And....... as i have now recovered all the data that i need. These bad
>> disks will be perfect for testing read strategies......., (and i do not
>> care if they do fail even more to the point where they cannot be read
>> anymore)....... which i know will happen at some point anyway.
>>
>>
>> So, this is why i think i have a perfect set of 'bad test-disks' for
>> the purpose of testing different reading strategies, as i know where the
>> errors and bad sectors are, and the time that it 'should' take for each
>> bad disk to be read 99.99%.
>>
>> So that way i can compare which overall strategy works best for each
>> different damaged disk.
>>
>>
>> So let me know if you want me to suggest strategies, for different
>> scenarios.
>>
>> my email is
>>
>> mark0958@hotmail
>>
>> if you want to ask me specific questions
>>
>>
>>
>> On 2017-03-11 20:23, Hamish McIntyre-Bhatty wrote:
>>> Your question #478211 on DDRescue-GUI changed:
>>> https://answers.launchpad.net/ddrescue-gui/+question/478211
>>>
>>> Hamish McIntyre-Bhatty proposed the following answer:
>>> Hi,
>>>
>>> Certainly I could add them, I was just waiting to see if anyone would
>>> want them XD
>>>
>>> Hamish
>>>
>>>
>>> On 11 Mar 2017, 11:28 +0000, Mark Edmunds <email address hidden>, wrote:
>>> Question #478211 on DDRescue-GUI changed:
>>> https://answers.launchpad.net/ddrescue-gui/+question/478211
>>>
>>> Mark Edmunds posted a new comment:
>>> Thank you Hamish, no problem with the delay. Its good that you take the
>>> time to make dd easier to use.
>>>
>>> I am not a programmer, so i cannot really assist you with how to
>>> implement the coding side, but i do have much experience using the
>>> command line versions of ddrescue / dd_rescue and dd_rhelp.
>>>
>>> Also i have never contacted Antonio about his perspective.
>>>
>>>
>>> I will get to the 'strategies' in another post.
>>>
>>> But i did want to know if it would be possible to add some extra settings that can be changed from the main GUI screen?
>>> For example the amount of sectors to be skipped, if the user wanted the 'jumps' to be bigger ie 128k/256k/512k.......8mb.....64mb...... or any amount for that matter.
>>> I know the command line version you can set these yourself, but on the gui version, having these selectable as a menu item would be a much faster way to change the defaults - even to be able to save a custom profile of different settings.
>>>
>>> --
>>> You received this question notification because you are subscribed to
>>> the question.
>>>

Okay, sure, sounds good :)

On 26 Mar 2017, 15:17 +0100, Mark Edmunds <email address hidden>, wrote:
Question #478211 on DDRescue-GUI changed:
https://answers.launchpad.net/ddrescue-gui/+question/478211

Status: Answered => Open

Mark Edmunds is still having a problem:
Thank you, for the ongoing help.

I will put forward some strategies, and explain why strategy x would be
benficial over strategy y - with a few example cases from my disks.

And maybe you could pre configure the various strategies/methods as a
separate selectable menu items

Mark

On 2017-03-26 15:28, Hamish McIntyre-Bhatty wrote:
Your question #478211 on DDRescue-GUI changed:
https://answers.launchpad.net/ddrescue-gui/+question/478211

Status: Open => Answered

Hamish McIntyre-Bhatty proposed the following answer:
Hi,

Sounds like a really good idea, I never had any properly bad disks to
test on, just some self-scratched CDs, and some rubbishy old memory
sticks. Thanks, that sounds like it would be really helpful, I shall
contact you when I have some ideas :)

Hamish

On 12/03/17 10:23, Mark Edmunds wrote:
Question #478211 on DDRescue-GUI changed:
https://answers.launchpad.net/ddrescue-gui/+question/478211

Status: Answered => Open

Mark Edmunds is still having a problem:
ok, great. Thank you again.

As well as having each ddrescue setting/switch selectable on its own, i
was also thinking a 'set' of settings could be pre-configured and
grouped into a list of selectable 'strategies'

Which is where my thoughts about recovery strategies would come in....

But first, here is just a quick bit of background to my situation
.......and why i think i have some useful input, regarding why my
different strategies could be taylor made for different data loss or
disk failure situations.

I had 3 disks full of my pictures over the years, (3 different disk
brands), so i thought i should be smart and backup the files, so i
copied all the content to 3 other new disks.................... 'Great'
i thought, now ive covered myself.

BUT made the ultimate dumb mistake of.....you guessed it...... putting 6
disks all in the same box. And the day came when we moved house, and
sure enough when i drove the van to the new place we had a crash on the
motorway, everyone was ok but all my belongings were now smashed all
over 2 lanes of the motorway - including my box of 6 disks which had
split open on impact. So each disk had been bouncing along the road for
several meters......

- of course what i should have done was put the 3 original drives in one
box on the first trip, and then come back for the 2nd set of disks on my
next trip. A classic example of learning from your mistakes.....

So, i picked the disks up, put them in another box, and stored them at
our new house for a few months while we moved in. OK so a few months
later, when everything had settled down, i started each disk up to see
how bad the damage was and started to copy files. Some disks would copy
only a few % other disks copied about 60% before locking up.

It was at this point that i noticed that the original 3 disks would copy
some files that the 'backup' 3 disks would not, and vice versa. So i got
confident that IF i was able to 'merge' the content of, (for example
'original disk1' with 'backup disk1'), then i should be able to get
'close to' 100% of the stuff back. As it would be unlikely that the same
picture would be damaged on BOTH disks.

And in the end i was right, as after merging the content of Disk1 with
Backup_Disk1, i think that out of about 14,000 files i probably lost
about 300 pictures, which was fine as i normally take a few multiple
pictures at the same time of the same scene anyway. I did this for
Disks2 and 3 with the same sort of results.... Which is great, i was
very satisfied with the result.

........The only thing thing to add, (in relevance to this thread), was
that before i started to copy and merge the files, i used ddrescue to
clone all 6 disks to 6 .img files on my computer -INCASE any of the bad
disks should fail catastrophically during the copy/merge process.
Surprisingly for me, even though the disks had been through a lot (with
many read errors), none of the 6 disks have failed at a hardware level,
and there all still readable even now.

And....... as i have now recovered all the data that i need. These bad
disks will be perfect for testing read strategies......., (and i do not
care if they do fail even more to the point where they cannot be read
anymore)....... which i know will happen at some point anyway.

So, this is why i think i have a perfect set of 'bad test-disks' for
the purpose of testing different reading strategies, as i know where the
errors and bad sectors are, and the time that it 'should' take for each
bad disk to be read 99.99%.

So that way i can compare which overall strategy works best for each
different damaged disk.

So let me know if you want me to suggest strategies, for different
scenarios.

my email is

mark0958@hotmail

if you want to ask me specific questions

On 2017-03-11 20:23, Hamish McIntyre-Bhatty wrote:
Your question #478211 on DDRescue-GUI changed:
https://answers.launchpad.net/ddrescue-gui/+question/478211

Hamish McIntyre-Bhatty proposed the following answer:
Hi,

Certainly I could add them, I was just waiting to see if anyone would
want them XD

Hamish

On 11 Mar 2017, 11:28 +0000, Mark Edmunds <email address hidden>, wrote:
Question #478211 on DDRescue-GUI changed:
https://answers.launchpad.net/ddrescue-gui/+question/478211

Mark Edmunds posted a new comment:
Thank you Hamish, no problem with the delay. Its good that you take the
time to make dd easier to use.

I am not a programmer, so i cannot really assist you with how to
implement the coding side, but i do have much experience using the
command line versions of ddrescue / dd_rescue and dd_rhelp.

Also i have never contacted Antonio about his perspective.

I will get to the 'strategies' in another post.

But i did want to know if it would be possible to add some extra settings that can be changed from the main GUI screen?
For example the amount of sectors to be skipped, if the user wanted the 'jumps' to be bigger ie 128k/256k/512k.......8mb.....64mb...... or any amount for that matter.
I know the command line version you can set these yourself, but on the gui version, having these selectable as a menu item would be a much faster way to change the defaults - even to be able to save a custom profile of different settings.

--
You received this question notification because you are subscribed to
the question.

--
You received this question notification because you are subscribed to
the question.