Verification - Block Repaired or Not?


Author
Message
ALM4
ALM4
New Member
New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)
Group: Forum Members
Posts: 21, Visits: 42
Recently did a Verify on a backup set made up of a 1.4TB full backup file and 54 incrementals.   The first pass resulted in 19 "Block Repaired" messages before terminating with a "Verification Failure" .    Minutes later, a second attempt went further with 40 "Block Repaired' before also failing.   This is all occurring within the initial P9-J A-00-00.mrimg file.(1.4TB, 111k photos and video files)

The fact that some of same block numbers appear (but not all) in both attempts raises the question whether a repair is actually being made.    After reading through the other message threads on Verification it leaves me puzzled on how any repair could be made if the process just compares hashes and does not go back to the original source data (which might have changed over time).
          [  jphughan"...the verification routine, even the one that can optionally be run immediately after a backup, checks a hash that was stored in the backup file at the time it was created against a hash that is newly calculated during the verification process."...]

Last question.   Given the final "Data Verification Failed" message should I consider the entire backup set  (111k photos and video files) still somewhat viable since I can mount it and browse & extract files?  With the expectation that some parts will be corrupted, but I will not know until I run into them?

Based on other threads I will be swapping out memory and disks to see if I have a hardware issue.

Thanks in advance - AL
Oops forgot the JPG



Edited 11 June 2021 4:44 PM by ALM4
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)
Group: Forum Members
Posts: 9.7K, Visits: 63K
The way verification works is that when a given backup is originally created, Reflect stores the source data and then also records a hash for each "block" of data in the backup.  The way hashes work is that regardless of the quantity of source data, you always get a fixed-length output, and altering even a single bit of data in the source will result in a completely different hash.  When you perform a verification, Reflect reads each data block, calculates a new hash for it at that point in time, and compares that newly calculated hash to the hash that was originally recorded in the backup file for that block.  If the hashes match, it means the data hasn't changed.  If they don't match, either the source data or its hash has been altered, meaning the file is not as it was when it was first created.  The purpose of the verification process is to ensure that all data within the backup is readable and that it has not been altered since it was originally created.  It is NOT to determine whether the backup contains an accurate copy of the source data, for reasons that were discussed in a thread where Macrium Support chimed in, which I can try to find if you're interested.

I do not believe the "scope" of the verification routine has changed for V8.  If it has, it wasn't mentioned, and that would be a significant change to make without any mention.  In terms of how repairs can be made, Macrium Support would undoubtedly be able to provide the best answer, but if the backup contains checksum data, that can be used to repair files, similar to the way a RAID 5 virtual disk array can rebuild the contents of a failed disk using the contents of the remaining disks, since they contain the necessary checksum data to calculate what should be on the failed disk.

However, in your case where different blocks are showing up as problematic on successive passes, it does seem possible that these errors may be the result of hardware issues in your system.  Memory and storage are possible culprits, as are the storage cables (SATA, USB, etc.).  One user recently even found that swapping their CPU resolved sporadic verification failures.

I unfortunately can't answer your final question.

Edited 11 June 2021 4:42 PM by jphughan
ALM4
ALM4
New Member
New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)
Group: Forum Members
Posts: 21, Visits: 42
Had not considered the possibility of a checksum used for repairs. Interesting.

So taking a mental leap, if Verification hits a hash mismatch (perhaps falsely triggered by hardware issues) it uses the checksum to make a correction and then generate a new hash for the block.   But during a subsequent Verification run, it can stumble over the same block if there again is a hardware failure (?).     Wow, I think I see a investment in new hardware in my future - grin.

jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)Macrium Evangelist (14K reputation)
Group: Forum Members
Posts: 9.7K, Visits: 63K
To be clear, I don’t know if the checksum idea is correct. That’s the only mechanism I can think of that would allow for repairs — which I didn’t even know was possible — but that would also increase the size of the backup to the point that a no-compression backup would be larger than the actual source data. But checking the source data itself wouldn’t be practical since with the exception of the auto-verify at the end of a backup, you can’t know for certain that the current state of that source data is correct for the backup state you’re verifying/reconstructing.

But even if a checksum is involved, it wouldn’t involve calculating and storing a new hash. For a repair to be successful, it would have to restore the data such that hashing it now resulted in a match to the originally recorded hash. So you’d calculate a new hash to confirm the repair, but you wouldn’t store a different hash in the backup.
Edited 11 June 2021 5:19 PM by jphughan
Nick
Nick
Macrium Representative
Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)
Group: Administrators
Posts: 2.6K, Visits: 15K
Thanks for the posts. Please see 'Automatically attempt repair' in the Verify KB article:

https://knowledgebase.macrium.com/display/KNOW80/Verifying+image+and+backup+files

Kind Regards

Nick - Macrium Support

Next Webinar


ALM4
ALM4
New Member
New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)
Group: Forum Members
Posts: 21, Visits: 42
jphughan - 11 June 2021 5:18 PM
To be clear, I don’t know if the checksum idea is correct. That’s the only mechanism I can think of that would allow for repairs — which I didn’t even know was possible — but that would also increase the size of the backup to the point that a no-compression backup would be larger than the actual source data. But checking the source data itself wouldn’t be practical since with the exception of the auto-verify at the end of a backup, you can’t know for certain that the current state of that source data is correct for the backup state you’re verifying/reconstructing.

But even if a checksum is involved, it wouldn’t involve calculating and storing a new hash. For a repair to be successful, it would have to restore the data such that hashing it now resulted in a match to the originally recorded hash. So you’d calculate a new hash to confirm the repair, but you wouldn’t store a different hash in the backup.

That makes sense, thanks for the clarification.
ALM4
ALM4
New Member
New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)
Group: Forum Members
Posts: 21, Visits: 42
Nick - 11 June 2021 5:22 PM
Thanks for the posts. Please see 'Automatically attempt repair' in the Verify KB article:

https://knowledgebase.macrium.com/display/KNOW80/Verifying+image+and+backup+files

Ok, have read through this.    The statement:
  
If selected and corruption is detected in a backed up data block then the current live file system is checked to see if it contains the original data. If found then the Image file is repaired by replacing the corrupt data.
To me this means if a block neatly containing Apple.jpg fails, then Reflect will go back to the original source partition and will import the Apple.jpg block again to make the repair?    But if something has changed so the is not possible, that would explain the "Verification Failure" (?).

Still leaves open the question how the majority of block repairs in the first attempt at verification reported fixed (see screenshot above), re-occur in the second attempt.    It is possible I have a hardware issue, but I am puzzled by the repeatability.

Thanks for your patience in explaining this.

Edited 12 June 2021 12:03 AM by ALM4
Nick
Nick
Macrium Representative
Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)Macrium Representative (4.5K reputation)
Group: Administrators
Posts: 2.6K, Visits: 15K
ALM4 - 12 June 2021 12:02 AM
Nick - 11 June 2021 5:22 PM
Thanks for the posts. Please see 'Automatically attempt repair' in the Verify KB article:

https://knowledgebase.macrium.com/display/KNOW80/Verifying+image+and+backup+files

Ok, have read through this.    The statement:
  
If selected and corruption is detected in a backed up data block then the current live file system is checked to see if it contains the original data. If found then the Image file is repaired by replacing the corrupt data.
To me this means if a block neatly containing Apple.jpg fails, then Reflect will go back to the original source partition and will import the Apple.jpg block again to make the repair?    But if something has changed so the is not possible, that would explain the "Verification Failure" (?).

Still leaves open the question how the majority of block repairs in the first attempt at verification reported fixed (see screenshot above), re-occur in the second attempt.    It is possible I have a hardware issue, but I am puzzled by the repeatability.

Thanks for your patience in explaining this.

Still leaves open the question how the majority of block repairs in the first attempt at verification reported fixed (see screenshot above), re-occur in the second attempt

It's most likely that there is a serious memory problem that's continually corrupting data when it's read, written, compressed or decompressed, or a problem with the read/write data path. So, even the repaired blocks are corrupt again when read back.

If the file is stored on an external USB drive, then try a different USB port, change the cable and, if there's a USB hub in the connection, then remove it. Finally, try verifying the same file on a different PC.

Kind Regards

Nick - Macrium Support

Next Webinar


Edited 12 June 2021 5:24 AM by Nick
ALM4
ALM4
New Member
New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)New Member (30 reputation)
Group: Forum Members
Posts: 21, Visits: 42

Will do some experimenting and let you know the outcome, may have to invest in more hardware to isolate the problem (oh darn - grin).
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search