Incremental Forever (Synthetic Full Backup)


Author
Message
phrab
phrab
Proficient Member
Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)
Group: Forum Members
Posts: 148, Visits: 321
I'm fairly new to Macrium Reflect, but not new to backups, having used Acronis True Image for over a decade.  My plan was to use the Incremental Forever option, backing up every other day for a month or two.  I always have options set to verify immediately.  Then I switch to another external drive (each of which is 2TB) & do the same thing.  My questions are these:
1.  When MR creates a Synthetic Full backup, what's the danger that this full backup becomes corrupt?  I'm worried that after, say, a month & 1/2, the original full backup, or even the first incremental, becomes corrupt.  Does my plan seem reasonable.
2.  If I want to delete an older backup set, can I do this after the fact or do I have to decide on "retention rules" before I run my backup set?
Thank you in advance,
Phil

Phil
Windows 10 Pro
Windows XP- SP-3

Richard V.
Richard V.
Most Valuable Professional
Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)
Group: Forum Members
Posts: 2K, Visits: 8K
RE #1: The is always some risk of corruption with any long chain of incremental backups and that risk is not greatly affected by choosing the Incrementals Forever option.  In fact, that option's consolidation function will "fail safe" (i.e. not alter the existing state) if it encounters a problem.  (See this KB article for more details and Macrium also provides some good YouTube videos.)  Immediate verification is a wise precaution, but it doesn't eliminate the possibility of post-operative corruption due to drive deterioration or other such causes.  Multiple backups on multiple drives with off-site storage for at least one of them is the generally recommended procedure for protecting critical data.

RE #2: Reflect allows manual deletion of old backup sets at any time you choose regardless of your retention rule choices.  You should always use the Reflect user interface for that and other backup set management and not any other methods such as using the Windows File Explorer to delete backup files. That's especially important if the same destination folder is used for more than one backup set.  For some general information on Reflect's management of backup sets, see this KB article.

Regards, Richard V. ("Arvy")
https://forum.macrium.com/uploads/images/afc5d4fe-5d25-4e25-be94-185e.png

Edited 28 May 2017 5:12 AM by Arvy
phrab
phrab
Proficient Member
Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)
Group: Forum Members
Posts: 148, Visits: 321
Thank you so much.  I appreciate your quick & thorough reply!


Phil
Windows 10 Pro
Windows XP- SP-3

Richard V.
Richard V.
Most Valuable Professional
Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)
Group: Forum Members
Posts: 2K, Visits: 8K
You're welcome.  I fully understand that familiarizing oneself with any new application and its multiple options takes some considerable time and effort.  Be assured that you've made a good choice to become a member of a large group of "converts" here. Smile

Regards, Richard V. ("Arvy")
https://forum.macrium.com/uploads/images/afc5d4fe-5d25-4e25-be94-185e.png

jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 8K, Visits: 55K
Fwiw, I use a very similar strategy to you, mixing Incrementals Forever with Synthetic Fulls and disk rotation.  Macrium assures me that they have extremely robust integrity checking on the consolidation operations, and that consolidations that fail midway through for whatever reason can be recovered from, which is confirmed by logs indicating exactly that.  I'm not sure if the Verify option will actually verify the newly consolidated Full, however; I think it only verifies the new Incremental, but Macrium would be able to give you a definitive answer there.  That said, I've done manual verifications on various files in my sets and so far they've all passed, and I've been running Incrementals Forever with Synthetic Fulls for several months now.  I don't verify backups as part of the job because while I did so while I was using tapes, I never really saw the point on disks, and given that Macrium only added a verification option to Reflect as of V6 (or maybe V5?), it seems they didn't for a long time either.  I don't manually check file hashes every time I copy a file from one disk to another, after all, and the verification time might cause my backups to run unacceptably long.

For #2, yes you can delete old backup sets, but in an Incrementals Forever backup, there's only ever one set on a given destination in the first place, so that doesn't really mean anything.  If you want to reduce the size of your Incremental chain, however, you obviously can't accomplish that by simply deleting the old ones.  Macrium does offer a standalone Consolidate.exe file to reduce the chain while maintaining its integrity, however, or you can simply reduce your retention policy later and Reflect will consolidate multiple files the next time the backup job runs if that's necessary to comply with your new rules.

Edited 28 May 2017 7:24 PM by jphughan
phrab
phrab
Proficient Member
Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)
Group: Forum Members
Posts: 148, Visits: 321
Thank you jphughan for this information.  For #2, I was actually thinking that I would keep a Synthetic Full on Drive D: & then do a Synthetic Full on Drive E: & switch back & forth.  At some point, I would want to delete an old backup chain to make room.  I feel better about not having the chains too long.  After all, if you do 60-70 backups (full followed by incrementals), & something happens to the 2nd incremental, your last 60 or so backups are illusory.

With True Image, I found that sometimes after numerous incremental backups, I would get a message that it couldn't find a previous incremental & therefore couldn't create a new one.  By keeping the backup chain shorter, that chance was lessened.  And, by the way, I'm not trying to knock True Image.  It served me well for over a decade.  But their last version, 2017, has caused me nothing but problems for almost 6 months.  Their support team tried to help, but I had numerous issues.  The only thing I use it for is their "Try & Decide" function, which you start & then can download or try out any app you want.  If you don't like it, just reject all the changes...it's faster than doing a backup & then restoring.

By the way, regarding verifying, I checked the MR user manual & on p.186 it said: "The entire backup set required to restore the selected backup file will be verified. To just verify the integrity of the selected Differential or Incremental select the option 'Verify just this Differential/Incremental."
Also, I found in their knowledge base for Auto verify:
Auto VerifyAutomatically verify the resulting backup file. This will add more time to the backup process but confirms if the resulting backup can be restored from. 
Thanks again!!  This seems like a wonderful group!





Phil
Windows 10 Pro
Windows XP- SP-3

Edited 28 May 2017 9:38 PM by phrab
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 8K, Visits: 55K
^ That note about verification applies to when you perform a RESTORE, where there is an option (NOT enabled by default) to verify the backup before beginning the restore. It does not mean the entire set is verified at every backup/consolidation operation.

I agree with keeping chains shorter, but I still don't understand what you mean about deleting old chains if you're using a Synthetic full. There's only one chain in that strategy on the disk. I personally use a rotation with a different disk for each day of the week and 7 Incrementals per disk, so on any given disk I have weekly backups going back 2 months with fairly few files. I use a tool called USB Drive Letter Manager to make sure each disk is always assigned the same letter since only one disk is ever connected at s time (the others are offline and/or off-site to guard against ransomware and natural disaster), though Reflect does have other ways to accomplish this native to it. Good luck!
Edited 28 May 2017 9:55 PM by jphughan
phrab
phrab
Proficient Member
Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)
Group: Forum Members
Posts: 148, Visits: 321
Hi jphughan,
Auto Verify Automatically verify the resulting backup file. This will add more time to the backup process but
confirms if the resulting backup can be restored from."  I guess I assumed that that would mean all the previous backups were OK, Otherwise, you couldn't restore from the last one.  However, I will check with support to be sure.

As far as what I meant by deleting old chains is that what I was planning to do is create Synthetic Full #1 on Drive D, then create Synthetic Full #2 on Drive E, Synthetic Full #3 on Drive D, etc.  At some point, I would go back & delete Synthetic Full #1.  I was thinking that this would keep the chains shorter than continuing with just one chain per external drive.

Thanks again!
Phil


Phil
Windows 10 Pro
Windows XP- SP-3

jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 8K, Visits: 55K
Ok, I was thinking of a different option.  The Verify option that applies to backup creation jobs, I believe, will only verify the most recently created backup file.  It will not go back and verify the entire set every single time, since that could take forever and would result in all of your data constantly being re-verified.  I believe the idea there is that there would be "transitive trust", i.e. presumably if you had that option set from the beginning, then you've already verified those earlier backups anyway. However, I'd be curious to know if that Verify option will verify a newly generated synthetic full after a consolidation operation occurs. But the option I was talking about above is one you can check immediately before initiating a restore.  That option WILL verify as much of the set is necessary to execute the restore, but again that's something that only occurs immediately prior to a restore, not after a backup is created.

You can't span a chain across multiple drives.  You might want to more clearly detail your intended schedule, settings, and disk rotation, because it sounds like you may be about to attempt something that won't work as you want it to.  You can't delete a Synthetic Full without killing all of the backups that were built off of it, and Synthetic Fulls aren't something that are typically created manually in the first place; they're created (and then updated) over the course of scheduled backups in order to comply with your specified retention policy.  The only way to keep your chains shorter is to reduce the number of Incrementals you retain, and if you want to do that while retaining the option to go back a given amount of time in the past, you'd either need to reduce the frequency of backups or add more disks to your rotation.  Like I said, I have a strategy that in aggregate gives me daily backups going back 2 months, but I only have a chain of 7 Incrementals on any given disk.

Edited 29 May 2017 12:21 AM by jphughan
phrab
phrab
Proficient Member
Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)
Group: Forum Members
Posts: 148, Visits: 321
jphughan - 29 May 2017 12:14 AM
You can't span a chain across multiple drives.  You might want to more clearly detail your intended schedule, settings, and disk rotation, because it sounds like you may be about to attempt something that won't work as you want it to.

Thank you again for your reply.  This is what I thought I could do, but please tell me if this won't work.  I have 2 external drives: D & E.  I wanted to be able to start with a Full backup & then do incrementals every other day.  I would make the decision how many incrermentals to keep flexible, which is why I chose the Synthetic Full backup.  My plan was to start on Drive D, choosing Incrementals Forever as the backup plan, backing up every 2nd day until I have about 30 incrementals (~2 months worth).  Then I would stop the schedule (although I don't know how to do that yet).  Then I would do the same thing on Drive E (~2 months of backups, doing it every other day).

Then I wanted to repeat those steps by creating a new Synthetic Full backup on Drive D, independent  of the previous Synthetic Full backup & do that for about 2 months.  Then I would repeat on Drive E.  My idea was to do this many times, having a few complete Synthetic Full backup sets on each external drive.  Actually, some of them might now be Synthetic Full because if there's space on the drive, it would just be a Full backup followed by ~30 incrementals.

At some point, I would want to delete an entire old backup set.  Am I able to do this or just confused?

By the way, I'm not sure how to edit the schedule of an existing backup plan to stop future backups.  I couldn't find a "Do not schedule" option.

Phil
Windows 10 Pro
Windows XP- SP-3

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search