Incremental Forever (Synthetic Full Backup)


Author
Message
phrab
phrab
Proficient Member
Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)Proficient Member (252 reputation)
Group: Forum Members
Posts: 148, Visits: 321
I'm fairly new to Macrium Reflect, but not new to backups, having used Acronis True Image for over a decade.  My plan was to use the Incremental Forever option, backing up every other day for a month or two.  I always have options set to verify immediately.  Then I switch to another external drive (each of which is 2TB) & do the same thing.  My questions are these:
1.  When MR creates a Synthetic Full backup, what's the danger that this full backup becomes corrupt?  I'm worried that after, say, a month & 1/2, the original full backup, or even the first incremental, becomes corrupt.  Does my plan seem reasonable.
2.  If I want to delete an older backup set, can I do this after the fact or do I have to decide on "retention rules" before I run my backup set?
Thank you in advance,
Phil

Phil
Windows 10 Pro
Windows XP- SP-3

jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 8K, Visits: 55K
Ok, this post is going to be a mixture of clarifications in terms of how Reflect and synthetic fulls work, and also some of my own best practice recommendations.

First, the only time Reflect ever generates a synthetic full is when it creates a new Incremental and the resulting number of Incrementals in the chain exceeds the number you've specified in your retention period.  For example, in my "Incrementals Forever with Synthetic Fulls" job, I specify to retain 7 Incrementals.  The first time my job executed, it generated a Full.  Then each of the next 7 times, Incrementals were generated.  The time after that, an 8th Incremental was generated, but since that exceeds my retention policy, at that point Reflect automatically merged Incremental #1 into my Full backup, thereby "carrying the Full forward" so that I have a Full equivalent to having actually captured a Full on the day Incremental #1 ran, even though I didn't actually run a Full that day -- hence the name "synthetic full".  Incremental #1 is deleted after the Full is updated with its contents (thereby keeping my backup set compliant with my retention policy), and of course I lose the ability to restore all the way back to the day I ACTUALLY ran that Full.  From that point on, this process of merging the oldest Incremental into the root Full continues each time a new Incremental is generated, thereby keeping your backup set current without the number of Incrementals ever exceeding the retention policy you specify.  But if you set a retention policy of 30 Incrementals, you won't ever get a synthetic full until Incremental #31 is being created.  The strategy you're describing above sounds like you'd only ever have a Full followed by 30 Incrementals, and then you'd manually capture a new Full -- nothing at all wrong with that strategy, but you'd never get a Synthetic Full out of that.

Now, a few best practice recommendations.  If you've got 2 targets, I would recommend that you instead rotate them back and forth on a weekly basis rather than keeping both connected all the time and backing up to one of them for 2 months before switching to the other.  Weekly rotations will allow you to keep some of your backups offline (safe from ransomware), which additionally gives you the option to store them off-site for protection against break-ins, natural disasters, etc.  And by rotating them weekly, you'll ensure that even if you do lose the drive with the most recent backup, the "fallback" drive is no more than a week out of date rather than 2 months.  Finally, you wouldn't even need 2 separate schedules.  If only one drive is ever connected at a time and it's assigned a consistent drive letter (there are tools to enforce this), you could just set up a single schedule and Reflect will simply back up to whichever set is available at the time a backup runs.  Backup sets on different disks are wholly independent from each other (i.e. you would never have a situation where an Incremental chain requires files from two different disks).  Alternatively, you can use Reflect's built-in support for specifying additional destinations and targeting them by Disk ID rather than drive letter, in which case Reflect will again use whichever drive is available at any given time when the backup runs, or the first available drive in the list if multiple targets are available, all regardless of drive letter assignment.  Easier scheduling and better resilience -- what's not to love? Smile

However, if you do want to temporarily pause a scheduled task, you can do so under the Scheduled Tasks tab by right-clicking it and choosing "Disable" -- although this will disable ALL scheduled backup types for that definition file.  If you want to stop all scheduled jobs for a task permanently, you'd just go to Edit Schedule and remove all scheduled backup types.  The definition file would still stay there in case you wanted to run it manually.

Yes, you can delete older backup sets if you want, but the idea of "Incrementals Forever with Synthetic Fulls" is that you don't ever have more than one set on a given disk.  Instead, you just decide how far back you'd ever need to restore, and set your retention period as appropriate to achieve that based on the frequency of backups.  For example, if you wanted 2 months of history on each of your 2 disks and you were rotating them weekly with backups running every day, you'd just set your retention policy to 27.  The Full plus those 27 Incrementals is 28 backups, or 4 weeks' worth -- but since each disk is only being used every other week, that allows you to go 8 weeks back in time, with each disk having alternate weeks' backups on it.   Does that make sense?

Edited 29 May 2017 11:42 PM by jphughan
Richard V.
Richard V.
Most Valuable Professional
Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)Most Valuable Professional (4.1K reputation)
Group: Forum Members
Posts: 2K, Visits: 8K
If I wanted to restore to 6/3, could I?
Yes.

If so, would I have to have both Drive D & E available or just Drive E?
Just "Drive E".  (And the recovery target drive, of course.)

Either of the two backup sets can be used for recovery to any of its "snapshots" that remain available after the most recent consolidation of that backup set.  In fact, if you wanted to, you could revert to the pre-6/3 status of the consolidated full in that set, but you'd need to look at it in Reflect's user interface (rather than Windows File Explorer) to determine exactly what date-time status that consolidation represents.

I'll leave it for you to ask if you want a more "long-winded" dissertation on the subject of backup set management, etc., but this KB article provides an overview.


Regards, Richard V. ("Arvy")
https://forum.macrium.com/uploads/images/afc5d4fe-5d25-4e25-be94-185e.png

Edited 17 June 2017 6:49 PM by Arvy
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 8K, Visits: 55K
Do not rely on the Date Created or Date Modified timestamps in Windows when using Synthetic Fulls or Incremental Merge. As Arvy said, I can give more detail as to why if you're curious, or you can just look at the Backup Date in Reflect's Restore tab or the Reflect property tab that's available when you right-click a backup file and select Properties.
Edited 17 June 2017 6:50 PM by jphughan
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 8K, Visits: 55K
Now that I'm not typing from a phone anymore, I'll provide the detail in case the OP or anyone else who happens on this thread later is curious:

The Date Created timestamp will be misleading on any backup file that has had a newer backup merged into it, i.e. the root Full in a Synthetic Fulls strategy. The reason is that the Date Created timestamp will always have the date that the file was originally created, and as the contents of that file are "carried forward", the Date Created timestamp is NOT updated -- after all, the merge simply updated an existing file as opposed to creating a brand new one. But this means that as soon as the first merge occurs, if you look at the Date Created timestamp in Windows, you'll appear to have a backup available from longer ago than you actually do. If you're using Incrementals Forever without Synthetic Fulls (called Incremental Merge), this will be observed on the oldest Incremental in the chain since it would be the file receiving merges rather than the root Full, which remains static.

The Date Modified timestamp can be problematic both for any backup file that has other backups merged into it and possibly ALL files in the backup chain; the latter case depends on whether you're using Delta Incremental Indexes, which is enabled by default in V7 and disabled by default in V6.  The first case is problematic because let's say you're using Synthetic Fulls and have a Full captured on May 1. It eventually gets the first Incremental in the chain from May 2 merged into it. The result is a Synthetic Full with a date of May 2 -- but the actual merge operation might have occurred on June 1, for example, and therefore the Full will have a Date Modified timestamp of June 1. Windows of course doesn't know or care that the file in question contains a backup from May 2. This leads to the strange result that if Delta Incremental indexes are enabled (more on this momentarily), your Full will appear to be the newest file in the set according to Windows (or second newest behind the most recent Incremental if you have "Run purge before backup" enabled), even though it of course contains the OLDEST backup.  If you're only using Incremental Merge rather than Synthetic Fulls, this again will be observed on the oldest Incremental instead.

The second issue revolves around Delta Incremental Indexes. In short, if you have them enabled, then the Date Modified timestamps on the Incrementals in your chain (except the first one in an Incremental Merge strategy) WILL correspond to the time that the backup jobs they contain actually completed. Reflect's Backup Date however shows the time the job started, so those will still always be a bit off. But if you have that option DISABLED, then each Incremental contains a full index of the data contained in the backup set up to the time of that Incremental, as opposed to Delta indexes where each Incremental only contains an index of the files contained in that specific Incremental.  The reason that becomes an issue is that when a merge occurs between older backup files in the chain, that of course changes the data in the set, and if all Incrementals contain a full index up to their own backup date, then all of them need to be modified with an updated index that reflects the changes made by the merge. Consequently, in a Synthetic Full strategy, as soon as the first merge occurs in your backup set, ALL files in the chain will always have a Date Modified timestamp of the most recent backup job's completion.  If you're using Incremental Merge instead of synthetic fulls, then this will still be true EXCEPT for the root Full, since that isn't being carried forward. Further reading on Delta Incremental Indexes if you'd like: http://knowledgebase.macrium.com/display/KNOW/Delta+Indexes+for+Incremental+Backups

Edited 17 June 2017 11:24 PM by jphughan
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search