Backup thinning with synthetic full?


Author
Message
kumar
kumar
New Member
New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)
Group: Forum Members
Posts: 2, Visits: 3
Hi all, 

Curious if there is a solution out there to implement backup thinning such as Time Machine/Borg Backup can do, eg. keep daily backups for a week, weekly backups for a month, monthly backups for a year, etc. Even better if the solution can work with the Synthetic Full backup scheme. I think I came across a post somewhere here saying it was possible, though I don't see anything built in to Reflect to do this. Did someone write a script to pull this off?
Froggie
Froggie
Master
Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)
Group: Forum Members
Posts: 999, Visits: 8.5K
You don't mention what type of "backups" (images) you are referring to.  Using the GFS scheme (DAILY Incrementals, WEEKLY Differentials and MONTHLY Fulls), this is pretty easy to do using the Reflect Retention schemes.
Edited 20 December 2018 2:39 PM by Froggie
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)
Group: Forum Members
Posts: 4.7K, Visits: 34K
Backup thinning over time is easy to accomplish with a GFS scheme.  With a Synthetic Full setup, Reflect's retention policy only allows you to choose to keep the last X number of backups, so you can't thin your backups out over time.  Theoretically if you were willing to use the standalone Consolidate.exe tool to manually "collapse" backups from periods where you don't care about retaining multiples anymore, you could achieve this, but Reflect's retention policy isn't designed to achieve that, and just thinking about it, I can't really picture a UI where that capability could be easily presented to a user while retaining the existing functionality.  Right now Reflect allows you to specify retention periods/quantities for each type of backup, which in conjunction with proper scheduling is how one can end up with fewer backups as time goes on, because the remaining older backups at less frequent intervals are different types of backups.  What you're asking for is a way to use only Incremental backups (except for the root Full) and have Reflect retain different quantities of that one backup type within different blocks of time.

I'm not familiar with Borg Backup, but Time Machine works completely differently from Reflect.  First, it uses file-level rather than block-level backups, and it also stores its backups essentially as one large database rather than a series of containers like Reflect and other traditional backup solutions.  There are pros and cons to each approach, but in this situation Time Machine does have a flexibility advantage.  Windows File History is probably the nearest Windows equivalent to Time Machine from a simplicity standpoint, but unlike Time Machine its backups cannot be used to restore the entire system; you can only back up data folders with it.  But I believe it can thin out older backups.  It's not explicitly configurable or even mentioned in the UI, but Time Machine doesn't make this configurable or explicitly known either.

Edited 20 December 2018 3:07 PM by jphughan
kumar
kumar
New Member
New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)New Member (3 reputation)
Group: Forum Members
Posts: 2, Visits: 3
Froggie/jphugan - thanks for the replies! Yeah, I was hoping to do some trickiness with consolidate.exe to remove old incrementals while keeping the chain intact, but it looks like that’s trickier than I’d hoped
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)Macrium Evangelist (6.9K reputation)
Group: Forum Members
Posts: 4.7K, Visits: 34K
kumar - 20 December 2018 3:09 PM
Froggie/jphugan - thanks for the replies! Yeah, I was hoping to do some trickiness with consolidate.exe to remove old incrementals while keeping the chain intact, but it looks like that’s trickier than I’d hoped

Well if you're willing to use Consolidate.exe, it's actually not that tricky.  If you have a week's worth of Incrementals that you want to bring down to one, then you'd just specify the two "ends" of the backup range you want to collapse in the From and To fields of Consolidate.exe and let it do its work.  The single backup you'll have left will be the newest one of the range you selected.  The problem is that again you'd be doing it manually all the time.  It would be interesting to achieve something like this in a script, but doing that reliably would require the script to have a way to determine the date that the backups in question were captured.  That's available in the file metadata, but I don't know if it can be retrieved with a script.  I've got a thread for a "Reflect PowerShell module" that would allow accessing this and other data and offer other cmdlets to do all kinds of things, but we don't have it today.  And even if we had that, I don't think Consolidate.exe currently supports command-line execution anyway.  But if you're ok with manual consolidations as your backups age, then Consolidate.exe is actually pretty easy to use.

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search