Free space threshold size question


Author
Message
MsT18
MsT18
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 15, Visits: 125
What is the recommended threshold for free space when scheduling images for a drive (for deleting old backups)? Would it be best to set it just a bit larger than the expected size of a full backup?

I'm doing the standard (almost standard) Full, differential and incremental imaging schedule - grandfather, father, son.
Edited 17 October 2020 8:15 AM by MsT18
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 7.8K, Visits: 54K
The free space threshold purge can be executed mid-job if needed, so you don’t have to set that based on the predicted size of a backup. It’s just a question of how much free space you want to have on your destination at a minimum. Back when drives were small, I tried not to let free space drop below about 20%, but in this era of multi-TB drives, that seems excessive. If you’ll be using this destination drive solely for backups, then you could set it quite low. Just be aware that the free space purge should NOT be seen as a substitute for setting a retention policy that is achievable based on the size of your backups and your destination capacity. It should instead be seen as a last resort. The reason is that the free space purge deletes entire backup SETS, whereas the regular retention policy can delete individual Diff or even Inc backups (through consolidation).

The free space purge option is there if you want to make sure that new backups will never fail due to lack of free space, even if that means having to sacrifice older backups earlier than the retention policy would have purged them. But you should still set a sustainable retention policy.
MsT18
MsT18
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 15, Visits: 125
Thank you @jphughan. Your reply is very helpful, particularly your warning. I think I understand it all better now. I'll keep the free space threshold low to avoid losing the oldest backup set.

Thinking out loud - the template I've chosen is to keep monthly full backups for a year, weekly differential backups for four weeks and daily incrementals for 14 days. That means there's some overlap and some opportunity to go back to recent backups if I need a previous version of a file (e.g. a newly created file has been changed, deleted or become corrupt or whatever). To keep full backups for more than a year I'd swap out the hard drives each year (or sooner) and stow them somewhere safe.

The times I've needed a backup are when a computer and/or hard drive fails or becomes corrupt, and (rarely) when a file is deleted or changed. That means 12 months of full backups would normally be ample. Doing the sums, I'd need larger hard drives than I'm currently using to accommodate a year of monthly full backups. Alternatively, I could swap the backup hard drives out at six or eight months and start over. (I probably should arrange for duplicate backups in any case. I'll give some more thought to the best backup system for my needs going forward.)
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 7.8K, Visits: 54K
Happy to help! One thing you might consider is swapping hard drives every week or so. That way if a hard drive storing your backups fails, your surviving drive has a backup that’s at most a week old. The issue is that due to quirks of human timekeeping practices, rotating every week is at odds with an “every month” Full schedule, since the intervals of weeks and months don’t align. The workaround to this would be to schedule your Fulls to occur every 4 weeks.
MsT18
MsT18
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 15, Visits: 125
jphughan - 18 October 2020 5:08 AM
Happy to help! One thing you might consider is swapping hard drives every week or so. That way if a hard drive storing your backups fails, your surviving drive has a backup that’s at most a week old. The issue is that due to quirks of human timekeeping practices, rotating every week is at odds with an “every month” Full schedule, since the intervals of weeks and months don’t align. The workaround to this would be to schedule your Fulls to occur every 4 weeks.

Good idea. Is that where I'd use the Alternative Locations for backups, entering the drive letter and folder of the alternate drive? 
When using two drives, what happens with incremental and differential backups? Does Macrium look at each drive/folder and back up according to what's on that drive, or does it remember what it backed up last time and only back up what's different to the last backup?
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 7.8K, Visits: 54K
There are a few ways you can handle a disk rotation:
  • Manually assign specific drive letters to each destination and then use the Alternative Locations feature to include the drive letters of all targets.  If you do this, pick letters far down in the alphabet, because if Windows ever assigns that drive letter to anything ELSE, even if by the "next available letter" mechanism, then it will "forget" that your destination disk was supposed to use that letter via manual assignment and will instead revert to "next available" with that disk.  So if you use letters early in the alphabet, that scenario becomes more likely.
  • Use an external application to guarantee that all destination disks are assigned the same drive letter whenever one of them is connected, in which case you can use a single destination in Reflect.  I use this mechanism at a client that has a rotation of 9 disks and is already consuming a fair number of letters with mapped network drives, which makes the above option a bit more problematic.
  • Switch Reflect to identifying destinations by unique Volume ID rather than drive letter.  In this case, Reflect will still show a drive letter, but it will actually FIND the destination by the partition's unique ID.  This way, even if the drive letter changes, Reflect will still see it (and automatically start showing a new drive letter in its interface).  You'd still need to use the Alternative Locations feature to list all possible destinations, and you'd also have to update that list if you ever repartitioned one of your existing drives, since that would change its Volume ID.  I believe simply reformatting the existing partition would do that as well, but I'm not sure.
But whichever mechanism you choose, Reflect always makes its backup sets "self-contained", meaning that you won't end up with Incremental #2 on Disk B being dependent on Incremental #1 that resides on Disk A.  If you have a given disk, then you have all that you need to restore any given backup on that disk.  If it were otherwise, then a disk rotation wouldn't be as helpful, because restoring a backup from Disk B might require you to have backups on Disk A, which might have failed.  This type of design would also require Reflect to maintain a local catalog of all backups it had created everywhere and generally make managing them more complex, especially if you wanted to be able to choose whether certain backups on different destinations should be linked to each other or deliberately independent of each other.  Instead, Reflect has a design whereby each time it runs a backup, it looks at the available destination folder for any existing backups and works with whatever is there at that time, thus removing the need for a local catalog, simplifying the relationship of backups to each other, and keeping your destination disks independent of each other. Smile

MsT18
MsT18
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 15, Visits: 125
Excellent. That saves in many ways.

First of all, I won't need to worry about when I swap the drives over - if I neglect to swap them as scheduled, a day or even a week or a month won't make much difference unless one of the backup drives fails. Also, it saves me having to keep many different drives running at the same time, using up power and hogging USB ports, drive docks and shelf space.

Thanks again, @jphughan. I'll get onto setting this up right away Smile
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 7.8K, Visits: 54K
Happy to help! The only thing you DO sort of have to worry about with a GFS schedule is swapping disks before a “Full day” or a “Diff day”, because if you don’t, then one disk will lose an older set/Diff earlier than expected and the other disk won’t have a brand new set/Diff as recently created as normal. This is one reason why I like using Incrementals Forever with Synthetic Fulls when using a rotation, since every backup is an Incremental, which means there’s no need to worry about this. The normal hazard of this strategy is that you only ever have one set at your destination, which means corruption within the Full or even a few unreadable sectors in there could render all of your backups useless — but with a disk rotation, you have multiple backup sets in total, so that risk is mitigated.
MsT18
MsT18
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 15, Visits: 125
jphughan - 18 October 2020 2:23 PM
Happy to help! The only thing you DO sort of have to worry about with a GFS schedule is swapping disks before a “Full day” or a “Diff day”, because if you don’t, then one disk will lose an older set/Diff earlier than expected and the other disk won’t have a brand new set/Diff as recently created as normal. This is one reason why I like using Incrementals Forever with Synthetic Fulls when using a rotation, since every backup is an Incremental, which means there’s no need to worry about this. The normal hazard of this strategy is that you only ever have one set at your destination, which means corruption within the Full or even a few unreadable sectors in there could render all of your backups useless — but with a disk rotation, you have multiple backup sets in total, so that risk is mitigated.

You're suggesting if the diffs and fulls are scheduled for a Monday, it's wise to swap the drives on a Sunday after the most recent incremental - is that right?
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)Macrium Evangelist (11K reputation)
Group: Forum Members
Posts: 7.8K, Visits: 54K
Exactly right. Smile If you MISS that swap and your Monday non-Incremental backup goes to the wrong disk, you’d have to decide whether to a) do the swap late and then manually run a Full/Diff to that new disk as a “makeup”, or b) skip the swap entirely and instead run an entire second week on that original disk. Swapping the disk late and NOT running the missed Full/Diff to that disk manually can mess up your retention policy a bit, in addition to creating a longer Incremental chain on account of the Diff/Full that was supposed to exist on that disk by that point but doesn’t.
Edited 19 October 2020 2:11 AM by jphughan
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search