Purge before backup is not working properly


Author
Message
sts_66
sts_66
New Member
New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)
Group: Forum Members
Posts: 14, Visits: 23
I have Reflect V4 on a Vista PC (PC1) and V7 on a new Win10 PC (PC2) I'm in the process of setting up - ran into the same problem I had with the PC1 where Windows Task Scheduler was not executing daily full backups or weekly disk cloning operations on the PC2 but managed to get that fixed (have no idea why this problem occurs). Because I'm making relatively large changes to PC2 when I add software or change system settings, I have two Reflect backup jobs (full, not differential) set up to run every night - a disk image of active sectors (or whatever that clone method is called), and a regular backup of my Documents folder. Just checked PC2 to make sure the fix to Task Scheduler held up over time, and it did, but noticed two problems, one of which also occurs on my PC1 (both PCs backup to their own external hard drive):

1. The daily Documents folder backups are set to save only 4 backup sets, purge before running backup, but the external HD's of both PCs contain a couple dozen backups - I just tried manually deleting them from PC2's HD, but got an error that they were protected files and I could not delete them - this does not occur on PC1's external HD, I can, and have to, delete those files or the HD will get filled up and I fear backups will fail at some point when the HD reaches capacity. Why is "purge before backup" not working on either PC for this Document folder backup, and why can I not delete the excess backup files?

2. I have PC1 set up to save 3 disk images, purge before running, and that works fine - but I just checked PC2's external HD (same rules), and there's only one disk image - that's NOT GOOD - I don't have PC2 attached to a UPS, so if power goes out while a disk clone job is running I could be in major trouble, have to start from scratch rebuilding the machine, which is exactly why I purchased Macrium Reflect, to avoid that huge mess I'd be in if the internal HD or external disk image got corrupted. Why is PC2 not saving more than one disk image? I've checked the XML def file multiple times, even changed the # of images to save, but it still only keeps one regardless of what I do - there is plenty of space on the external drive (until the daily Documents job fills it up!), so that's not an issue.

Please help!


jphughan
jphughan
Most Valuable Professional
Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)
Group: Forum Members
Posts: 3.4K, Visits: 25K
The issue of deleting backups within Windows is the result of Macrium Image Guardian, which you can read about here.  That said, you shouldn't ever have to perform manual deletions, at least not as of V6, since there's a disk space threshold purge option that is meant to deal with that specific issue.  That said, the recommended practice is to set a retention policy that your destination can accommodate, since the disk space threshold purge acts on entire backup sets when triggered, rather than individual older backups as the normal retnetion policy strategy does.

For the purge issues, please post the following screenshots:
- Your retention policy configuration for your Documents folder job and your image job.
- The most recent job log for each.  Make sure not to include your Reflect key that's included at the end of every job log.

I also don't really see how PC1 and PC2 can have the same retention rules if one is on V4 and the other is on V7 given that Reflect V6 introduced a completely different retention policy engine.

Edited 6 January 2018 8:43 PM by jphughan
jphughan
jphughan
Most Valuable Professional
Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)
Group: Forum Members
Posts: 3.4K, Visits: 25K
In addition to the above, unless you're really tight on disk storage, you might want to consider unchecking "Run purge before backup" to avoid the exact power outage risk you're talking about.  That's especially true if you tend to run a lot of Full backups anyway.  Purging before the backup makes sense if your strategy involves a lot of Diff/Inc backups built on a Full and you're low on space.  In that case, by purging an older backup before running the new one, you can avoid triggering the disk space threshold purge, since that purges entire sets rather than just older backups like the regular retention policy purge.  But if that's not a factor in your strategy, then yes running the purge before the backup can indeed create some risk, which may be avoidable in your case.

And as a general note, if the Documents folder is on the same disk as the one you're capturing an image of every night, it doesn't make a lot of sense to back them both up on the same schedule and with essentially the same retention policy.  The disk image of course contains the entire Documents folder anyway, and you can extract individual files/folders out of image backups, so you're essentially just backing up the same data twice in two different ways, which is an inefficient use of storage.  If you need to retain the Documents folder backups only a bit longer than you're currently retaining your image backups, consider simply increasing the retention policy of your image backup job to cover that instead, which may be possible if you're no longer consuming destination storage with redundant backups of the Documents folder.  Backing up a specific folder that's already covered in an image backup as a separate File & Folder job only makes sense if you'll be handling the File & Folder job/backups significantly differently, e.g. running it more frequently than the image job, retaining backups significantly longer, or maybe replicating those backups elsewhere but not doing so with the image backups.

Lastly, installing software and making system changes are not relatively large changes by any stretch of the imagination, and certainly don't necessitate running a Full every night.  People run Incrementals even after upgrading their system to the next release of Windows 10.  At some point you may find that the Incrementals are too large to be space-efficient anymore, but that certainly wouldn't be every night.  Running Fulls every night is just costing you time and wear and tear on your destination disk by unnecessarily writing a lot of the same data every night.  If you doubt this, try running a manual Incremental just one night and see how the size compares.  You may want to think about weekly/monthly Fulls and daily Incs instead, which should further reduce your destination storage constraints, since it sounds like that's an issue for you anyway.

Edited 6 January 2018 9:19 PM by jphughan
sts_66
sts_66
New Member
New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)
Group: Forum Members
Posts: 14, Visits: 23
sts_66 - 6 January 2018 8:19 PM
I have Reflect V4 on a Vista PC (PC1) and V7 on a new Win10 PC (PC2) I'm in the process of setting up - ran into the same problem I had with the PC1 where Windows Task Scheduler was not executing daily full backups or weekly disk cloning operations on the PC2 but managed to get that fixed (have no idea why this problem occurs). Because I'm making relatively large changes to PC2 when I add software or change system settings, I have two Reflect backup jobs (full, not differential) set up to run every night - a disk image of active sectors (or whatever that clone method is called), and a regular backup of my Documents folder. Just checked PC2 to make sure the fix to Task Scheduler held up over time, and it did, but noticed two problems, one of which also occurs on my PC1 (both PCs backup to their own external hard drive):

1. The daily Documents folder backups are set to save only 4 backup sets, purge before running backup, but the external HD's of both PCs contain a couple dozen backups - I just tried manually deleting them from PC2's HD, but got an error that they were protected files and I could not delete them - this does not occur on PC1's external HD, I can, and have to, delete those files or the HD will get filled up and I fear backups will fail at some point when the HD reaches capacity. Why is "purge before backup" not working on either PC for this Document folder backup, and why can I not delete the excess backup files?

2. I have PC1 set up to save 3 disk images, purge before running, and that works fine - but I just checked PC2's external HD (same rules), and there's only one disk image - that's NOT GOOD - I don't have PC2 attached to a UPS, so if power goes out while a disk clone job is running I could be in major trouble, have to start from scratch rebuilding the machine, which is exactly why I purchased Macrium Reflect, to avoid that huge mess I'd be in if the internal HD or external disk image got corrupted. Why is PC2 not saving more than one disk image? I've checked the XML def file multiple times, even changed the # of images to save, but it still only keeps one regardless of what I do - there is plenty of space on the external drive (until the daily Documents job fills it up!), so that's not an issue.

Please help!


Ok, figured out the problem with disk image on PC2 - partly because of the multiple Documents backup files, there wasn't enough space on the external HD partition to save a 2nd disk clone image - original C: drive size was only 23GB, now it's grown to 100GB. So I really need to figure out how to remove those excess Document backup files and delete them, and get Reflect to properly purge the excess ones so I can have enough room for at least 2 disk images.

jphughan
jphughan
Most Valuable Professional
Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)
Group: Forum Members
Posts: 3.4K, Visits: 25K
You can delete them from within Reflect, even when Macrium Image Guardian is active, by going to the Restore tab, selecting the "File/Folder Backups" sub-tab, and then making sure that the destination containing those backups is in the "Folders to search" list.  If so, your Documents folder backups will be listed there, in which case click Other Actions > Delete backup.  But again, it sounds like you could benefit greatly from a rethink of your entire backup strategy, i.e. which jobs you run (and which jobs you therefore don't need to run), whether it really makes sense to run Full backups every day, etc.  I suspect if you look critically at this, you'll find that you'll be able to retain a greater history of backups on your destination or store your desired history while consuming much less storage.

In terms of why the purge of your Documents folder backup isn't working (assuming you even decide to keep that job, which again you may not need to), I have an idea there, but I'll wait to see those screenshots and logs first if you decide you still require assistance on that front rather than finding the solution yourself as it sounds like you just did with the image issue.

Edited 6 January 2018 9:09 PM by jphughan
sts_66
sts_66
New Member
New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)New Member (18 reputation)
Group: Forum Members
Posts: 14, Visits: 23
jphughan - 6 January 2018 8:25 PM
The issue of deleting backups within Windows is the result of Macrium Image Guardian, which you can read about here.  That said, you shouldn't ever have to perform manual deletions, at least not as of V6, since there's a disk space threshold purge option that is meant to deal with that specific issue.  That said, the recommended practice is to set a retention policy that your destination can accommodate, since the disk space threshold purge acts on entire backup sets when triggered, rather than individual older backups as the normal retnetion policy strategy does.

For the purge issues, please post the following screenshots:
- Your retention policy configuration for your Documents folder job and your image job.
- The most recent job log for each.  Make sure not to include your Reflect key that's included at the end of every job log.

I also don't really see how PC1 and PC2 can have the same retention rules if one is on V4 and the other is on V7 given that Reflect V6 introduced a completely different retention policy engine.

Ok, sure glad I discovered this problem and posted here - the retention rules/settings for the V4 and V7 are either different, or I didn't set up the Win10 backup job for the "Documents" job correctly - I had it set to save 3 full and 3 differential backups for "3 weeks", not "3 backups" - no wonder I had so many! Will check settings on the Vista PC as well, may have the same mistake there - can't believe I've been using Reflect for a decade and made a mistake like this - Doh! Also, thanks for the MIG tip - I turned it off temporarily and was able to delete the excess files so tonite's disk image should end up with two images on the external drive. Still need to change some partition sizes to make more room, but that's a different issue.

Thanks for the help - if things don't work as planned tonite after the changes I'll be back, otherwise assume I got everything working properly.

jphughan
jphughan
Most Valuable Professional
Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)Most Valuable Professional (4.9K reputation)
Group: Forum Members
Posts: 3.4K, Visits: 25K
Great! FYI even if you hadn’t deleted the extra backups manually, they would have been purged during the next occasion of that backup job in order to comply with your updated retention policy. Hope all goes well! Smile
Edited 6 January 2018 9:45 PM by jphughan
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Reading This Topic

Login

Explore
Messages
Mentions
Search