Need advice on scaling back backups to save disk space.


Author
Message
HGinDC
HGinDC
New Member
New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)
Group: Forum Members
Posts: 24, Visits: 48
In one week I have managed to almost up fill my new 6TB external disk (ESmile dedicated to backups. I am doing one image of C: and one file-and-folder backup of a directory on D:. I believe I will have to adjust retention settings. What would you suggest. The file usage on E: and the specs for the backups are shown below. Thanks!

​​



​​​​
Froggie
Froggie
Master
Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)
Group: Forum Members
Posts: 944, Visits: 8K
A quick look says you're most likely not using any compression on your OS drive images... this is a big waste of space.  You should be using Reftect's recommended COMPRESSION... that should cut down your image sizes by at least 40%.  That can be changed by editing your backup definition and in the "Advanced Options" section under "Compression," set it for MEDIUM compression and make sure the "Intelligent Sector Copy" is selected.

A second look says you're using DIFFERENTIAL imaging for your between FULL operations, but they aren't growing very much so much of your DIFFERENTIAL space is getting re-imaged.  I would switch that image to INCREMENTAL imaging for smaller between FULL snapshots.

...and 6tB of storage will never hold the 26-weeks of FULL images your retention is set for... you need to cut that down to hold much less.

...and since Reflect images block changes on your disk, you should not be defragmenting your disk except right before your scheduled FULL images

Edited 8 January 2019 12:17 AM by Froggie
HGinDC
HGinDC
New Member
New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)
Group: Forum Members
Posts: 24, Visits: 48
Froggie - 8 January 2019 12:13 AM
A quick look says you're most likely not using any compression on your OS drive images... this is a big waste of space.  You should be using Reftect's recommended COMPRESSION... that should cut down your image sizes bu at least 40%.  That can be changed by editing your backup definition and in the "Advanced Options" section under "compression," set it for MEDIUM compression and make sure the "Intelligent Sector Copy" is selected.

A second look says you're using DIFFERENTIAL imaging for your between FULL operations, but they aren't growing very much so much of your DIFFERENTIAL space is getting re-imaged.  I would switch that image to INCREMENTAL imaging for smaller between FULL snapshots.

...amd 6tB of storage will never hold the 26-weeks of FULL images your retention is set for... you need to cut that down to hold much less.

Thanks. I was using medium compression with Intelligent Sector Copy enabled. I have, however, changed the settings to keep only 8 weeks of full images and 7 incremental backups.
HGinDC
HGinDC
New Member
New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)
Group: Forum Members
Posts: 24, Visits: 48
Froggie - 8 January 2019 12:13 AM
A quick look says you're most likely not using any compression on your OS drive images... this is a big waste of space.  You should be using Reftect's recommended COMPRESSION... that should cut down your image sizes by at least 40%.  That can be changed by editing your backup definition and in the "Advanced Options" section under "Compression," set it for MEDIUM compression and make sure the "Intelligent Sector Copy" is selected.

A second look says you're using DIFFERENTIAL imaging for your between FULL operations, but they aren't growing very much so much of your DIFFERENTIAL space is getting re-imaged.  I would switch that image to INCREMENTAL imaging for smaller between FULL snapshots.

...and 6tB of storage will never hold the 26-weeks of FULL images your retention is set for... you need to cut that down to hold much less.

...and since Reflect images block changes on your disk, you should not be defragmenting your disk except right before your scheduled FULL images

Also,I wasn't aware that I was defragmenting this disk at all. Did you see something indicating I was defragmenting it?​
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)
Group: Forum Members
Posts: 4.2K, Visits: 30K
Ok, right off the bat I see some larger problems.  First, your image backup is called "Backup image of C", but in the list of Operations underneath, the D drive is mentioned, right after a tiny 16MB partition that appears to be the MSR partition.  So it appears that your image job is including a partition you don't intend to be backing up in that job, and seems NOT be including one or more partitions you do want, like your C drive.  The "or more" part is because if you have an MSR partition, you would also likely have an EFI partition that I don't see included in that Operations list.  If you want your image backups to be usable for system restores, you really want to back up all of the system partitions.  Reflect can show you what you SHOULD be selecting if you click "Create an image of the partitions required to back up and restore Windows" in the Backup Tasks area of the interface on the left.  The partitions selected by default in the wizard that appears after clicking that should be considered your "minimum viable backup" for system restore purposes.  So if you're not currently backing up all of those partitions in your image job, you should fix that asap

In terms of the original question, how large would a Full backup of an image job that contains the proper partitions be?  The real answer here might be getting more storage for backups.  I see that your Full backup of your F&F job is 1.4 TB.  The Full backup of the image job in your screenshot also weighs in around that much, but that seems to be because it's backing up the wrong partition.  The similarity to the size of the F&F job actually jumped out at me and suggested something might be amiss, such as you potentially backing up the same data two different ways.

If a proper image job isn't very large (relative to your F&F job), then 6TB on the destination might still be enough to have a "safe" strategy.  I would however make sure that you're able to retain at least 2 Fulls from each job.  It looks like you're only planning to retain one from each at the moment, which means if there's ever a problem reading part of that Full backup, you have no usable backups at all.  Of course if there's ever a problem with that 6TB disk, you have the same issue, so apart from all of this, you might want to consider a rotation or replication strategy so that you have backups on multiple destinations.  One option you might consider would be running a Full backup every 2 weeks and running Incrementals rather than Differentials all other days.  The additional Full will make your backups more robust, and Incrementals will be smaller than Differentials.

Additionally, how large is the bzvol folder you're excluding from your F&F job?  If it's not too large (and you don't need to worry about restoring custom NTFS permissions), you might want to consider switching your D drive backup to an image job (with just that one partition) rather than an F&F job.  That will further shrink your Incrementals because Reflect will then be able to back up only the changed portions of files.  With an F&F job, if the file has changed, Reflect has to back up the entire file again, even if only a tiny part of a very large file changed, which obviously means larger Incrementals.  But if your bzvol folder is large, then I suppose image backups might be larger overall.

Finally, I would suggest setting your retention policy in number of backups rather than number of days.  I've seen cases where people use time-based retention policies, then go on vacation during which time they perform no backups, and then the first time they run a backup after returning, Reflect deletes many or even all of their backups because they're all technically "expired", and that's normally not what they intend.  Specifying retention in number of backups avoids that outcome. (UPDATE: And as Froggie mentioned above, you should set your retention policy to something that's realistic based on the size of your backups and the amount of storage available on your destination.  You really don't want to rely only on the free disk space threshold purge.)

Edited 8 January 2019 12:47 AM by jphughan
HGinDC
HGinDC
New Member
New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)
Group: Forum Members
Posts: 24, Visits: 48
jphughan - 8 January 2019 12:30 AM
Ok, right off the bat I see some larger problems.  First, your image backup is called "Backup image of C", but in the list of Operations underneath, the D drive is mentioned, right after a tiny 16MB partition that appears to be the MSR partition.  So it appears that your image job is including a partition you don't intend to be backing up in that job, and seems NOT be including one or more partitions you do want, like your C drive.  The "or more" part is because if you have an MSR partition, you would also likely have an EFI partition that I don't see included in that Operations list.  If you want your image backups to be usable for system restores, you really want to back up all of the system partitions.  Reflect can show you what you SHOULD be selecting if you click "Create an image of the partitions required to back up and restore Windows" in the Backup Tasks area of the interface on the left.  The partitions selected by default in the wizard that appears after clicking that should be considered your "minimum viable backup" for system restore purposes.  So if you're not currently backing up all of those partitions in your image job, you should fix that asap

In terms of the original question, how large would a Full backup of an image job that contains the proper partitions be?  The real answer here might be getting more storage for backups.  I see that your Full backup of your F&F job is 1.4 TB.  The Full backup of the image job in your screenshot also weighs in around that much, but that seems to be because it's backing up the wrong partition.  The similarity to the size of the F&F job actually jumped out at me and suggested something might be amiss, such as you potentially backing up the same data two different ways.

If a proper image job isn't very large (relative to your F&F job), then 6TB on the destination might still be enough to have a "safe" strategy.  I would however make sure that you're able to retain at least 2 Fulls from each job.  It looks like you're only planning to retain one from each at the moment, which means if there's ever a problem reading part of that Full backup, you have no usable backups at all.  Of course if there's ever a problem with that 6TB disk, you have the same issue, so apart from all of this, you might want to consider a rotation or replication strategy so that you have backups on multiple destinations.  One option you might consider would be running a Full backup every 2 weeks and running Incrementals rather than Differentials all other days.  The additional Full will make your backups more robust, and Incrementals will be smaller than Differentials.

Additionally, how large is the bzvol folder you're excluding from your F&F job?  If it's not too large (and you don't need to worry about restoring custom NTFS permissions), you might want to consider switching your D drive backup to an image job (with just that one partition) rather than an F&F job.  That will further shrink your Incrementals because Reflect will then be able to back up only the changed portions of files.  With an F&F job, if the file has changed, Reflect has to back up the entire file again, even if only a tiny part of a very large file changed, which obviously means larger Incrementals.  But if your bzvol folder is large, then I suppose image backups might be larger overall.

Finally, I would suggest setting your retention policy in number of backups rather than number of days.  I've seen cases where people use time-based retention policies, then go on vacation during which time they perform no backups, and then the first time they run a backup after returning, Reflect deletes many or even all of their backups because they're all technically "expired", and that's normally not what they intend.  Specifying retention in number of backups avoids that outcome.

Y
OMG y​ou are exactly correct: I made a mistake specifying the image backup source! I intended to point it to my C: drive (a 1TB SSD where Win 10 is installed) but mistakenly pointed it to my D: drive, where my media files (documents, pictures, music) are stored. That was a big mistake. As you noticed as well, the F&F backup​ is ALSO pointed at the D: drive, but that is what I intended. Thank you for catching this!

Now what to do? I'm thinking I'll just delete the image definition file and start over with it. Would deleting the definition file also delete the backups on the E: drive? If not, then I assume that I would want to delete them manually so I can start over with a correct backup of my C: drive. Do you agree? Thanks for your advice.

--Howard​​​​​
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)
Group: Forum Members
Posts: 4.2K, Visits: 30K
Glad I could help!  And good thing you found out now rather than after a failure where you needed to restore something!

Deleting a definition file only deletes the definition file, not any backups it generated.  To delete backups, go to the Restore tab in Reflect, then select any backup you want to delete, and click Other Actions > Delete file.  The dialog that will pop up at that point will allow you to select multiple backups to delete in a single pass.  At that point, simply edit the definition file to include the correct partitions based on the advice I provided above and save your changes.  Future jobs will run properly.

Hopefully the other advice about the question you actually asked is useful too! Smile

HGinDC
HGinDC
New Member
New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)
Group: Forum Members
Posts: 24, Visits: 48
jphughan - 8 January 2019 1:07 AM
Glad I could help!  And good thing you found out now rather than after a failure where you needed to restore something!

Deleting a definition file only deletes the definition file, not any backups it generated.  To delete backups, go to the Restore tab in Reflect, then select any backup you want to delete, and click Other Actions > Delete file.  The dialog that will pop up at that point will allow you to select multiple backups to delete in a single pass.  At that point, simply edit the definition file to include the correct partitions based on the advice I provided above and save your changes.  Future jobs will run properly.

Hopefully the other advice about the question you actually asked is useful too! Smile

Your other advice is definitely useful and I intend to follow it. Thanks again for your help. The level of assistance available on this forum certainly enhances the value of Macrium's product.
HGinDC
HGinDC
New Member
New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)New Member (29 reputation)
Group: Forum Members
Posts: 24, Visits: 48
jphughan - 8 January 2019 12:30 AM
Ok, right off the bat I see some larger problems.  First, your image backup is called "Backup image of C", but in the list of Operations underneath, the D drive is mentioned, right after a tiny 16MB partition that appears to be the MSR partition.  So it appears that your image job is including a partition you don't intend to be backing up in that job, and seems NOT be including one or more partitions you do want, like your C drive.  The "or more" part is because if you have an MSR partition, you would also likely have an EFI partition that I don't see included in that Operations list.  If you want your image backups to be usable for system restores, you really want to back up all of the system partitions.  Reflect can show you what you SHOULD be selecting if you click "Create an image of the partitions required to back up and restore Windows" in the Backup Tasks area of the interface on the left.  The partitions selected by default in the wizard that appears after clicking that should be considered your "minimum viable backup" for system restore purposes.  So if you're not currently backing up all of those partitions in your image job, you should fix that asap

In terms of the original question, how large would a Full backup of an image job that contains the proper partitions be?  The real answer here might be getting more storage for backups.  I see that your Full backup of your F&F job is 1.4 TB.  The Full backup of the image job in your screenshot also weighs in around that much, but that seems to be because it's backing up the wrong partition.  The similarity to the size of the F&F job actually jumped out at me and suggested something might be amiss, such as you potentially backing up the same data two different ways.

If a proper image job isn't very large (relative to your F&F job), then 6TB on the destination might still be enough to have a "safe" strategy.  I would however make sure that you're able to retain at least 2 Fulls from each job.  It looks like you're only planning to retain one from each at the moment, which means if there's ever a problem reading part of that Full backup, you have no usable backups at all.  Of course if there's ever a problem with that 6TB disk, you have the same issue, so apart from all of this, you might want to consider a rotation or replication strategy so that you have backups on multiple destinations.  One option you might consider would be running a Full backup every 2 weeks and running Incrementals rather than Differentials all other days.  The additional Full will make your backups more robust, and Incrementals will be smaller than Differentials.

Additionally, how large is the bzvol folder you're excluding from your F&F job?  If it's not too large (and you don't need to worry about restoring custom NTFS permissions), you might want to consider switching your D drive backup to an image job (with just that one partition) rather than an F&F job.  That will further shrink your Incrementals because Reflect will then be able to back up only the changed portions of files.  With an F&F job, if the file has changed, Reflect has to back up the entire file again, even if only a tiny part of a very large file changed, which obviously means larger Incrementals.  But if your bzvol folder is large, then I suppose image backups might be larger overall.

Finally, I would suggest setting your retention policy in number of backups rather than number of days.  I've seen cases where people use time-based retention policies, then go on vacation during which time they perform no backups, and then the first time they run a backup after returning, Reflect deletes many or even all of their backups because they're all technically "expired", and that's normally not what they intend.  Specifying retention in number of backups avoids that outcome. (UPDATE: And as Froggie mentioned above, you should set your retention policy to something that's realistic based on the size of your backups and the amount of storage available on your destination.  You really don't want to rely only on the free disk space threshold purge.)

I'd like to follow up on the .bzvol directory question. I use an online ("cloud") backup service called BackBlaze. Backblaze creates a ".bzvol" directory at the top level of every drive it backs up. Inside this directory is a tiny file that identifies this hard drive for the rest of time. BackBlaze support says that if you clone a drive that is selected for backup by Backblaze, the cloned image will also contain the ".bzvol" directory. This confuses Backblaze, because now the two different hard drives have the same id, and two hard drives should never have the same bzvolumeid. BackBlaze recommends that you set your cloning software to exclude the .bzvol folder, and remove the contents of any .bzvol folder that has already been cloned.   

That's why I excluded the .bzvol ​​directory. But is Macrium a simple cloning software or are the compressed backup files unreadable by BackBlaze such that it would be OK to include them in the image?

jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)Macrium Evangelist (6.1K reputation)
Group: Forum Members
Posts: 4.2K, Visits: 30K
Great question!  The clone scenario applies because in a clone scenario, the end result is that you have two actual drives, both online simultaneously, with identical contents.  So if you were using Reflect for cloning, then you'd have to worry about that -- but you're not.  You're capturing images.  As a result, all of the data from that partition is wrapped up in a file being stored somewhere, not "online" and visible to the rest of the system.  So you can absolutely switch to image backups without having to worry about that.  The only concern might be the size of the Backblaze folder itself, since an image backup would include that.  However, that might not be an issue depending on how Backblaze works.  If that folder is simply a virtual directory that essentially maps to files stored in the cloud so that it appears that your files are on your system even though they aren't, you don't have anything to worry about since that content doesn't actually exist on your partition.  If on the other hand Backblaze stores a local cached copy of your cloud content for faster and offline access and then syncs that up to the cloud in the background, then that content would be included in an image backup of that partition.

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Reading This Topic

Login

Explore
Messages
Mentions
Search