Macrium Support Forum

Incremental Forever option & time/date stamps on files

https://forum.macrium.com/Topic3088.aspx

By versapress - 29 July 2015 10:01 PM

I have a problem with our current method of backup. We're doing daily incremental backups to a NAS with a 30 day retention period. I have the backups split into separate jobs, because the drives I'm backing up are anywhere from 2-5+ TB of data on each drive. After the backups complete, there is a separate scheduled task to run a RoboCopy script to sync the files from our first NAS to a second NAS for disaster recovery purposes. The backups are set to merge the oldest incremental with the other incrementals after the 30th backup. Originally, we had it set for a synthetic full, but I abandoned that option because after the 30 retention period, the full backup started merging with the oldest incremental and the date/time stamps were changing each day, and the sync between the NAS's was never completing within the time window each day because I was essentially re-copying 10+ TB of data each day.

I changed this to not do a synthetic full because I wanted the original full backup to remain intact, with the same size file, and same date and time stamp on it as well. The issue is now that I've hit the 30 day retention period, the oldest incremental file is now merging with the second-oldest incremental file, but the time/date stamps are changing on ALL of the incremental backup files. The original full backup remains unchanged, as I hoped and expected.

This really isn't too big an issue on two of the three data drives being backed up, because their incrementals are not too large, collectively, but my biggest drive (which is 5+ TB in the original full backup) has enough changes in the data each day, that by the 30th day of backups, all of the incrementals are now almost 3TB in size, collectively. Because all of the incrementals have new date and time stamps, RoboCopy wants to copy these each night. This means I now have to copy 2+ TB of data between the NAS's each night. This is not going to work for me.

How come when the incremental merge occurs, it changes the time and date stamp on ALL of the incrementals? Does it really make a change to any of the incrementals that are not part of the merge? Is there something else I can do to get what I want?

Essentially, I want to have my one full backup, with at least the last 30 days of incrementals. I want to run the sync script to copy these files between NAS's after the Macrium backups are completed. I would only want to have to copy the files that have really changed since the last full or incremental backup. 

Is this possible?
By Nick - 29 July 2015 10:12 PM

@versapress

Thanks for posting. Sorry you are having problems. 

When backups are consolidated subsequent incrementals backups in the same set are updated to reflect the consolidation. Please see this KB article for more information on this:

http://knowledgebase.macrium.com/display/KNOW/Backup+Folder+Synchronization

We are developing a new 'Delta Incremental' method that will stop consolidations from affecting later files in the set. This will be released in Macrium Reflect v6.1 in the next few weeks. I'm afraid I can't give a precise date for this. 
By versapress - 29 July 2015 10:16 PM

Well, that's good to know. Thank you for that information. I will deal with this until that version update comes available. Hopefully I'll be able to manage through it for a while.

Thanks again.
By versapress - 14 September 2015 5:21 PM

Any idea when this update will be out? I've been struggling with our full and incremental backups synchronizing to our secondary NAS for a month and a half now. Most times, it takes well over two days to complete. It would be great for the syncs to complete within a reasonable amount of time again.
By Nick - 14 September 2015 11:45 PM

@versapress

Thanks for getting back. 

Sorry for the delay. We plan to release Delta Incremental Indexing in the next few days.  

Thanks for your patience. 
By versapress - 1 October 2015 3:21 PM

Two weeks have passed. Is this coming soon? I am really in a jam with the way the backups run and archive now. I literally have to sync over 10TB of data every day because of the time/date stamp issue. There isn't enough time during the day to get this sync done, even on our 10Gb network. The backups never complete within a day. The syncs are always behind, and I'm beyond frustrated with this. 

Will this be fixed in the near future? Do you have a more defined time table besides "next few days"? 
By Nick - 2 October 2015 12:32 PM

Hi versapress

Apologies for the delay with this release. We hope to make this publicly available on Monday 5th October. 
By Keith Weisshar - 3 October 2015 11:07 PM

Is Monday the official release date for Macrium Reflect v6.1?
By Nick - 8 October 2015 10:12 AM

Sincere appologies for the delay with this release. The release has taken longer to get through QA than expected. Mon 12th is now the target date. 
By Arvy - 8 October 2015 6:19 PM

Never a need to apologise for good QA so far as this user is concerned.  Would much prefer waiting a few days rather than deal with the consequences of something going "out the door" before it's proven completely and reliably ready, especially in the case of backup and recovery protection.
By Drac144 - 8 October 2015 7:50 PM

Arvy,

I think the apology was for having given a release date on this forum and then not meeting that stated date. This is different than WHY the date was missed.

As a Canadian, and thus super polite, I would think you would understand that an apology is always appropriate. Smile
By Arvy - 9 October 2015 12:08 AM

I said nothing about apologies being inappropriate, only that the reasons given in this case made an apology unnecessary so far as I myself was concerned.  I'll leave it for others to judge the appropriateness, relevance and helpfulness of comments based on nationality stereotypes.
By Drac144 - 9 October 2015 11:51 PM

So you are saying you are NOT polite, friendly and helpful like the stereotypical Canadian?  Sad