Could Macrium be a possible cause of my System Restore points disappearing?


Author
Message
terrypin
terrypin
New Member
New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)
Group: Awaiting Activation
Posts: 35, Visits: 124
For several months my System Restore points have been repeatedly vanishing and despite much research I have still not discovered the cause. It appears to be a fairly wide problem. This is one of the several threads in which it's being discussed
https://www.tenforums.com/backup-restore/146930-restore-points-being-deleted-4.html
and in post #14 of that someone suggested that Acronis could be a cause, referring to
https://forum.acronis.com/forum/acronis-backup-117/windows-10-system-restore-show-more-restore-points-box-missing-and-restore-points-being-deleted-or-missing#comment-form

In post #54 of the Win 10 forum thread at least one of the sufferers then confirmed that.

So, although it's admittedly a leap (sign of my desperation!) I'm wondering if the experts have any view on whether Macrium could have a similar issue please?


Edited 22 January 2020 4:46 PM by terrypin
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)
Group: Forum Members
Posts: 10K, Visits: 66K
Depending on the circumstances under which you're seeing this, then Reflect could be an indirect cause.

First, it's important to understand that Reflect and many other image backup solutions (and various other types of tools) rely on a component in Windows called VSS (Volume Snapshot Service). Omitting a bunch of details, VSS can generate a point-in-time snapshot of a live partition, which is what Reflect uses as the source for its backup and what makes it possible to create a valid backup of a partition or set of files that are actively being used.  System Restore points are actually "permanent" VSS snapshots that are retained until they consume the amount of space you've allowed for snapshots (I think the default is 15% of the partition size), after which the oldest snapshots start getting deleted.  The snapshots that Reflect requests are temporary VSS snapshots, i.e. they get deleted after Reflect finishes its backup.  However, the snapshot consumes some space while it exists, and the more write activity that occurs on the partition during a backup, the larger the snapshot has to be in order to preserve the original state of the data at the time the backup began -- yes, the snapshot actually grows during its lifetime rather than being a fixed size at its creation.  This is called "copy-on-write" behavior.  So if you started a backup job and then wrote a ton of data to the partition while the job was running, the size of that temporary snapshot could grow to the point that some or even all of your permanent snapshots that are used for System Restore points have to be deleted in order to keep the total size of your snapshots within the configured quota on the partition.  And if you kept on writing data so that the size of the temporary snapshot on its own exceeded your entire snapshot quota, then Windows would delete the temporary snapshot itself and the Reflect job would fail with a note that the snapshot was deleted before the backup completed.  This is one reason it's a good idea to perform backups during relatively "quiet" periods.

In addition, if you ever restore a Reflect image backup and notice that the System Restore snapshots that would have been there when that backup was created aren't there now that you've restored that backup, that's because part of the VSS snapshot mechanism I described earlier involves notifying applications that a snapshot has been requested (e.g. by Reflect), and applications that have registered to be notified of these snapshot requests are then given an opportunity to use a component called a VSS writer that is allowed to make changes to the contents of the snapshot before Windows tells the requesting application that the snapshot is ready for use. Database applications use this capability to make sure that the data actually written to disk is internally consistent, for example.  VSS writers are also allowed to delete data specifically from a snapshot (while preserving it on disk), and Windows itself has some VSS writers that do this.  System Restore points are not included in VSS snapshots.  The Windows Search service also deletes the search index database because that can grow quite large and is automatically recreated when it's missing.  One way to avoid this outcome is to capture backups from Rescue Media, which does not use VSS because in the Rescue Media environment, the partitions aren't online/active, so VSS isn't necessary -- and consequently, the changes introduced by VSS writers don't occur.

Hopefully one of those accounts for what you're seeing.




Edited 22 January 2020 5:14 PM by jphughan
terrypin
terrypin
New Member
New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)
Group: Awaiting Activation
Posts: 35, Visits: 124
Much appreciate that fast and detailed response, thank you, although I confess it's largely above my know how level!

I had checked that these two VSS related services were running
Volume Shadow Copy
Microsoft Software Shadow Copy Provider
Both are set to Automatic.
Reflect is set to do a full backup every Sunday morning. No differential, no incremental., although I may pursue those options at some time.
The last few steps I've taken were:
1. Set up a new entry in Task Scheduler so that an SR would be created at
each startup. using instructions here:
https://www.windowscentral.com/how-create-system-restore-points-automatically-startup-windows-10

2. RPGlobalInterval created in regedit and set to 2 days = 172800 secs
https://techjourney.net/change-the-frequency-or-backup-interval-of-system-restore-in-windows/

3. Enabled Automatic System Restore Points Using Group Policy
https://www.itprotoday.com/windows-10/how-set-automatic-restore-points-windows-10

Will reboot shortly and then thought I'd make a couple of new SR points, wait a day or two, check if they still exist, force a new run of my full backup, and check again.

Is there any other test I can do to rule out Reflect please?




jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)
Group: Forum Members
Posts: 10K, Visits: 66K
The real test would be to check which System Restore snapshots you have immediately prior to a Reflect job starting, and then seeing what snapshots you have afterward.  If you lost some snapshots during the backup, it's likely because the temporary snapshot that Reflect needed in order to create the backup required space that forced some older snapshots to be deleted in order to keep within your partition's snapshot quota.  The way to minimize the likelihood of this (or the quantity of deletions required) would be to minimize the amount of write activity that occurs on your system while a backup is running, and/or to increase the VSS snapshot quota for the partition(s) in question.  The latter won't change the behavior directly, but it will allow you to preserve more snapshots overall -- so if your concern for example is that you want to be able to keep snapshots going back a particular period of time and right now they're getting deleted sooner than that, then increasing your VSS snapshot quota will mean you'll be able to hang onto more of them.

terrypin
terrypin
New Member
New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)
Group: Awaiting Activation
Posts: 35, Visits: 124
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)
Group: Forum Members
Posts: 10K, Visits: 66K
Well if you can narrow down the timeframe when the SR points get deleted by checking them frequently, you may be able to find useful information in Event Viewer that might explain the reason for the deletion. And with respect to Sygnus21’s reply over there, just for accuracy’s sake, it wouldn’t be Reflect deleting SR points. It would still be Windows deleting them in order to remain within the VSS snapshot storage quota. The REASON they would be deleted in the scenario I described would be to preserve the new temporary snapshot that Reflect is using to perform a backup, but it’s still Windows doing the work.
terrypin
terrypin
New Member
New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)
Group: Awaiting Activation
Posts: 35, Visits: 124
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)Macrium Evangelist (16K reputation)
Group: Forum Members
Posts: 10K, Visits: 66K
That’s not a conflict. It looks like what I explained above is what’s happening, which is what’s DESIGNED to happen. If the system doesn’t have enough space in the snapshot quota to maintain a new snapshot that grows as needed based on disk write activity, then Windows starts deleting the oldest snapshots as needed to recover space. The only alternative behaviors would be Windows exceeding your storage quota, which defeats the point of a quota, or else deleting the most recent snapshot that Reflect is using in order to preserve the old snapshot AND remain below the quota — but that would cause backups to fail.

If you want to be able to maintain more snapshots, then increase the VSS snapshot storage quota on your C volume. And/or just consider that when you have Reflect backups that you’re making on a regular basis, the need for System Restore is dubious are best. Even among people who DON’T make regular image backups, System Restore is sort of a “nice to have....when it actually works” sort of thing.
Edited 26 January 2020 3:18 PM by jphughan
terrypin
terrypin
New Member
New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)New Member (43 reputation)
Group: Awaiting Activation
Posts: 35, Visits: 124
jphughan - 26 January 2020 3:15 PM
That’s not a conflict. It looks like what I explained above is what’s happening, which is what’s DESIGNED to happen. If the system doesn’t have enough space in the snapshot quota to maintain a new snapshot that grows as needed based on disk write activity, then Windows starts deleting the oldest snapshots as needed to recover space. The only alternative behaviors would be Windows exceeding your storage quota, which defeats the point of a quota, or else deleting the most recent snapshot that Reflect is using in order to preserve the old snapshot AND remain below the quota — but that would cause backups to fail.

If you want to be able to maintain more snapshots, then increase the VSS snapshot storage quota on your C volume. And/or just consider that when you have Reflect backups that you’re making on a regular basis, the need for System Restore is dubious are best. Even among people who DON’T make regular image backups, System Restore is sort of a “nice to have....when it actually works” sort of thing.

posted 27 January 2020 10:29 AM
terrypin
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Awaiting Activation
Posts: 15, Visits: 50
Last active: 27 January 2020 10:27 AM
    
On advice elsewhere I ran this from a command prompt:

vssadmin list shadows


It seems to tell me there are four 'shadow copies', including the 21/1/20 that's no longer shown in the SR dialog shown in my screenshot. The latest entry coincides with this morning's MR image run. In total it seems a modest amount of storage, well below the 12 GB reserved, which I would not expect to give rise to the deletions you describe? That indication of small volumes seesm confirmed by the TreeSize display I've included too.


Microsoft Windows [Version 10.0.18362.592]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\WINDOWS\system32>vssadmin list shadows
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2013 Microsoft Corp.

Contents of shadow copy set ID: {0164a638-0dc6-484e-a796-08aca6111170}
Contained 1 shadow copies at creation time: 21/01/20 06:17:24
Shadow Copy ID: {270cffa2-c841-404c-b984-1ae55f377b19}
 Original Volume:
(CSmile\\?\Volume{0aa994fb-aeb9-4b19-863d-c873ece9fcb0}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: ClientAccessibleWriters
 Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

Contents of shadow copy set ID: {48c6716c-32d4-4334-928b-b372055fffdf}
Contained 1 shadow copies at creation time: 23/01/20 09:45:24
Shadow Copy ID: {39e3e10d-b889-4918-911b-cd2c8159480e}
 Original Volume:
(CSmile\\?\Volume{0aa994fb-aeb9-4b19-863d-c873ece9fcb0}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: ClientAccessibleWriters
 Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

Contents of shadow copy set ID: {1d82cfe8-524e-4ac3-829b-e275c0325c10}
Contained 1 shadow copies at creation time: 23/01/20 16:42:19
Shadow Copy ID: {f452886c-179a-4b52-b3f7-e172ec77eccf}
 Original Volume:
(CSmile\\?\Volume{0aa994fb-aeb9-4b19-863d-c873ece9fcb0}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy6
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: ClientAccessibleWriters
 Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

Contents of shadow copy set ID: {b62323b3-95bf-479e-874d-0bb2b17a745d}
Contained 1 shadow copies at creation time: 04/11/18 05:38:05
Shadow Copy ID: {1ee712ad-ede6-4741-99fa-c75f24c0feee}
 Original Volume:
(GSmile\\?\Volume{00028375-0000-0000-0000-100000000000}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: DataVolumeRollback
 Attributes: Persistent, No auto release, No writers,
Differential
 
--------------------

TreeSize implies there's only 1.5 GB consumed by SR.
https://www.dropbox.com/s/gus81khya0k5xyz/SRs-TreeSize26Jan2020-2.jpg?raw=1

BTW, that's strangely only half of that shown in the SR window. Also, it's apparently made up entirely of MS Apps, hardly any of which I use!

P.S. I see that parts of the addresses displayed by that VSS command get poorly interpreted here, but I hope they're still intelligible.


Nick
Nick
Macrium Representative
Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)Macrium Representative (4.6K reputation)
Group: Administrators
Posts: 2.6K, Visits: 16K
terrypin - 27 January 2020 10:36 AM
jphughan - 26 January 2020 3:15 PM
That’s not a conflict. It looks like what I explained above is what’s happening, which is what’s DESIGNED to happen. If the system doesn’t have enough space in the snapshot quota to maintain a new snapshot that grows as needed based on disk write activity, then Windows starts deleting the oldest snapshots as needed to recover space. The only alternative behaviors would be Windows exceeding your storage quota, which defeats the point of a quota, or else deleting the most recent snapshot that Reflect is using in order to preserve the old snapshot AND remain below the quota — but that would cause backups to fail.

If you want to be able to maintain more snapshots, then increase the VSS snapshot storage quota on your C volume. And/or just consider that when you have Reflect backups that you’re making on a regular basis, the need for System Restore is dubious are best. Even among people who DON’T make regular image backups, System Restore is sort of a “nice to have....when it actually works” sort of thing.

posted 27 January 2020 10:29 AM
terrypin
New Member
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Awaiting Activation
Posts: 15, Visits: 50
Last active: 27 January 2020 10:27 AM
    
On advice elsewhere I ran this from a command prompt:

vssadmin list shadows


It seems to tell me there are four 'shadow copies', including the 21/1/20 that's no longer shown in the SR dialog shown in my screenshot. The latest entry coincides with this morning's MR image run. In total it seems a modest amount of storage, well below the 12 GB reserved, which I would not expect to give rise to the deletions you describe? That indication of small volumes seesm confirmed by the TreeSize display I've included too.


Microsoft Windows [Version 10.0.18362.592]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\WINDOWS\system32>vssadmin list shadows
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line
tool
(C) Copyright 2001-2013 Microsoft Corp.

Contents of shadow copy set ID: {0164a638-0dc6-484e-a796-08aca6111170}
Contained 1 shadow copies at creation time: 21/01/20 06:17:24
Shadow Copy ID: {270cffa2-c841-404c-b984-1ae55f377b19}
 Original Volume:
(CSmile\\?\Volume{0aa994fb-aeb9-4b19-863d-c873ece9fcb0}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: ClientAccessibleWriters
 Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

Contents of shadow copy set ID: {48c6716c-32d4-4334-928b-b372055fffdf}
Contained 1 shadow copies at creation time: 23/01/20 09:45:24
Shadow Copy ID: {39e3e10d-b889-4918-911b-cd2c8159480e}
 Original Volume:
(CSmile\\?\Volume{0aa994fb-aeb9-4b19-863d-c873ece9fcb0}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: ClientAccessibleWriters
 Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

Contents of shadow copy set ID: {1d82cfe8-524e-4ac3-829b-e275c0325c10}
Contained 1 shadow copies at creation time: 23/01/20 16:42:19
Shadow Copy ID: {f452886c-179a-4b52-b3f7-e172ec77eccf}
 Original Volume:
(CSmile\\?\Volume{0aa994fb-aeb9-4b19-863d-c873ece9fcb0}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy6
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: ClientAccessibleWriters
 Attributes: Persistent, Client-accessible, No auto release,
Differential, Auto recovered

Contents of shadow copy set ID: {b62323b3-95bf-479e-874d-0bb2b17a745d}
Contained 1 shadow copies at creation time: 04/11/18 05:38:05
Shadow Copy ID: {1ee712ad-ede6-4741-99fa-c75f24c0feee}
 Original Volume:
(GSmile\\?\Volume{00028375-0000-0000-0000-100000000000}\
 Shadow Copy Volume:
\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4
 Originating Machine: Terry-2016
 Service Machine: Terry-2016
 Provider: 'Microsoft Software Shadow Copy provider 1.0'
 Type: DataVolumeRollback
 Attributes: Persistent, No auto release, No writers,
Differential
 
--------------------

TreeSize implies there's only 1.5 GB consumed by SR.
https://www.dropbox.com/s/gus81khya0k5xyz/SRs-TreeSize26Jan2020-2.jpg?raw=1

BTW, that's strangely only half of that shown in the SR window. Also, it's apparently made up entirely of MS Apps, hardly any of which I use!

P.S. I see that parts of the addresses displayed by that VSS command get poorly interpreted here, but I hope they're still intelligible.


Thanks for posting.

Macrium Reflect doesn't create 'Persistent' shadow copies, It creates non persistent 'Differential' shadows using VSS Writers that only persist during the backup.  As described by @jphughan, maintenance of the shadow copy storage area is handled entirely by the Windows VSS subsystem.  If the shadow copy diff area grows significantly during a backup then this could cause existing shadows to be purged while the backup is running. This is normal and expected. If you are seeing unexpected purges of older shadows then you could try increasing your shadow storage to handle the temporary diff area created by non persistent shadows or create images during periods of low disk activity. 

You will only see Reflect shadow copies using 'vssadmin list shadows' while a backup is running. The Shadow type is 'Backup', and the attributes shows 'Differential'. Here's an example:

Contents of shadow copy set ID: {b7e70817-a9b0-4bf4-a609-231de97ed27f}
 Contained 1 shadow copies at creation time: 27/01/2020 11:47:54
  Shadow Copy ID: {431e03d5-aa90-4bd4-b36b-b31d7e3207ef}
   Original Volume: (C: )\\?\Volume{7d01874d-3650-45bb-8fce-b96b76cce09a}\
   Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy7
   Originating Machine: MS-W-0.macrium.local
   Service Machine: MS-W-0.macrium.local
   Provider: 'Microsoft Software Shadow Copy provider 1.0'
   Type: Backup
   Attributes: Differential, Auto recovered


Kind Regards

Nick - Macrium Support



Edited 27 January 2020 2:03 PM by Nick
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search