Experiencing poor system performance since recent updates when imaging main disk - system nearly...


Experiencing poor system performance since recent updates when imaging...
Author
Message
Clu
Clu
New Member
New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)
Group: Forum Members
Posts: 10, Visits: 24
Heya everyone,

i'm running the most recent Macrium Reflect 8 version (v8.1.7638 - 4th September 2023) and experience poor general system performance when the imaging of the os drive C: runs. Drive D: in the same imaging job is fine. File and Folder Backups from C: are also fine and without problems.
The os drive is an nvme with good read / write speeds (Model CSSD-F960GBMP510). The image gets written to an WD HDD, connected internally over SATA3. While the SSD is being utilized at 100%, with read speeds fluctuating between 20 to 1500 MB/s, the destination HDD gets barely utilized. All drives are shown as fine in CrystalDiskInfo and test fine with CrystalDiskMark (having no general system slowdowns).
The priority setting doesnt seem to do anything. I currently have it on the 2. setting from the left. The Changed Block Tracker is enabled for the drives getting imaged.
The image job runs daily for incremental, weekly for differential and monthly for full. Both incremental and differential images slow down the system. The final file sizes for incremental were 1GB at most in the last days and 9GB for the differential image.

Surprisingly, running a full backup doesnt show this behaviour.
The Windows Resource Manager shows Reflect accessing a ShadowCopy when this happens.

Relevant PC Specs
AMD 3600
64 GB of RAM (F4-3000C16D-32GISB x2)
MSI X370 Xpower Motherboard
D: drive (not problematic, gets imaged in the same image job, ADATA Ultimate SU800 2TB)
C: drive (Corsair Force Series MP510 960GB problematic in imaging)
F: drive (destination, HDD, Western Digital WD Purple 4TB)

For further info / overview find several screenshots attached.

Some help for fixing this behaviour is greatly appreciated.

Attachments
Drac144
Drac144
Master
Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)Master (1.7K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 4K
Note that incremental and differential backups WILL have high read usage but low write usage - just as you describe.  Only the changes are written.  However if CBT is being used, the reads should only be for the changed blocks.  You say CBT is enabled but you did not say that the log indicates it is actually being used.  Can you post your log file (with your ID removed) or verify that the log indicates that CBT is being used for that backup?
Clu
Clu
New Member
New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)
Group: Forum Members
Posts: 10, Visits: 24
I can confirm that CBT is checked for both drives and the log mentions that CBT was successfully initialized. D: gets checked first for this image because of the disk order in Reflect. CBT Initialization is only mentioned for D: tho in this log. It's not again mentioned for C: in the same image run.
I do have another image job running for C:, where the image gets written to another device. D: is excluded from that image. The log for that job indicates that CBT was successfully initialized for C:.


Attachments
cbt initialised with drive d .png (2 views, 54.00 KB)
cbt is active.png (2 views, 32.00 KB)
CE01737DF1AD2632-10-10.txt (2 views, 3.00 KB)
Dan Danz
Dan Danz
Macrium Hero
Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 9.5K
VSS uses so-called Shadow Storage to create its lists of disk addresses, and it's this shadow that drives the backup.   It is possible to have shadow storage on one drive that is used for all drives, because when allocating the storage, the options are FOR: drive-1 and ON: drive2.   So, if it's unbalanced (i.e., FOR: and ON: are different, then you would have an imbalance of I/O requests.    It's easy enough to find out:  from an elevated command prompt:  vssadmin list ShadowStorage


L.W. (Dan) Danz, Overland Park KS
Reflect v8.1.7847+ on Windows 11 Home 23H2 22631.3085+ | Reflect v8.1.7853+ on Windows 10 Pro 22H2 19045.3996+
Reflect v8.1.7784+ on 2 systems Windows 10 Home 22H2 19045.3803+

Clu
Clu
New Member
New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)
Group: Forum Members
Posts: 10, Visits: 24
i cant quite follow your reply, but here is what the command gave me:
vssadmin 1.1 - Volume Shadow Copy Service administrative command line utility
(C) Copyright 2001-2013 Microsoft Corp.

Shadow copy storage association
  For Volume: (C: )\\?\Volume{cb455346-3735-4c0d-9c4c-456349bcd32a}\
  Shadow Copy Storage Volume: (C: )\\?\Volume{cb455346-3735-4c0d-9c4c-456349bcd32a}\
  Shadow Copy Space Used: 7.71 GB (0%)
  Shadow copy space allocated: 8.08 GB (0%)
  Max Shadow Copy Storage Area: 10.0 GB (1%)
translated with google translate
edit: added spaces, so : ) doesnt create smileys

Edited 27 September 2023 8:53 PM by Clu
Dan Danz
Dan Danz
Macrium Hero
Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 9.5K
Clu - 27 September 2023 8:52 PM
i cant quite follow your reply, but here is what the command gave me:
vssadmin 1.1 - Volume Shadow Copy Service administrative command line utility
(C) Copyright 2001-2013 Microsoft Corp.

Shadow copy storage association
  For Volume: (C: )\\?\Volume{cb455346-3735-4c0d-9c4c-456349bcd32a}\
  Shadow Copy Storage Volume: (C: )\\?\Volume{cb455346-3735-4c0d-9c4c-456349bcd32a}\
  Shadow Copy Space Used: 7.71 GB (0%)
  Shadow copy space allocated: 8.08 GB (0%)
  Max Shadow Copy Storage Area: 10.0 GB (1%)
translated with google translate
edit: added spaces, so : ) doesnt create smileys

Strange -- Earlier you included a screen shot cbt initialised with drive d .png  that shows image backup of both D:\ and C:\  yet  the above does not show that the D:\ volume has any shadow storage allocated on it, so I think that you are using the large (10GB) area on C: to hold snapshots that are taken when you are creating an image of both drives, so that means Reflect will be creating then reading the snapshot of D: from C: while at the same it is reading data blocks from D:.

You later provided a part of a log from a backup of both volumes c:\ and d:\, and it shows shadow copies for both drives being temporarily stored on the same hard disk volume, futher evidence that reflect has to involve both volumes when imaging D:\, while it also has to read both the snapshot data AND the data being backed up when imaging the C:\ volume as well as writing the image file on the F:\ volume.

If you wanted to eliminate that imbalance, you could use vssadmin ResizeShadowStorage option, as described in this Microsoft article by adding a shadow storage area on D:\ and perhaps decreasing the size of the 10GB area on C:\.   Or you could decide that you'd rather have all the shadow storage on C:. But you'd have to include that I/O load when trying to decide where the bottleneck is?  My thoughts are that the slowup is the disk write load on F: ?  What model of drive is that and how is it attached to your system?  If it is an SSD, does it use CMR or SMR recording techniques?  
Edited delete the request for information already provided.

For the record,  Reflect's shadow storage usage is temporary and shadow copies are removed at the end of Reflect's use of them (but System Restore Points kept  in the same area are retained until deleted). But if you use just one shadow storage area to hold all snapshots, it will have to be large enough to hold BOTH snapshots while Reflect is doing the backup.  

From that same screen shot, when it was taken CBT was NOT being used on the volume c:\ (the changes reflected in the MFT for C: are being used to drive the image creation for volume c:\).

The screenshots of text in ??German?? are hard for me to read -- google translate doesn't work on pictures of text.

As for having to jump thru hoops to stop the forum from interpreting some character sequences as emoticons, you can turn off the display of emoticons in the forum by clicking on your picture in the header, then Edit Account, and select Topics -- You'll find the option like this:

Again, it's your choice.

L.W. (Dan) Danz, Overland Park KS
Reflect v8.1.7847+ on Windows 11 Home 23H2 22631.3085+ | Reflect v8.1.7853+ on Windows 10 Pro 22H2 19045.3996+
Reflect v8.1.7784+ on 2 systems Windows 10 Home 22H2 19045.3803+

Edited 28 September 2023 6:17 PM by Dan Danz
Clu
Clu
New Member
New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)
Group: Forum Members
Posts: 10, Visits: 24
Dan Danz - 28 September 2023 6:11 PM
Clu - 27 September 2023 8:52 PM
i cant quite follow your reply, but here is what the command gave me:
vssadmin 1.1 - Volume Shadow Copy Service administrative command line utility
(C) Copyright 2001-2013 Microsoft Corp.

Shadow copy storage association
  For Volume: (C: )\\?\Volume{cb455346-3735-4c0d-9c4c-456349bcd32a}\
  Shadow Copy Storage Volume: (C: )\\?\Volume{cb455346-3735-4c0d-9c4c-456349bcd32a}\
  Shadow Copy Space Used: 7.71 GB (0%)
  Shadow copy space allocated: 8.08 GB (0%)
  Max Shadow Copy Storage Area: 10.0 GB (1%)
translated with google translate
edit: added spaces, so : ) doesnt create smileys

Strange -- Earlier you included a screen shot cbt initialised with drive d .png  that shows image backup of both D:\ and C:\  yet  the above does not show that the D:\ volume has any shadow storage allocated on it, so I think that you are using the large (10GB) area on C: to hold snapshots that are taken when you are creating an image of both drives, so that means Reflect will be creating then reading the snapshot of D: from C: while at the same it is reading data blocks from D:.

You later provided a part of a log from a backup of both volumes c:\ and d:\, and it shows shadow copies for both drives being temporarily stored on the same hard disk volume, futher evidence that reflect has to involve both volumes when imaging D:\, while it also has to read both the snapshot data AND the data being backed up when imaging the C:\ volume as well as writing the image file on the F:\ volume.

If you wanted to eliminate that imbalance, you could use vssadmin ResizeShadowStorage option, as described in this Microsoft article by adding a shadow storage area on D:\ and perhaps decreasing the size of the 10GB area on C:\.   Or you could decide that you'd rather have all the shadow storage on C:. But you'd have to include that I/O load when trying to decide where the bottleneck is?  My thoughts are that the slowup is the disk write load on F: ?  What model of drive is that and how is it attached to your system?  If it is an SSD, does it use CMR or SMR recording techniques?  
Edited delete the request for information already provided.

For the record,  Reflect's shadow storage usage is temporary and shadow copies are removed at the end of Reflect's use of them (but System Restore Points kept  in the same area are retained until deleted). But if you use just one shadow storage area to hold all snapshots, it will have to be large enough to hold BOTH snapshots while Reflect is doing the backup.  

From that same screen shot, when it was taken CBT was NOT being used on the volume c:\ (the changes reflected in the MFT for C: are being used to drive the image creation for volume c:\).

The screenshots of text in ??German?? are hard for me to read -- google translate doesn't work on pictures of text.

As for having to jump thru hoops to stop the forum from interpreting some character sequences as emoticons, you can turn off the display of emoticons in the forum by clicking on your picture in the header, then Edit Account, and select Topics -- You'll find the option like this:

Again, it's your choice.

Thank your for your help so far, appreciate it.
I tried the command you linked, but it's not functional. I've read some more and apparently i can't add a shadowcopy"area" to d:. this is reserved for a server os. the resize command doesnt work either (Error: The specified volume shadow copy storage association was not found.)

What's strange to me is, that these performance problems occured recently (3 months~). if there's only a shadowcopy area on c:, i'd expect it to have shown system slowdowns before. The system is configured like this for atleast 2 years and i cant recall having issues before. the only thing that got added recently was another ssd for 2 other partitions, which were formerly on the same drive as partition d:. in the pic cbt is active.png you see a big chunk of unallocated storage besides d:, thats where these now moved partitions were originally.
the moved partitions aren't imaged at all and the new drive is only used for these 2 partitions and nothing else (new drive is samsung 870 evo 4tb, MZ-77E4T0B).

Dan Danz
Dan Danz
Macrium Hero
Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 9.5K
Funny, I don't have any trouble with shadow storage on non system drives, and none of my systems are servers...all Win 10 Home.  Please show the results of your interaction with vssadmin where it errors.

I wonder if you might have disabled System Protection on D:\.  Look in Settings/System/About/ related links System Protection.   Have you denied it on that drive? if so, you don't need to resize it.  Enable it on that drive, then allocate the percentage of the disk from the System Protection settings controls. It won't automatically take free space on the drive like you thought it would.  You have to tell it how much to use. Then vssadmin list shadowstorage should show it.


L.W. (Dan) Danz, Overland Park KS
Reflect v8.1.7847+ on Windows 11 Home 23H2 22631.3085+ | Reflect v8.1.7853+ on Windows 10 Pro 22H2 19045.3996+
Reflect v8.1.7784+ on 2 systems Windows 10 Home 22H2 19045.3803+

Clu
Clu
New Member
New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)New Member (22 reputation)
Group: Forum Members
Posts: 10, Visits: 24
Ok, i added the system protection for drive d:, now it shows up when i list the shadowstorage areas. It didnt solve the IO impact tho. C is still getting utilized 100% when the image is being created. Just to clarify, because i dont know if i was clear enough: the Slowdown happens when Reflect is saving the changes from drive c to the image file. Not when D is getting imaged in the same run.

I removed drive d now from that image job.

Thx for the help tho, appreciate it.

Seekforever
Seekforever
Master
Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)Master (1.6K reputation)
Group: Forum Members
Posts: 1.1K, Visits: 24K
You've tried a lot and you have your Crystalmark data, but if you want to try one more thing, and haven't done so,  run CHKDSK /R on the C drive partition. 

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search