CBT Kernel Memory Resource Utilization


Author
Message
Otter
Otter
New Member
New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)
Group: Forum Members
Posts: 31, Visits: 48
Has anyone had issues with the CBT driver consuming too much Non-Paged Pool (kernel memory not backed by paged memory - basically this is physical RAM).  I'm surprised by how much (currently 58MB) of this relatively scarce (well, okay, that depends a LOT on how much RAM you have) resource is currently being used by MRCBT.SYS (Macrium's Change Block Tracking device driver - an upper-volume filter driver, also upper-filter on Snapshot devices).

On  my system, without any jobs with incrementals, this driver is still the third-highest consumer of nonpaged pool, and that's without having done much FS I/O during my current boot session (although that apparently is irrelevant as this driver appears to persist its bitmap across boot sessions as I see it storing data under System Volume Information in a DenyDefrag flagged file, which makes total sense).  I dumped strings from the driver's binary just to verify that the tags I'm seeing match those used by MRCBT.SYS. 

Here's my current kernel memory allocation from the driver MRCBT.SYS, from poolmon.exe:

Tag Type   Allocs Frees Diff Bytes    Per Alloc
 Cbt Nonp  1      0     1    160      160  
Cbt0 Paged 19     16    3    3264     1088  
Cbt0 Nonp  2091   2059  32   58384992 1824531  


That first tag is an oddball, by the way.  I don't think it's padded with a space, as in DWORD dwTag = 'tbC ' (note the space after the 'C') because poolmon.exe can't find it using a command with a space added before the 'C' (poolmon.exe "-i Cbt") which makes me think it was actually declared using something like DWORD dwTag = 'tbC' (no space after the C) which makes it a bit awkward to use with DDK/WDK tooling.  Better to have four characters or at least an actual space character like 'tbC '.  I'm nit-picking here, sorry.

However - a more memory efficient data structure for the CBT bitmap would be nice.  I'm currently considering just uninstalling the CBT driver and using differentials instead because of this. Just be careful if you switch to some tree-like data structure, and if you use recursion, that you can guarantee the recursive depth of your calls don't go beyond some max threshold that can blow the kernel stack (last time I checked the kernel stack was only 12KB and so recursive routines can easily blow it causing a BSOD).  This means if you use a tree structure, be sure it's balanced and provably limited in depth, and then for those recursive routines you use, minimize the local variables declared in those recursive functions so that you don't use much stack memory.  BigGrin
Froggie
Froggie
Macrium Hero
Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)
Group: Forum Members
Posts: 1.6K, Visits: 17K
You can uninstall CBT and still have full REFLECT capability, including Incrementals... it just does its calculations differently (using MetaData differences rather than maintain a bitmap).  For most users, they wouldn't know the difference.

Otter
Otter
New Member
New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)
Group: Forum Members
Posts: 31, Visits: 48
@Froggie

That's great, right?  I'm really enjoying this product the more I play around with it.  And with super fast disk subsystems like NVMe in RAID, etc (which I'm using) it's ridiculous how fast you can do even these full images or images using comparison rather than CBT.  So yes the drawback of the old days of doing imaging without CBT have lost a lot of their pain.  Now of course CBT is still essential for the truly large volume and high I/O scenario.  I would not want to do comparison-based differentials or incrementals on such volumes.

I haven't used Windows in several years as all of my dev efforts have been under macOS and Linux, but I just build a new Win 11 dev machine and initially I was thinking of using something like First Defense ISR to have quick roll-back ability (similar to a VM with its snapshot hierarchy) but the thing is that imaging with NVMe is so crazy fast these days that you can just do a full system restore in like 20 seconds (on my box) without having to fuss around with Int 13h and extended Int 13h hooks in your boot sector code from something like FD ISR, which likely wouldn't support RST and VROC RAID arrays anyway.

Edited 28 November 2023 9:03 PM by Otter
Otter
Otter
New Member
New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)
Group: Forum Members
Posts: 31, Visits: 48
Froggie - 28 November 2023 8:46 PM
You can uninstall CBT and still have full REFLECT capability, including Incrementals... it just does its calculations differently (using MetaData differences rather than maintain a bitmap).  For most users, they wouldn't know the difference.

Oh... wait.  Does it really use FS metadata (such as NTFS journal info) to do the comparison?  I would much prefer an actual sector-by-sector comparison of the in-use sectors.  Acronis did the whole metadata comparison techique too, at least I believe it did, back in the day, and that technique was fraught with issue from my past recollection (I could be wrong).  I just don't have warm fuzzy feelings about trusting that technique.  Are you sure that's how Macrium does the comparison when CBT info is lacking?  If so, is there any way you know of to force Macrium to do a sector comparison?
Otter
Otter
New Member
New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)
Group: Forum Members
Posts: 31, Visits: 48
@Froggie


Are there any other aquatic species in here?  lol
Froggie
Froggie
Macrium Hero
Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)
Group: Forum Members
Posts: 1.6K, Visits: 17K
Nope, just the two of us w00t

Edited 28 November 2023 9:25 PM by Froggie
Froggie
Froggie
Macrium Hero
Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)
Group: Forum Members
Posts: 1.6K, Visits: 17K
Otter - 28 November 2023 8:55 PM
Froggie - 28 November 2023 8:46 PM
You can uninstall CBT and still have full REFLECT capability, including Incrementals... it just does its calculations differently (using MetaData differences rather than maintain a bitmap).  For most users, they wouldn't know the difference.

Oh... wait.  Does it really use FS metadata (such as NTFS journal info) to do the comparison?  I would much prefer an actual sector-by-sector comparison of the in-use sectors.  Acronis did the whole metadata comparison techique too, at least I believe it did, back in the day, and that technique was fraught with issue from my past recollection (I could be wrong).  I just don't have warm fuzzy feelings about trusting that technique.  Are you sure that's how Macrium does the comparison when CBT info is lacking?  If so, is there any way you know of to force Macrium to do a sector comparison?

The technique works just fine.  Identifying only changed clusters/blocks to examine is way more efficient that comparing entire DATA blocks (sectors).  Knowing those same changed clusters is what allows RDR (Rapid DELTA Restore) to work so well.  The technique has been in use since 2015 and has proven very successful (also used by at least 2 or 3 other current imaging product vendors).
Edited 28 November 2023 9:33 PM by Froggie
Froggie
Froggie
Macrium Hero
Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)Macrium Hero (2.8K reputation)
Group: Forum Members
Posts: 1.6K, Visits: 17K
The speed of RDR, especially with NvME (or even SATA SSDs) was fast enough for me to dump Snapshot software such as RollbackRX and EazFix which provided their services completely under the Windows radar.  At the time that was very dangerous, and having many years of experience using snapshot software, I wound up helping a considerable # of users, assisting them in trying to get back months/years worth of DATA when the hiccups occurred.

I'm much happier using a product that's based on the use of Windows APIs and provides most of the speed of the older snapshot programs.  I am a happy frog Cool

Otter
Otter
New Member
New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)New Member (46 reputation)
Group: Forum Members
Posts: 31, Visits: 48
Ah, so that's what the RDR is.  When I saw that in the WinPE session I was really surprised by it and pleasantly so - very cool feature - but my thought at the time was that the same CBT driver was running in the WinPE session and had picked up and was tracking changes since some prior point-in-time and had determined it could use the CBT map in order to hasten the recovery (knowing the PIT of the selected image).  That would require a lot of CBT juggling especially retention of multiple PITs, but wow, pretty impressive.  Now I realize I completely misunderstood how this RDR feature was implemented and it makes more sense - you're saying it doesn't use CBT info at all, rather it analyzes the metadata and uses that to then determine which sectors have changed and then backs those up, thereby avoiding a sector-by-sector comparison of all in-use sectors.

Well.... if this has been used successfully without anyone reporting FS corruption for years, then than indeed makes me feel a bit better.  It was many years ago that I worked with this kind of stuff and back then techniques like RDR were not well reputed.

Also I'm very glad you're a happy frog LOL!
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)Macrium Evangelist (22K reputation)
Group: Forum Members
Posts: 14K, Visits: 85K
I wrote this post recently about CBT use cases if it’s helpful:

CBT adds the most value in two main use cases:
  • Very frequent Diff/Inc backups, such as every 15 minutes. If you have frequent small backups, then conventional change detection will make up a greater percentage of your overall backup time, and CBT can eliminate that.
  • Volumes that contain very large files, such as VHDX files or databases. In this case, conventional change detection can identify which files have changed since the last backup relatively rapidly. That occurs during the "Looking for changes" phase. However, as the backup runs, Reflect will still have to scan those changed files in their entirety to determine which specific blocks within them have changed and therefore need to be backed up. So if you have a 50 GB VHDX file that only contains minor changes since the last backup, you'll still incur the time overhead of reading the whole 50 GB. CBT eliminates this.
If your scenario doesn't fit either of the above use cases, CBT's benefit will likely be minimal.

I don’t know of a way to force a sector-level comparison except maybe by putting Reflect into forensic imaging mode. But if the file system metadata isn’t a reliable indicator of where changes have been made within the volume, then you will likely have much larger problems anyhow.

But the non-CBT change detection mechanism isn’t arduous by any means, and it wouldn’t be affected directly by the quantity of I/O that occurred between backups.

And of course you can always temporarily disable CBT to see how much of a difference it makes in your specific scenario.

Edited 28 November 2023 10:00 PM by jphughan
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search