Please publish a way to catch the backup method used for an existing backup file


Please publish a way to catch the backup method used for an existing...
Author
Message
RefDM
RefDM
Talented Member
Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)
Group: Forum Members
Posts: 80, Visits: 220
This thread is a continuation topic for an older one I originally posted in this V6 Wish List Topic.

We are currently somewhat stuck in our project of migrating over to Macrium (we have a few workstations happily running MR7 but many are still using another backup solution). There are not many obstacles to overcome to continue and finish the migration, maybe the most prominent is that I have not had enough time to study and find a robust way to catch the backup method used for an existing backup file. And I mean to catch it not manually but instead automatically from inside a program or script.

Please refer to the V6 Wish List topic I linked above to get the idea what I 'm trying to accomplish. Basically it's a question of integrating MR with our in-house developed backup & archive management system. 

So, I wonder if there has been any progress, or if any new methods (officially supported or not; documented or not - but robust methods) have been developed, invented or found during the last four years to obtain the backup method used to create a specific backup file xxxxxxx.mrimg?

All ideas are greatly welcomed (as long as they do not suggest to change the way we use to manage our backups... Smile)


BGregory
BGregory
Proficient Member
Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)Proficient Member (241 reputation)
Group: Awaiting Activation
Posts: 150, Visits: 307
RefDM - 9 January 2021 4:01 PM
This thread is a continuation topic for an older one I originally posted in this V6 Wish List Topic.

We are currently somewhat stuck in our project of migrating over to Macrium (we have a few workstations happily running MR7 but many are still using another backup solution). There are not many obstacles to overcome to continue and finish the migration, maybe the most prominent is that I have not had enough time to study and find a robust way to catch the backup method used for an existing backup file. And I mean to catch it not manually but instead automatically from inside a program or script.

Please refer to the V6 Wish List topic I linked above to get the idea what I 'm trying to accomplish. Basically it's a question of integrating MR with our in-house developed backup & archive management system. 

So, I wonder if there has been any progress, or if any new methods (officially supported or not; documented or not - but robust methods) have been developed, invented or found during the last four years to obtain the backup method used to create a specific backup file xxxxxxx.mrimg?

All ideas are greatly welcomed (as long as they do not suggest to change the way we use to manage our backups... Smile)


If I am understanding you correctly, if you are trying to get the type of backup after it is complete, why not just read the HTML file that is created?
All backup information is contained there.  Do you have any onsite programmers\script writers\Powershell gurus that can do this?

Bob Gregory
RefDM
RefDM
Talented Member
Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)
Group: Forum Members
Posts: 80, Visits: 220
I see and agree that it could be done that way (at least if we could trust that the html file format would not change, and so long the html log files are not purged). The need for detecting the backup method however remains even long after the backup was taken, and in some cases preferably even on a computer other than where the backup was taken. So that would not be straightforward to maintain.

I would very much like to dig the information out of the backup file itself, because then there would not be any external dependencies.

I have learned to like MR a lot during the last years but I feel it's really a pity that there does not seem to be a way to find the information of the backup method used from the backup file itself - in a robust and controlled way... unless someone has find a way since 2017 when I asked this previously.

This is a non-profit organization so almost all of the work is done by volunteers. I believe that I can write a program to parse & read the info from the html file, but then I'd have to start maintaining that information in some database and take care of replicating that database along with the backup files... and soon it would not be any longer a simple, robust system at least as far as I can think of it.

(FWIW, I think that Macrium is the first backup program that I have used since mid 1980's where getting this information out of the backup file is not trivial. Quite often that information can be obtained directly from the backup file name... but I do understand that when MR was first developed, the developers wanted to be able to split the backup file into fixed length chunks and use the file name for storing that information instead of the backup method.)
dbminter
dbminter
Macrium Hero
Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)Macrium Hero (2.7K reputation)
Group: Forum Members
Posts: 2K, Visits: 19K
I'm not entirely sure I understand what you're asking for.  Do you mean a way to tell if an image was a Full, Incremental, or Differential?  If that's what you're looking for, it's already in the software.  Under the Restore tab, open an image.  There's a Type field in the list of metadata at the bottom of the page of information and operations to perform.

Beardy
Beardy
Talented Member
Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)Talented Member (144 reputation)
Group: Forum Members
Posts: 115, Visits: 458
I use a simple scheme, only doing fulls & incrementals, with file splitting thus I can't use consolidation, but CAN determine this from the filename.

Anything ending 00-??.mrimg/mrbak is a full or part thereof 
Anything ending ??-??.mrimg/mrbak & not matching the above first is an incremental
Any backups where the 16 character hex names match are part of a set

I'm not sure how differentials, or consolidation, would play into that, but this alone may solve the issue if you don't use them.

So far as other methods go, subject to time on your hands, & not minding a lot of disk activity, & some specific criteria being met, namely:

  1. You have WSL installed along with some Linux distro,
  2. Your backup definitions are separate for Full, Differential, & Incremental
  3. You don't encrypt your backups (at least, this is untried with encrypted backups)

Then since the definition file is embedded in each backup, then the following, issued in a command window in the directory where the backups are stored:

wsl grep -iaHEm 1 "<method>[0-3]</method>" *.mrimg *.mrbak >> BkpType.txt

Apart from grep spitting an error into the command window if either type is missing this will *EVENTUALLY* spit out a list in the format:

FILENAME1   <method>N</method>
FILENAME2   <method>N</method>

Where N translates as follows

0 = Full
1 = Incremental
2 = Differential
3 = Auto (or specified on command line as happens in my backup script)

Which is no use to me, all my backups are method 3, but prove if nothing else I made the attempt, unfortunately the scripting/Windows automation tools I tried wouldn't return anything useful, & the info needed seems to be an inconsistent number of lines from the end, so
wsl tail -n XXX
 

Piped to grep turned out less useful than I'd hoped speeding up the process, I think a variable length index is stored afterwards, or something, though there may be some scope there.

In terms of optimising this, anything [Sixteen Digit HEX]-00-??.extension is always a Full, I'll leave scripting skipping those to you, but Linux (and thus WSL) has good regex handling, so it should be possible to only check to distinguish Diffs. from Incs.

I get tied in knots by tokens, delims, etc in a DOS batch file, though searching the files using Windows native abilities is probably also possible.

RefDM
RefDM
Talented Member
Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)
Group: Forum Members
Posts: 80, Visits: 220
Thank you all for your replies so far!

As I mentioned in the original V6 Wish List topic I linked into my OP, "we need to use that information inside a batch script or program code".

So, we need to dig the information out entirely independently of Macrium Reflect backup jobs and outside and independently of the Macrium User Interface. And since the need may exist even after the backup log files are gone (or in some special occasions even after backup file naming has been reset and identical backup file names with different contents may exist on the backup disk and on an external archive disk), I think that the most robust way to obtain the backup method must be to dig it out of the backup file itself.

And the problem is naturally in differentiating differential backup files from incremental bakup files.

Edit: regarding Beardy's idea: I like that idea because that's exactly digging the info out of the backup file itself. However, unfortunately our backup files are encrypted and some of them (there are several independent backups jobs) have a file size of about 600 GB currently, performing a sequential scan thru the backup files is practically not an option.
Edited 10 January 2021 11:24 AM by RefDM
Froggie
Froggie
Macrium Hero
Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 11K
@RefDM, there is a MetaData attribute used by REFLECT called "Backup Method."  You can see it in Explorer by <right-click>ing on Explorer's TITLE BAR (NAME header) and selecting "Backup Method" under the "More..." option.  After doing this, the Backup Method is available in the Columns available.  If it's there, I'm sure there's a Windows API available that can get you that information, especially if Explorer can get it.
Froggie
Froggie
Macrium Hero
Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 11K
Since "Backup Method" is part of the EXTENDED FILE PROPERTIES of any FileType, here's a DISCUSSION on using Power Shell to access the EXTENDED PROPERTIES of any FileType.

Extended file properties are part of the System's MetaData associated with that file within the NTFS environment.

Edited 10 January 2021 1:27 PM by Froggie
RefDM
RefDM
Talented Member
Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)Talented Member (101 reputation)
Group: Forum Members
Posts: 80, Visits: 220
@Froggie Thank you! This is so far closest to what I'm aiming to... unfortunately "Backup Method" is not listed in any of the 267 entries the PowerShell script produced.

If it were, that would have been my solution!

Froggie
Froggie
Macrium Hero
Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)Macrium Hero (2.1K reputation)
Group: Forum Members
Posts: 1.2K, Visits: 11K
I think the list of available attributes is up to 311 currently... maybe a script change could get you what you need.
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search