Macrium Support Forum

MS-DOS batch file invocation - command line switches and scheduling

By Bill R - 17 January 2016 2:07 PM

Please forgive this new user, but I can't get my mind straight about how reflect.exe handles grandfather scheduling when invoked from a .bat file.  I'm trying to set up a .bat file which will, on completion of a daily incremental backup, temporarily connect to a network server and robocopy the new backup file(s) across to the server.  Eventually, I'm thinking also of arranging for the less frequent differential and full backups to do likewise but only when explicitly triggered (I anticipate the larger network transfer for these to take hour(s) and I only want them to run when it suits me).

From what I've can see so far, some aspects of the scheduling of a normal .xml definition appear to be integrated into the .xml definition, although the Windows Task Scheduler lists separate tasks for each of full, diff, and inc.  So am I right in thinking that, provided the backups are otherwise "similar", reflect.exe somehow gets to "see" all of the scheduled  tasks before they are run to work out which one to actually perform?

Given that, I note that the reflect.exe has command line switches -full, -diff, and -inc, and the command help is pretty explicit in stating that if none of those switches is specified, then a full backup is performed ... "by default".   Sorry to be pedantic, but what does "by default" actually mean?  Does it actually perform the backup implied by the definition backup plan info in the .xml file?

And if I create a .bat file, right click it in Reflect and attempt to schedule it, am I scheduling the .bat file, inserting some backup plan info into the associated .xml file, or both?

Or ... and I suspect this may be the case ... is it just that the Definition Backup Plan info in a .xml file doesn't apply / is ignored when reflect.exe is run from a .bat, .vbs, or .ps1 file?

Bill R
By Drac144 - 17 January 2016 10:09 PM

By default means that if you do not specify which type of backup to perform (full, differential, incremental) then a full backup will be done.

The XML file provides info to Reflect on what do for a backup.  The XML file name is passed as a parameter to Reflect.exe in the batch file. 

Reflect does not "see" anything.  When a scheduled task runs, the task will invoke Reflect with the XML file you specified to use for THAT scheduled task.  If you have multiple scheduled tasks then each will pass the XML file name that is specified IN that task (whether its format is DOS Batch, VBScript or PowerShell).  Tasks are independent and have no knowledge of each other.

Normally YOU do not create a file to run as a scheduled task.  Reflect will create that file for you, then you can modify that file to add logic to, for example, copy the backup to another drive.  Since Reflect has created that backup, and it does so by YOU specifying WHICH XML file to use when running that backup, each backup is related to one and only one XML file.

By Arvy - 17 January 2016 11:06 PM

=Bill R --- I'm trying to set up a .bat file which will, on completion of a daily incremental backup, temporarily connect to a network server and robocopy the new backup file(s) across to the server.  Eventually, I'm thinking also of arranging for the less frequent differential and full backups to do likewise ...

Perhaps I've misunderstood that initial part of your plan, but does it indicate an intention to begin by copying only the newly created incremental backup files and not the full and differential backup files on which the incrementals are based?   If so, it should be clearly understood that incremental backups alone (i.e., without the prior backups which are their "points of departure") will not provide a valid backup set for recovery purposes.  In fact, any "missing link" (or a corrupted one) in an incremental chain invalidates everything thereafter.
By Bill R - 17 January 2016 11:48 PM

=Arvy .. sorry, I phrased that badly.  I should have said "The less frequent differential and full backups will also be robocopied likewise to the network server, but  I'm thinking of eventually arranging for those to be run and robocopied only when explicitly (i.e. manually) triggered (I anticipate the larger network transfer for these to take hour(s) and I only want them to run when it is suitable to do so, space- and time-wise)."
By Arvy - 18 January 2016 12:02 AM

No problem, Bill.  I suspected that the warning might not be required, but just wanted to be sure that you weren't headed unaware for a potentially disastrous result.

I would have thought that you'd probably want "extra insurance" for your most recent full and differential backups whenever they're created, perhaps even more than for your latest incrementals, but that's strictly an individual judgement call, of course, and I can certainly understand your wanting to trigger the processes during idle time.
By Bill R - 18 January 2016 12:35 AM

I understand that creating ONE grandfather-father-son Backup Definition prompts me to enter THREE separate schedules, one for each type, and results in THREE separate corresponding entries in my Windows Task Scheduler list, EACH referring to the same XML file but with an appropriate -full, -diff, or -inc parameter specified.  I understand from other posts (e.g. ) that "if they are all set up for the same time ... Macrium figures out the right thing to do and does it (Full over Diff and Inc, Diff over Inc, etc)".

If, as you say, "Tasks are independent and have no knowledge of each other" what is it that enables Macrium to figure out "the right thing to do" .. i.e. to initiate only the dominant type of backup?  Or have I got the wrong end of the stick?

I'm realising that if I want to go on to creating an equivalent MS-DOS Batch file (using the Reflect UI) which I can modify, I will actually need to create three MS-DOS batch files, one for each type, and will need to schedule each one separately.  If I do so, will the mechanism which ensures that only the dominant type of backup is created still to continue to work? - I can't think why it shouldn't, except for the fact that I don't actually know how Macrium does this.

I appreciate that an alternative solution would be to imbed some conditional logic into just one batch file, but unless I also put more logic into the scheduling I could still finish up with three instances of the batch file being initiated simultaneously.  If Macrium can sort that out for me, so much the better (or rather, easier).

Bill R
By Arvy - 18 January 2016 3:57 AM

AFAIK Bill, Reflect does load and run its own service (file = ReflectService.exe, described in the registry as Macrium Reflect Scheduling Services) as an intermediary for handling Windows Task Scheduler operations when running under the regular Windows working environment (not WinPE).  My understanding is that any scheduled operations with simultaneous triggers are handled sequentially, but I'm really not sure why that would be an issue for you with each type of backup (full, differential and incremental) having its own task definition and set-up entry in the scheduler.  If you're experiencing actual problems and need more in-depth technical assistance, you might want to open a direct support request ticket with Macrium.
By Bill R - 18 January 2016 10:38 AM

Arvy, tx for that info. I don't have any actual problems at the moment, but am just trying to ensure I don't create problems.

Scenario -
1) Three scheduled .bat files start simultaneously, -full, -diff, -inc, all referencing the same .xml file
2) They all invoke macrium.exe, which, using its own scheduling service, ensures that only one creates a backup.
3) In each .bat file, macrium.exe returns to the .bat file which then invokes robocopy.
4) According to how it works, that potentially could result in two or more of my robocopy steps overlapping not only with themselves but with the creation of the "dominant" backup file.
5) .. with undesirable results.

Possible solutions
1) Find some way of testing the macrium.exe outcome, to suppress the robocopy phase if the particular type of backup was skipped
2) Introduce some form of interlocking, either before or after the macrium invocation, to prevent overlapping
3) Only run the -full and -diff manually when they won't clash with the .inc or with each other (which would be a shame given the effort macrium has put into trying to deal with that situation).

It all depends on how it actually works at present, but I don't fancy the "interlock" solution, especially in a .bat context. I'm an "old-school" systems application designer, and my experience with interlocks is that they're best left to someone else ...

Bill R
By Arvy - 18 January 2016 11:52 AM

Given that Reflect will handle simultaneous triggers sequentially, I'm still not completely certain that I fully understand your dilemma, but if you're not locked into using .BAT file logic for some reason, you might want to consider some other option such as a VBScript.  Reflect will generate one for you if you right-click on the .XML task definition and select the "Generate a VBScript File" option.  It includes a GetBackupTypeParameter function that allows main process differentiation based on the type of backup (full, differential, or incremental) that is specified on the command line as follows:

Function GetBackupTypeParameter
 Dim i

 for i = 0 to Wscript.Arguments.Count - 1
     If Wscript.Arguments(i) = "-full" OR _
        Wscript.Arguments(i) = "-inc" OR _
        Wscript.Arguments(i) = "-diff" Then
         GetBackupTypeParameter = Wscript.Arguments(i)
         Exit Function
     End If

 GetBackupTypeParameter  = ""

End Function

By Bill R - 18 January 2016 1:09 PM

Ahhh .. I think I've discovered the source of one of my quandries  ...

When generating a .BAT file, the Macrium UI asks for and fixes the backup type when it creates the .BAT file.  If I then use the Macrium UI to schedule that .BAT file, it only permits me to add schedule(s) of that backup type. Hence I have to create three differently-named .BAT files - one for each backup type.

When generating a .VBS or .PS1 file, the Macrium does NOT ask for the backup type when it creates the file.   If I then use the Macrium UI to schedule that file, when I add a schedule it THEN asks me for the backup type, and THAT permits me to add a schedule for each type using just one .VBS or .PS1 file.

I suspect the .BAT file is treated differently, because otherwise conditional parameter logic would need to be included in the .BAT file, and although it could be made to work, it would be rather messy.

So, let's assume I'm working with a single .VBS file (or .PS1 file - I don't think it makes any difference).  In which case, my other quandry remains.  There will still be three schedules in place for that .VBS file, one for each backup type.  If those schedules are actioned simultaneously (not unlikely), then, according to how reflectservice.exe works, that could result in up to three simultaneous instantations of the .VBS file.    Three instances of robocopy could then find themselves being executed simultaneously.  And THAT could result in undesirable duplication, or even triplication, of the network transfers, or I suppose there could even be a data corruption scenario possible in there somewhere.

I think my ideal solution would be a return code from reflect.exe to indicate when one of the backup types is suppressed for the above reason. The .VBS /.BAT /.PS1 file could then suppress any following robocopy.  Any idea what code is returned in that event?   It would be nice if it was not an "error" code as such, since it wouldn't merit the backup being flagged as having failed.

But, as I said, it all depends on how reflectservice.exe actually makes this feature work ...

Bill R
By Arvy - 18 January 2016 6:24 PM

=Bill R --- I think my ideal solution would be a return code from reflect.exe to indicate when one of the backup types is suppressed for the above reason.

There may be a list of return codes posted somewhere, but I've not seen one.  There was an answer (from Nick, if I recall correctly) to another forum question saying that both cancelled backups and xml validation errors return exit code '1', but I recall seeing no other return code specifics.  I think you're going to need some input directly from Macrium, either here or via a support ticket, on that question.
By Drac144 - 18 January 2016 10:39 PM

The following is MY understanding of how Reflect works.

If 3 tasks are scheduled to run at the same time, whichever task the scheduler runs first will be executed.  The other two will be sent to Reflect but not executed until the first task completes.  Reflect does NOT look at all 3 and decide which one to execute first.  If you want then done in a specific order, schedule them a minute apart in the order YOU want.

You can set the task scheduler to only execute a task on certain days so you can skip doing an incremental (and/or) differential on the day you run a full backup.  It makes no sense to run an incremental or differential immediately after running a full backup.

By Bill R - 19 January 2016 12:24 AM

Drac144,  Well, that seems to be at variance with what I've read in a number of other posts, in particular ...      (the end of this last post)
the last two posts being from Nick, a Macrium Rep.   Could I ask a Macrium Rep to confirm which understanding is correct, please?  (If necessary, I'll raise a ticket).
(And sorry about the poorly formatted links - I really struggled with the forum tools to get those in at all).

Separately, I've noted another comment from Nick at  the end of  in which he writes ...
You've hit on a weakness with the scripting and scheduling options in Reflect. Unfortunately, scripts and XML files are scheduled independently, and the new backup plan templates include 'Retention' rules that are only available in the XML. The solution is to 're-wire' the scripting logic so that scripts are called by the XML def. That way you only need to schedule the XML not the scripts. This functionality was originally going to be included in the initial v6 release but has now been re-scheduled for v6.1. v6.1 will be a Free upgrade for all v6 users

Since we are now on 6.1, I get the feeling this change may not have happened, but it certainly sounds like it might affect my robocopy issues.  I'll raise a separate post in the Scheduling section to ask about this (which, I guess, is where i should have placed this post in the first case).

Bill R

By Drac144 - 19 January 2016 9:42 PM

Interesting.  The information I gave you is, apparently, correct for running tasks involving DIFFERENT XML files.  It sounds like Reflect does some different logic if it gets multiple requests involving the SAME XML file.  I (obviously) did not know that.

I wonder how much time must pass for the logic to treat the multiple requests as tasks to run in order. If I issue a Full at 9:00 AM and an incremental at 9:01 AM will it ignore the incremental and just complete the full, or will it stack the incremental for completions AFTER the Full completes?  If the order is reversed, will it abort the incremental and do the full instead or stack them for sequential execution?  What are the timing rules in play here?
By Bill R - 19 January 2016 10:49 PM

Drac144, Exactly the questions that were springing to my mind.  I got the impression from one of the postings that the resolution logic would only take into account backups of the same set that were SCHEDULED to start at exactly the same time, regardless of what time they actually started.  That suggests to me that the logic might inspect the Task Scheduler queue and then act accordingly.  I guess each such scheduled task could choose to 1) suppress any less-dominant matching backup task queued up behind it, and 2) abandon itself in the presence of any more dominant matching backup task queued up behind it.  But that's only a guess. "reflectservice.exe" seems to play a role somewhere in this process.
By Arvy - 19 January 2016 11:52 PM

@Bill R ---  My own experience as well as previous answers in these forums do indicate that simultaneously scheduled Reflect operations are usually handled sequentially (i.e., each one being run and completed in turn) as I said.  But it is equally clear from the other authoritative answers that you've quoted that the "Macrium Reflect Scheduling Services" cannot possibly be limited to just that single method for handling simultaneously scheduled jobs when they are based on the same XML task description.  It may be noteworthy that the Windows scheduler entries do also include a {GUID} value as well as the -full, -diff, -inc command line argument.  I, for one, would be grateful if you would pass along whatever else you're able to find out about it by whatever means you may pursue.

In view of Macrium's reticence on the subject here so far, and as an alternative to opening a direct support ticket which you seem reluctant to do, have you considered an experimental approach that would log the results (return codes, or whatever else might be useful) for each instance of a minor (even miniscule) scheduled Reflect operation of the same "Grandfather-Father-Son" type as your primary project?  I'd do it for you myself, but I'm somewhat preoccupied with other issues at the moment ... and/or just too lazy, take your pick. Smile
By Gork - 20 January 2016 4:43 AM

I've found myself wondering the same thing as I've been reading this thread.  What piqued my interest is the fact that if you use the "Grandfather, Father, Son" template it sets the Full, Differential and Incremental to all run at 0900 hrs on the first Mon of every month.  (Diff and Inc actually run every Mon at 0900.)  While I've read in the past that Reflect will run multiple schedules at the same time sequentially, it didn't make any sense to me that an actual template would be set up to run a Full, Diff and Inc all at the same time.  So I figured there is something else at play here and have been watching the thread hoping to learn.
By Bill R - 22 January 2016 12:34 AM

Well, I've just run some tests, and the results were contrary to my expectations.  It looks like there was NO process co-ordinating the backup tasks other than to ensure that they only ran one at a time, just as Drac was saying.

1) I ran a small backup job, with -full, -diff, & -inc schedules, using the generated VBScript.  In the first instance, I placed msgboxes before and after the reflect.exe invocation in the VBscript.  This permitted me to progress all three backup tasks to a point just prior to the reflect.exe invocation, so that I could then release the tasks in whatever order I wanted.  When I did so, the actual backup processes ran to completion in whatever order I released them, with no evidence of coordination other than to serialise the processes.  Any backups that ran AFTER the -full backup, completed with return code 0, but with an unsurprising message that no changes had been detected (as a result of which no backup file was created).

2) I then ran the same backup job, with no msgbox BEFORE the reflect.exe, in case the msgbox pause was interfering with any process attempting to coordinate the tasks.   The tasks tended to run in the order -full, -diff, -inc, but I fairly quickly had one set of tasks that ran in the order -inc, -full, -diff, creating TWO backup files as a result, with the 3rd not created because of lack of further changes.  And as they say, it only takes one false outcome to disprove a theory ...

For each test run, I was re-scheduling the backups to run simultaneously after a delay of 1-2 minutes.
I guess, it's still possible that some coordination takes place when the schedules are set using the standard XML, rather than generated script files, but I haven't tested that since my robocopy plans require that I use a generated script. So it looks like I will be resorting to Plan C - robocopy for the incrementals, with manual invocation for my -diffs and -fulls.

If I get the time, I may repeat these tests to confirm the results.

And, of course, I wasn't able to determine what return code would arise if a backup task was suppressed (other than for lack of changes) since that event never arose.

Bill R

By Arvy - 22 January 2016 3:43 AM

Well, there's certainly no doubt that running simultaneously scheduled tasks one after the other is Reflect's normal way of handling them, but it does seem to be a rather inelegant method with this kind of Grandfather-Father-Son arrangment to say the least.  It's not surprising that you expected something a little more sophisticated based on those other replies that you quoted.  I'm quite sure that none of them were deliberately misleading and maybe you're right in supposing that they only apply when the scheduler runs XML task definitions directly.  I am surprised that no one from the Macrium team has jumped in here with some clarification by now, but they have been strangely quiet here recently and not just on this subject.  Too busy squashing development bugs in their forthcoming new super fast file system change block tracker, I suppose. Wink

In any case, many thanks for taking the time to pass along what you were able to discover by experimentation.  And the best of good luck getting things set up to work as you want them to.
By Drac144 - 22 January 2016 7:35 PM

That is much more as I would expect.  It was surprising that Reflect would have some kind of process to detect the specific instance of multiple backups using the same XML occurring within a few seconds of each other.  Processing requests in the order received is much simpler.  I am surprised someone from Macrium has not responded with the actual operation of Reflect in the conditions we are discussing.  I would not think this is a company secret.
By Gork - 24 January 2016 6:22 AM

I've posted this before, but I don't even schedule fulls and incrementals on the same day.  I update the Windows Task Schedule entry to run on days 1, 8, 15 and 27 of each month.  I updated the VBS file which created within Reflect to check what day of the month it is when it's executed.  If it's day 1 of the month it runs a full backup, else it runs an incremental.  This is how I've gotten around any possible problems associated with scheduling full and inc/diff at the same time.  And this is why I'm probably not fully aware how Reflect handles these instances.

I too would like to hear from Macrium on this subject.
By Nick - 24 January 2016 6:26 PM


Apologies for not chiming in earlier on this topic.

To ensure that simultaneous schedules of the same XML file, in an MSDOS, PowerShell or VBScript script, do not run a Full, Diif or Inc at the same time, edit the source code and add the '-g' (GUID) parameter to the end of the reflect command line. Note for MSDOS .bat files this needs to be added to each bat file. For PowerShell or VBScript only one script file is required. 

-g "any unique text"

The '-g' parameter can be any text and is used when multiple backups for the same XML def are scheduled to run at exactly the same time. In this case the following happens:

'Full' backups take precedence over 'Differentials' or 'Incrementals'. 
'Differential' backups take precedence over 'Incrementals'.

Scheduled backups that are 'Trumped' by another scheduled backup configured to run for the same XML file at the same time will intentionally fail with error '0x06 Conflict' in the 'Last Result' column of the 'Scheduled Backups' tab. This processing is handle by 'Reflect.exe' and doesn't use any functionality in 'ReflectService.exe'

Hope this helps
By Arvy - 24 January 2016 6:37 PM

Ahah! Thanks, Nick. That explains a lot.  I knew when I said (back on page 2) that "it may be noteworthy that the Windows scheduler entries do also include a {GUID} value as well as the -full, -diff, -inc command line argument" there had to be some task management reason.  This makes their presence in entries based on the same XML task definitions much clearer and quite a neat practical solution if I may say so. Cool

So exactly what does ReflectService.exe do anyhow, if it's not an ULTRA-class trade secret?  The Windows registry name and description for that service remain confused and confusing despite a promise last June that they would be "fixed in a later release".

By Nick - 24 January 2016 7:46 PM

Hi Arvy

'ReflectService.exe' actually has several functions dependant on the Edition of Macrium Reflect installed. The only functionality relating to scheduled tasks is to check if there are any missed backups when the service starts. Other functionality relates to Server Plus and SPLA licensing so it isn't appropriate to discuss this within the context of this thread. 
By Arvy - 24 January 2016 8:48 PM

Right.  Got it.  Thanks.  Still, it would be nice if, whenever someone can get around to it, the registry name and description didn't contradict one another. Smile
By Bill R - 28 January 2016 12:17 AM

Thanks for the info, Nick.   However, when I tried the tests I had used before, I wasn't able to trigger the condition.  They all seemed to complete successfully (with result 0x00), either creating a backup, or indicating that there were no files to be copied, according to the order of invocation.  Using a VBScript, my command line reads
ExitCode = Backup ("""C:\Program Files\Macrium\Reflect\Reflect.exe"" -e -w -g ""test"" <BACKUP_TYPE> ""N:\Backups-Macrium-Test\My Test Backup.xml""")

I also tried it without quotes around "test", and with/without a msgbox preceeding the Backup invocation, to optionally control the order of invocation.  Although you said
 The '-g' parameter can be any text, does it actually have to be the appropriate {GUID}?  Or have I positioned the -g parameter incorrectly?  Or, I'm running the Home Edition of Macrium reflect (6.1). Could it be that this function is not in that edition?
Bill R

By Arvy - 28 January 2016 1:10 AM

The ultimate authoritative answer should come from Nick, of course.  In the meantime, however, you might want to try your scripts' command lines in the same way that Reflect itself adds such simultaneous items to the Windows Task Scheduler when asked to do so based directly on an XML task definition, which is as follows:

C:\Program Files\Macrium\Reflect\reflect.exe -e -w "C:\Users\Username\Documents\Reflect\My Scheduled Backup.xml" -full -g {GUID1}
C:\Program Files\Macrium\Reflect\reflect.exe -e -w "C:\Users\Username\Documents\Reflect\My Scheduled Backup.xml" -diff -g {GUID2}
C:\Program Files\Macrium\Reflect\reflect.exe -e -w "C:\Users\Username\Documents\Reflect\My Scheduled Backup.xml" -inc -g {GUID3}

That scheduled task command format seemed to work in the manner that Nick described it when I tried it.  The {GUID} values seem to be random, but they are placed last (no quotes) following the task definition filename and the -<BACKUP_TYPE> argument and note the space after the -g.  I guess I should have been more specific when mentioning it in my page 2 reply.  Sorry about that.

Macrium should try to automate the -g {GUID} inclusion where appropriate in Reflect's script creation process, but the various script types, task definitions and scheduling templates make it a little more complicated than it may seem at a glance.  Alternatively, they should implement that previous idea of Nick's stated in one of your quoted replies about making the XML task definitions call the scripts rather than vice versa.  I guess that new "super fast" file system change block tracker has taken over top priority. Crying
By Nick - 28 January 2016 8:38 AM

Hi Bill 

Thanks for getting back. 

Sincere apologies. On closer inspection the '-g' parameter is only designed to work with XML files scheduled directly on the Windows Tasks Scheduler. (Your command parameters were fine by the way). 

We will update Reflect to handle VBScript, PowerShell and BAT files in the next release, due on Monday. 

By Arvy - 9 February 2016 7:00 PM

UPDATE:  As promised, the v6.1.1081 release notes include script scheduling improvements as follows: "Concurrent Script scheduling now works in a similar manner to XML file schedules. Scheduling multiple scripts (VBScript or PowerShell) running the same XML file at the same time will now use order of precedence favouring Full over Diff over Inc."

I note, however, that only Reflect's direct XML task scheduler entries follow the -g parameter with a {GUID) value on the command line.  Reflect's script generation (vbs, ps1 and bat) adds the -g parameter to the scripted command line, but without including any {GUID} value.  Is the difference intentional, or is the user required to add some (random) value manually?
By Nick - 9 February 2016 10:14 PM

Hi Arvy

'-g' is actually all that's required to enable exactly the same behaviour as with XML schedules. The XML file name referenced in the Reflect parameters is used as the 'map' to determine whether backup tasks are considered conflicting. 

The GUID that's included after the -g in XML schedules is actually redundant and isn't used in this process. My sincere apologies for my misleading post regarding this. The GUID was originally intended to be used for this purpose but is now superfluous, however, we may use it for future scheduling logic implementations. 
By Arvy - 9 February 2016 11:59 PM

Nick - 9 February 2016 10:14 PM
Hi Arvy

'-g' is actually all that's required to enable exactly the same behaviour as with XML schedules. The XML file name referenced in the Reflect parameters is used as the 'map' to determine whether backup tasks are considered conflicting. 

The GUID that's included after the -g in XML schedules is actually redundant and isn't used in this process. My sincere apologies for my misleading post regarding this. The GUID was originally intended to be used for this purpose but is now superfluous, however, we may use it for future scheduling logic implementations. 

Thanks for the clarification, Nick.  No problem about the previous post.  I think we were all "re-exploring" this one a bit and it's good that it got sorted out as a result.