Can VSS handle apps that modify several files?


Author
Message
idf
idf
Junior Member
Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)
Group: Awaiting Activation
Posts: 50, Visits: 152
My email program checks for new mail every 3 minutes. When new arrivals, several mailboxes and their indexes get updated, as well as a few other tables. Since thsi program is not a VSS Writer, it seems to be that an occasion may occur (and may have already occured) where some of these updates have been finished, and some not yet started, as the time VSS makes a snapshot. The solution might be to not run the mail program during backups - which isn't practical.

I've also wondered about multiple drive consistency. Some data files on the D drive ought to be coordinated with Registry hives and Program Data on the C drive. But Reflect starts a new snapshot for each such drive. Seems any cross-drive data could get out of synch.

My main reason for using Refelect for file backup is to get the VSS protection. In fact, after I get my backup, in a post-execution bat file I mount it and robocopy the contents to another location (where modified files get backed up to Time Machine and CrashPlan's cloud service, both of which support frequent backups and version control). The .mrbak files are really just work files I can delete. Would a straight robocopy of the source folders have equally good integrity?

My system as it is is not foolproof. I mount using the "LAST_FILE_CREATED" parameter, which so far has worked, but could eventually fail to mount the right file. And, I always mount it to N, but sometimes N might still be mounted (usually due to a glitch the last time the job ran), in which case some other letter is assigned, and the robocopy works on the wrong files. I could avoid all of that by just running robopies from the source data. Depends on just how much benefit non-writer applications like mine, that write to multiple files , get from VSS.







Nick
Nick
Macrium Representative
Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)
Group: Administrators
Posts: 1.9K, Visits: 11K
Hi 

Thanks for posting.

My email program checks for new mail every 3 minutes. When new arrivals, several mailboxes and their indexes get updated, as well as a few other tables.


Without a VSS writer your email program data is backed up in a crash consistent state,  i.e, it is in the same state as if the power were turned off.  Many programs without VSS writers can handle power failure by using journalling for their internal db updates so a restore will not present a problem.  

I've also wondered about multiple drive consistency. Some data files on the D drive ought to be coordinated with Registry hives and Program Data on the C drive. But Reflect starts a new snapshot for each such drive. Seems any cross-drive data could get out of synch.


To avoid synchronization issues across multiple drives during the snapshot phase multiple snapshots are created simultaneously. This is not supported by 32 bit Windows less than Windows 8.  (All 64 bit Windows OS support this, and both 32 bit and 64 bit for Windows 8 and later). 


Kind Regards

Nick - Macrium Support

Next Webinar


Edited 7 March 2016 12:15 PM by Nick
idf
idf
Junior Member
Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)
Group: Awaiting Activation
Posts: 50, Visits: 152
Thanks for the info. My email program has no journalling feature.

Exchange apparently is a VSS writer. But what about standalone Outlook? If not, does it do journalling to maintain integrity of the .pst file?

My file backups are small enough that I can eyeball the modified dates of Reflect's "Files backed up" log, or my robocopy log. It seems that any files whose modified dates are within a few seconds of the time a snapshot happens are at risk of inconsistency and need to be reviewed.

I'm going to add a couple of Wish List feature requests to assist in recognizing potential backup inconsistencies.

EDIT: Is it possible that ANY file written by a non-VSS-aware application could be in an inconsistent state even if it's a file that has no relationship to any other file? Presumably VSS pauses while in-progress file system activity completes, so that file system indexes and so forth are in a consistent state, but what of an application that performs several sequential writes in order to make a single "logical" change? That isn't really different than updating several coordinated files, and so couldn't a crash during this process, or a snapshot taken during this process, be "corrupt" in the sense of leaving a file in an unintended state?





Edited 10 March 2016 10:51 AM by idf
Nick
Nick
Macrium Representative
Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)Macrium Representative (3.3K reputation)
Group: Administrators
Posts: 1.9K, Visits: 11K
@idf

Thanks for getting back. 

The VSS framework is in the control of Microsoft and MS applications that don't have a dedicated VSS writer probably don't need one. Even though there is no specific VSS writer for Outlook, the file system is 'flushed' and any uncommitted NTFS cached writes will be written to disk. This is actually better than simply losing power and Outlook .pst files will maintain transactional consistency. 

My file backups are small enough that I can eyeball the modified dates of Reflect's "Files backed up" log, or my robocopy log. It seems that any files whose modified dates are within a few seconds of the time a snapshot happens are at risk of inconsistency and need to be reviewed.


Having a modified time stamp at the time of the snapshot does not indicate that there are any inconsistency issues at all. Most (all modern) applications will handle 'crash' consistent termination without loss of data.  

EDIT: Is it possible that ANY file written by a non-VSS-aware application could be in an inconsistent state even if it's a file that has no relationship to any other file? Presumably VSS pauses while in-progress file system activity completes, so that file system indexes and so forth are in a consistent state, but what of an application that performs several sequential writes in order to make a single "logical" change? That isn't really different than updating several coordinated files, and so couldn't a crash during this process, or a snapshot taken during this process, be "corrupt" in the sense of leaving a file in an unintended state?


It is possible for a poorly written, non VSS aware application that uses it's own database format and makes no attempt for atomic transactional writes to be inconsistent after power failure or snapshot. This won't be a modern or serious application though. 

The MS VSS framework does a great job of ensuring that snapshots of a running system are in a consistent and clean state, however, If you are running applications that are not VSS aware and that can irreparably 'break' due to power failure then you might want to consider whether to let them run during snapshot creation. 

Kind Regards

Nick - Macrium Support

Next Webinar


Edited 10 March 2016 1:22 PM by Nick
idf
idf
Junior Member
Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)Junior Member (72 reputation)
Group: Awaiting Activation
Posts: 50, Visits: 152
I guess that my Wish List VSS suggestions aren't going to be implemented then Smile.

You are correct - my email application isn't remotely modern. It was last updated in 2006! Lots of people do still use it because of its unique feature set. It's a miracle it continues to work, even through Windows 10 I'm told. 32-bit only, so its days are numbered. It does make backups of a couple of critical mailboxes, though they may be old, and most mailboxes have no protection at all.

I'm still looking into a way to shut down the mail program automatically as part of a pre-execution program. Certainly when you do things like Shutdown, applications are notified and Windows won't stop until programs close. Programs are not force quit unless the user requests that. So there must be a way.

Sysinternals offers pskill and pssuspend, but kill would have the potential of not just creating an inconsistent backup, but of actually making the current mailboxes inconsistent. Suspend wouldn't help - the process might be suspended in the middle of a sequence, no different from a snapshot which could equally be mid-sequence.

If anyone knows of a simple way to send a graceful shutdown message to a program, and wait for its termination before proceeding, I'd like to learn of that. Maybe there's a way to do it in vbscript or powershell.

















GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Login

Explore
Messages
Mentions
Search