Bug report: Enabling encryption mid-set not properly handled


Author
Message
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)
Group: Forum Members
Posts: 3.5K, Visits: 26K
UPDATE: As of 7.1.3147, the following behaviors trigger an automatic Full backup even if an Inc/Diff was specified:
- Enabling encryption when it was not enabled before
- Disabling encryption when it was enabled before
- Changing the key length (as long as the password is not changed)
(Changing the password when encryption is enabled causes subsequent Inc/Diff backups to fail.)

Steps to reproduce:

1. Create a definition file for an F&F backup (this also applies to image backups, but F&F is easier for testing). Leave encryption disabled.
2. Capture a Full backup.
3. Edit the definition file to enable encryption; I used 128-bit if it matters.
4. Modify the source data set in some way, then choose to capture an Inc or Diff.

Expected behavior
: Reflect would automatically run a Full instead since I wouldn't have thought it possible/desirable to append an encrypted backup onto an unencrypted set.
Actual behavior: Reflect creates an Inc/Diff, and although the job log says that encryption is enabled, the backup is created unencrypted, as confirmed by being able to restore from this backup without entering a password.

This bug seems especially dangerous for users employing the Incrementals Forever strategy since it could conceivably cause their backups never to be encrypted even though the log indicates that they are.  If the steps above are reversed, i.e. encryption is enabled for the Full and then disabled before capturing an Incremental, then the Incremental job completely fails with an error about an incorrect password rather than automatically falling back to a Full, although I can see a case for the current behavior in that scenario.

Additional unexpected case:
1. Create a definition file for an F&F backup with encryption enabled.  Set the retention policy to 1 Full backup and leave the default "matching backups" setting.
2. Capture a Full backup.
3. Edit the definition file to disable encryption.
4. Capture a new Full.

Expected behavior
: Reflect would not purge the old backup since its encryption and the fact that the definition file no longer contained the password would prevent Reflect from determining the old backup's contents and therefore whether it is a matching backup.
Actual behavior: Reflect purges the original encrypted backup, indicating that even encrypted backups have enough data stored unencrypted that Reflect can determine whether they match a new unencrypted backup.  Is that expected/appropriate?  What data is stored always unencrypted in both image and F&F backups to facilitate this?  Partition layout for the former and path(s) of the root folder(s) for the latter?  Partition layout seems innocuous enough, but folder paths may contain sensitive information.

Edited 17 April 2018 6:28 PM by jphughan
Nick
Nick
Macrium Representative
Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)
Group: Administrators
Posts: 1.5K, Visits: 8.5K
jphughan - 13 March 2018 6:17 PM
Steps to reproduce:
1. Create a definition file for an F&F backup (this also applies to image backups, but F&F is easier for testing). Leave encryption disabled.
2. Capture a Full backup.
3. Edit the definition file to enable encryption; I used 128-bit if it matters.
4. Modify the source data set in some way, then choose to capture an Inc or Diff.

Expected behavior
: Reflect would automatically run a Full instead since I wouldn't have thought it possible/desirable to append an encrypted backup onto an unencrypted set.
Actual behavior: Reflect creates an Inc/Diff, and although the job log says that encryption is enabled, the backup is created unencrypted, as confirmed by being able to restore from this backup without entering a password.

This bug seems especially dangerous for users employing the Incrementals Forever strategy since it could conceivably cause their backups never to be encrypted even though the log indicates that they are.  If the steps above are reversed, i.e. encryption is enabled for the Full and then disabled before capturing an Incremental, then the Incremental job completely fails with an error about an incorrect password rather than automatically falling back to a Full, although I can see a case for the current behavior in that scenario.

Additional unexpected case:
1. Create a definition file for an F&F backup with encryption enabled.  Set the retention policy to 1 Full backup and leave the default "matching backups" setting.
2. Capture a Full backup.
3. Edit the definition file to disable encryption.
4. Capture a new Full.

Expected behavior
: Reflect would not purge the old backup since its encryption and the fact that the definition file no longer contained the password would prevent Reflect from determining the old backup's contents and therefore whether it is a matching backup.
Actual behavior: Reflect purges the original encrypted backup, indicating that even encrypted backups have enough data stored unencrypted that Reflect can determine whether they match a new unencrypted backup.  Is that expected/appropriate?  What data is stored always unencrypted in both image and F&F backups to facilitate this?  Partition layout for the former and path(s) of the root folder(s) for the latter?  Partition layout seems innocuous enough, but folder paths may contain sensitive information.

Thanks for your post.

Encryption is not part of the algorithm to determine what is matched when matching backup sets for purging. Although the file system data inside the data blocks is fully encrypted, the meta data that describes the filters and root folders is not encrypted. It may be considered that the folders chosen to backup is a security risk, although this certainly hasn't been raised as a cause for concern in the past. Actually this is more secure that Windows Encrypted File System as that present all folders and file names in the clear not just root folder names. 

Both the encryption and compression settings are only effective when creating the next Full backup. When appending to an existing backup set the existing password is required, so changing a password mid backup set will cause a failure. Creating a new Full in this scenario would indeed be more helpful, and we'll look at changing this behaviour. 

The log entry showing the backup definition setting for encryption, not the backup set setting is indeed a bug. Thanks for bringing this to our attention, it will be fixed in a future update, although this won't be an issue if we change to creating a new backup set if encryption changes. 

Thanks again

Kind Regards

Nick - Macrium Support

Edited 13 March 2018 9:35 PM by Nick
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)
Group: Forum Members
Posts: 3.5K, Visits: 26K
Thanks Nick!  I would recommend that a new backup set be created if encryption is enabled and the existing backups are unencrypted.  Just updating the log to say, "Encryption specified but not enabled because the existing set is unencrypted" would certainly be better than the current state -- and I would argue that would be the best choice for handling changes to the compression setting -- but if the user never looked at those log entries, they could still end up without encrypted backups they thought they were getting, potentially forever in the Incrementals Forever case.  I feel the mind set should be, "If the definition file specifies encryption, then under no circumstances should Reflect generate an unencrypted backup."

For cases where the encryption simply changes (as opposed to being newly introduced), I can see the virtue of both responses.  Automatically creating a new Full would be convenient for cases where the password change was deliberate.  However, if the password change was the result of accidental or malicious direct modification of the XML file, then automatically creating a new Full would mean Reflect would now be creating backups that the user would not be able to decrypt later, which the user might not discover until they had to restore something.  The current failure behavior instead potentially gives them a warning about this problem upfront, although not if the first backup after the accidental/malicious change was a Full, I suppose, or even in the case above where encryption is newly added.  If MIG's protection against modification/deletion by non-Macrium applications were extended to definition files, I would say that creating a new Full automatically would be the way to go.

Edited 13 March 2018 10:26 PM by jphughan
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)
Group: Forum Members
Posts: 3.5K, Visits: 26K
Reflect 7.1.3147 fixes the main cause for concern reported here, but there are a few related behaviors from the discussion above that persist when creating Inc/Diff backups:

- Changing a password when no encryption is enabled causes Reflect to continue using the set's original password. This seems somewhat hazardous, even though lack of encryption would technically make the data recoverable regardless of whether the correct password is known.
- Adding or removing a password when no encryption is enabled and then running an Inc/Diff causes Reflect to display the new definition file setting, not the set setting that will actually apply to the generated file.
- Changing the compression setting results in the same discrepancy noted above.

Nick
Nick
Macrium Representative
Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)
Group: Administrators
Posts: 1.5K, Visits: 8.5K
jphughan - 17 April 2018 6:44 PM
Reflect 7.1.3147 fixes the main cause for concern reported here, but there are a few related behaviors from the discussion above that persist when creating Inc/Diff backups:

- Changing a password when no encryption is enabled causes Reflect to continue using the set's original password. This seems somewhat hazardous, even though lack of encryption would technically make the data recoverable regardless of whether the correct password is known.
- Adding or removing a password when no encryption is enabled and then running an Inc/Diff causes Reflect to display the new definition file setting, not the set setting that will actually apply to the generated file.
- Changing the compression setting results in the same discrepancy noted above.

Thanks for the feedback. We decided that creating a new backup set only if the encryption type was changed. Changing on password could be deemed dangerous as was pointed out....

..... However, if the password change was the result of accidental or malicious direct modification of the XML file, then automatically creating a new Full would mean Reflect would now be creating backups that the user would not be able to decrypt later, which the user might not discover until they had to restore something.


So, we just addressed the situation where the encryption type was changed. Compression change wasn't part of this modification. Most users would be unaware that a new backup set would be required for a change of either compression or encryption, but changing the encryption type would warrant a forced new backup set. 

Adding or removing a password when no encryption is enabled and then running an Inc/Diff causes Reflect to display the new definition file setting, not the set setting that will actually apply to the generated file.
- Changing the compression setting results in the same discrepancy noted above..


Apologies, these bugs have yet to be addressed.

Kind Regards

Nick - Macrium Support

Edited 17 April 2018 7:24 PM by Nick
jphughan
jphughan
Macrium Evangelist
Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)Macrium Evangelist (5K reputation)
Group: Forum Members
Posts: 3.5K, Visits: 26K
Ok thanks. I’ve gone back and forth on that thought about the malicious modification. The flip side I had in mind when I wrote that new post was:

- User runs Incrementals Forever w/ Synthetic Fulls.
- User changes their password deliberately for whatever reason.
- User subsequently forgets/destroys the record of their original password, perhaps long after the backups created under the original password had been merged out, on the assumption that the original password would no longer ever be needed — except the original password is still actually in use in the set.

But I guess sometimes there’s no design that solves for all problems, and I grant that a user running a password with no encryption here in 2018 is unlikely on its own, never mind doing that AND using Incrementals Forever.
Edited 17 April 2018 7:19 PM by jphughan
Nick
Nick
Macrium Representative
Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)Macrium Representative (2.7K reputation)
Group: Administrators
Posts: 1.5K, Visits: 8.5K
jphughan - 17 April 2018 7:17 PM
Ok thanks. I’ve gone back and forth on that thought about the malicious modification. The flip side I had in mind when I wrote that new post was:

- User runs Incrementals Forever w/ Synthetic Fulls.
- User changes their password deliberately for whatever reason.
- User subsequently forgets/destroys the record of their original password, perhaps long after the backups created under the original password had been merged out, on the assumption that the original password would no longer ever be needed — except the original password is still actually in use in the set.

But I guess sometimes there’s no design that solves for all problems, and I grant that a user running a password with no encryption here in 2018 is unlikely on its own, never mind doing that AND using Incrementals Forever.

You are absolutely correct and this feature change/bug fix is ongoing. We implemented this current change for an explicit  requirement in Site Manager and will extend in a future update.  As always, thanks for the great feedback.

Kind Regards

Nick - Macrium Support

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Reading This Topic

Login

Explore
Messages
Mentions
Search