Advice On Backup Regime Changes

Fozzie Bear · 23 Jan 2022

I have a 24x7x365 "Home Server" system which acts as a file server and Plex media server. I am using 6 x 2Tb Hdds and DriveBender as a storage aggregator to give a single 8Tb storage pool. DriveBender is a cross between software raid and Windows native Storage Spaces.

Long story short due to an email change I missed the alerts of a failing hard drive until it was too late and the drive died before it could be safely replaced. However all data such as Documents, Photos, Videos etc are backed up to Onedrive. (M Plex library movies, TV Shows, Music etc is backed up to another Plex Server). However on attempting to restore my missing data (which was only partial as the data is spread across all internal disks) I have seen a flaw in my backup strategy.
Currently I am using SyncToy to back-up nightly any new or changed files to a directory on separate 2 Tb Hard drive. This directory is the OneDrive root folder so the contents are backed up up to OneDrive.
Previously this appeared to work well and I am using the Echo option i.e
- New and updated (renamed and deleted) files are copied LEFT to RIGHT only
- Use this option if you want to backup data files one-way, from your computer to an external drive.
This means that any old files I no longer need and delete are deleted from the backup which prevents the backup from steadily growing with old unwanted files. However failure of the hard drive is equivalent of a deletion as far as Synctoy is concerned so it has deleted the lost files from the backup!

My best option is to change from Echo to Contribute i.e
- New and updated (renamed) files are copied LEFT to RIGHT only. Deleted files are not mirrored.
- Use this option if you want to backup data files one-way, with the exception of deleted files. as this would avoid the possibility of accidental deletion also deleting from the backup. The downside is that the space taken by the backup will continue to grow. I am nearly up to 940Gb of my 1Tb Onedrive allocation due to the number of home videos and family photos we have.

I have considered using another sync session that runs say once every month or even three months using Echo which would detect any "deliberate" deletions of files and remove these from the backup drive. I don't think this will be easy with Synctoy as all folder pairs are run at the same time so may have to switch to another option like FreeFileSync where you can use batch scripts to run different backups at different times. I have used Robocopy before but not very fluent in scripting different backup types and would prefer a GUI interface.

Any suggestions on a Backup Regime change to fulfil what I am trying to achieve would be appreciated including other software or ways of doing it. As an aside I also Macrium Reflect Free Edition installed for OS restore.

dalchina · 23 Jan 2022

Hi, first I'd image all drives regardless. The frequency and manner of doing so would be your choice.

Then I'd have e.g. Crystal Diskinfo (free) set to alert me against thresholds of disk degradation, running in the background.

The very act of imaging detects CRC errrors, so an failure to image is also an indication of a problem, and potential disk failure.

Then I'd have

a. scheduled data backup focussed on key data.

As with disk imaging, this can be done differentially, incrementally- options depend on the program used. Thus you would have multiple dates to which to restore both disks and backed up data.
b. synchronised backups for the fastest changing key data.
- but ideally without the issue you describe.

Set against all of that is the need to have greater security e.g. against power problems/ransomware etc- and be able to keep appropriate backups off line securely stored.

I don't have such complex needs as you have, but I use:
- periodic disk imaging (differential)
- Genie Timeline Pro - configured such that connecting my backup disk initiates a backup run of key data.
This is quite limited in my case - basic financial and family tree data.

I tried 3 or 4 programs before settling on that - others simply didn't run the backup when the disk was connected or didn't have that option. It also has an 'intelligent' mode which I don't fully understand from its description - somewhere between straight synching and scheduled. But that means the backup disk must be connected of course.

Note Genie Timeline says this- significant given your comments:

There are many other options of course, and it becomes a matter of choosing one that
a. does what you specifically want
b. you feel confident with
c. makes recovery easy

Just some outline thoughts- others have described their imaging routines.

Fozzie Bear · 23 Jan 2022

Many thanks Dalchina,
Firstly thank you for a very quick and detailed response to my dilemma.
Whilst I do not disagree with your suggestion to image each disk I dont think this will be possible when aggregated using the DriveBender Software. Windows and any program sees my Data drive as a single volume. I would have to remove each drive from the pool and give it a drive letter in order to image it. To be fair DriveBender should alert me of any errors with the disk well in time before failure but unfortunately I forgot to change the email address used when I ceased the old BT Yahoo emails.
I will look at Genie Timeline Pro as you suggest as an alternative to SyncToy. However its being able to differentiate between purposefully deleted files and files which are no longer there because the drive on which they resided is no longer available and so appears to SyncToy to have been deleted.
I am looking for a way to backup everything safely whilst being able to manage "official" deletes which avoid the backup size growing. One thing that FreeFileSync does which I have just noticed is it will detect moved files using the OS file ID. This avoids duplicates when a file is moved to a new location something I don't believe SyncToy has the functionality to do.
Thank you again for responding

dalchina · 23 Jan 2022

I tried FreeFileSync and really didn't like it. But as I said, it depends what suits you - a personal decision.
I also tried O&O's Autobackup I think it is and Aomei Backupper amongst others.

Note my screenshot added above - I'm not going to suggest Genie Timeline is necessarily the best or most appropriate for you.. but that preservation of data is what you would need to look for in any such program to avoid the loss of data.

I'd not heard of Drivebender before- sounds a bit like Storage Spaces.

My comments about continuous monitoring of disk status stand though - and are particularly applicable to MS's Storage Spaces from what I've seen.