File naming and backup solution - Part 2
This is the second and final post around the file naming and backup solution that I, over a period of years have developed and now completely trust to look after not only my client, but also my personal photographs.
In the previous post we covered the extraction of images from multiple cameras to two locations, a main, working location on a PC, in my case with Adobe Lightroom, then a second, backup location, in my case a Synology NAS.
We covered how to rename your files and the file structure that I would recommend in order make the import work well in Lightroom as well as being able to easily take photographs on multiple cameras without having to add an additional processes into the system.
In this article we are going to focus on the Synology NAS and in particular the backup solution that I have developed to, I believe, provide the level of security that I require for personal and client photography.
My end to end solution is quite complex and hence I would suggest that you may like to just use some of the ideas within any solution you have rather than the full suite.
Backup Strategy Goals
When developing a backup strategy for you photography, it's key to understand the goals that you are trying to deliver against. In my case this has a number of elements
- No lose of data given single disk failure
- Quick recovery to a recent snapshot of data given multiple disk failures or NAS failure
- Ability to recover from multiple snapshots (quickly) given corrupt recent backups
- Zero cost long term (5 years +) backup for client photography with rapid recovery
- Minimal cost for lifetime backup strategy for personal photography
- Full automation
Before launching straight into the solution, it's worth understanding the hardware setup that I have that relates to my backup and recovery solution.
Always on PC
I'm running windows but this could be any OS with the ability to schedule jobs. This is used for many things including acting as a mail server, DHCP and DNS server
Synology DS1812+ with 1 volume
This is my primary NAS for the backup photographs saved down by Nikon Transfer, as well as a number of other uses
Synology DS1812+ with 2 volumes
This is my secondary NAS, used for backups of primary NAS as well as a number of other uses
No lose of data given single disk failure
This is really simple to achieve if you are using good NAS drives with 4 or more drives with each volume. Personally, I have 4 drives per volume and this is running under Synology’s own version of RAID5. This provides 1 disk of redundancy that can be swapped out and replaced if required; Synology supports hot swapping (you don’t have to turn the NAS off) but this isn’t critical as long as you can replace a drive and the NAS then repairs itself
Quick recovery to a recent snapshot of data given multiple disk failures or NAS failure
To achieve this, I replicate all my backup saved images from my primary NAS to my secondary NAS. Synology makes this easy by having a replicate function but equally you could do this manually by just copying your backup photo directories to a secondary location; ideally, a RAID5 NAS once again
Ability to recover from multiple snapshots (quickly) given corrupt recent backups
This I have applied to the finished, edited photographs rather than the backup RAW images. In my case I have a script running on my server that backs up the entire catalogue of processed images from my primary NAS to a my secondary NAS.
This script runs weekly, uses standard windows backup, and saves down the backup file as a unique file, importantly, not adding as a backup file into an existing catalogue as this results in two issues. Firstly, if the catalogue file is corrupted then you can loose all backups. Secondly, if you then want to cull your backups it’s very difficult to do whereas with each backup being a different file, you can just delete the files you don’t want.
Zero cost long term (5 years +) backup for client photography with rapid recovery
For this I use a free solution provided by a Symform.com In principle, you allocate some drive space for them to use as they wish and also tell them what you want backed up. Your data is compressed, divided up into very small amounts and saved out on other members drives that they have allocated space on.
I won’t go into detail but it’s very secure, doesn't cost anything other than some of your own spare drive space and is very quick to recover from. I have chosen this solution partly because of the integration into my primary NAS which means the whole backup process is automated. I can get the data back when I want without cost; the only drawback being you have to allocated twice the amount of space than you use. Hence, this is only for the important client work that I may need to get back quickly
Minimal cost for lifetime backup strategy for personal photography
Personal photography is critical to my family and is something that never want to loose, although, to be honest, if I do then I’m also in no hurry to get back as long as, over a period of time I can.
Amazon run a service called Glacier that delivers this very cheaply. Amazon, given the size and strength of the organisation has the longevity that I am looking for in relation to personal photographs. Whilst Symform, above, are good and I'm sure will be around for a few years it’s not the same pedigree of company and if anything happens then I can find a new offsite backup for client work; and to be honest, I don’t suppose that I’ll ever need to restore from it given the other backup solutions I have in place.
Personal photography, for me is different, I am looking for the most reliable solution possible and Amazon as an organisation, whilst not 100% guaranteed, I expect to be around in a few years yet. Synology have an Amazon Glacier app on their NAS; however, this was released some time after I started to use the system; I therefore have this running under a system called FastGlacier on my server that although isn't 100% automated just involves a 2 minute task once every few weeks. I could change at some stage but to be honest I'm happy with the way things are working at present
Amazon Glacier is NOT suitable as a backup solution I would suggest for data that you are ever likely to need back. This may sound strange, however, the pricing model is based around being very cheap to store data, but expensive to retrieve if you need it back quickly. It has been described as a WORN strategy (Write Once Read Never). However, you can recover a small percentage each month if required, so as long as you don't mind taking a long time, or paying, then this is a really good and cheap solution. Ideally, you'll never need to get the photographs back. To put this in context around the price, I have around 30,000 personal photographs backed up with Amazon for less that $2 / month
Crucially, all of the above, with the exception of FastGlacier requires no effort on my part to keep running. All synchronisation, backups and offsite Symform backups are 100% automated; which, in truth mean they happen as if left to me they most likely would be forgotten!
We've looked at a number of automated methods of keeping your data safe, from the type of drives and RAID configuration to full automated off site backup solutions. A lot of the functionality that I have described is based upon Synology's excellent NAS drives; however, a lot of the principles can be applied elsewhere on other NAS systems or even by using a PC for some of it.
I hope this review of the second step in my file management workflow has been of interest. If you found this helpful it would be appreciated to show your support by "Liking" my Facebook page TonyHalfordPhotography do so easily by clicking on the Like button, top right.
All the best!