Dumb Luck and Backing Up Your Images

Do as I Say, Not as I Did

Golden Corridor, Almost Lost

You have heard it many times before, but it can never be repeated too often. Back up your work! This week I will describe how I have attempted to back up my photographs, not as an example of best practice, but as a cautionary tale about what can go disastrously wrong and how dumb luck saved me. 

Several weeks ago I received an order for a copy of one of my favorite recent images. I had printed this image before so I knew that the file was in my edited images directory, ready to print. I found the file, but when I attempted to open it in Photoshop the computer whired for long minutes and then froze. My heart sank as the same thing happen on every file I tried to open from this directory. Nothing worked, and I realized that I had lost thousands of hours of work on more than 4000 of my best images, collected over more than a decade.




During over ten years of more serious New England photography

CD Backup

and especially since digital imagery has come to dominate my work, I have taken various approaches to backing up my photographs. In the early years, the image files were so small that I could fit several months on a single CD. From 2001 until early 2008, I felt secure keeping one set of CDs at home and a second copy in my Keene, New Hampshire office, but as my cameras went from 2 to 6 to 12 and then 21 mega pixels, this solution was no longer practical. Today I would require several DVDs to backup each month's work. This October I captured over 1500 autumn images totaling 32 Meg's, which would fill 7 DVDs. With my ballooning backup requirements I have become shamefully sloppy in my "strategy", with files scattered across several hard drives both on internal and external drives. My first PC had a single 120meg hard drive which at the time seemed liked “more storage than I would ever need”. Now I have a total of 32 Terabytes of capacity spread over 15 drives, and yet it seems I am constantly running out of space. I still organize the images by month, but I had some directories backed up on multiple drives and others hanging precariously on just one fallible drive. I was having nightmares about the fact that EVERY physical drive fails eventually. I knew it was an inevitability NOT a possibility. I had to do something and I knew that the best protection was in redundancy.

Tiers of Redundancy

Save the Drobos First
The first measure was to improve the reliability of the storage.

Desktop Backup

Redundancy is the key. One part of my approach was to move my critical data to Drobos. Drobos are redundant, pooled backup drives which contain multiple drives, making it possible for a drive to fail without losing the data. When failures occur, the system automatically compensates and distributes the data across the remaining drives. I now have two Drobo units each with four hard drives adding up to a total of 20 terabytes, but no matter how reliable, any drive or array of drives can fail. The second tier of protective redundancy is to backup to more than one physical drive.

My current backup workflow starts with translating my camera's Canon proprietary raw files to Adobe’s open source DNG format. I like DNG format since it is nonproprietary and because the image meta data is stored within the image file and not on those annoying sidecar XMP files. I use the DNGs for editing and archiving, but before I do anything else, I save the original Canon raw files to a separate drive as a backup. As I work on my edited files, they are stored in a separate drive as well. This leaves me with two copies of all of my raw images, but only one full copy of my finished, edited images. I figured that if I lost my edited images, I would still have the original raw files. I didn’t allow myself to think about the potential loss of my massive investment in photo editing. Stupid! And deep down I knew it.

I also knew that a fire, flood or tornado could still wipe out all of

partridgebrookreflections.com : archive

my images. In case of fire my evacuation priorities are: 1) Drobos, 2) Dog, 3) Wife, but it is still dangerous to have all my images in one location. Recently I have been investigating approaches to “off-site” backup as my third tier of protection. There are a number
of good “cloud based” services (eg Back Blaze, Carbonite and Crash Plan) available to archive my files, but given my massive amount of data, it would take months to upload all my images over the Internet. I thought that If I could backup my current archive, I could then consider using a automatic cloud solution for my future work. In the meantime, I have started backing up my new, full resolution, edited images to my new Zenfolio Website.








I eventually decided to start by creating my own off-site backup

Bob's Closet Archive

and here is where my dumb luck kicked in. After stalling for months, I copied my RAW directories and my precious collection of edited images to a 3TB external drive. My plan was to store this archive drive in my neighbor’s house, protecting me from fire but not necessarily from tornado or nuclear blast. It took more than a day to copy over the files. As planned I squirreled away the drive in Bob’s closet. I slept better that night, but I had no idea how soon that backup would become critical. 


Disaster
Just two weeks later, as if it knew it was safe to retire, my edited
image drive failed. Suddenly I could see the images in the directory but, despite all efforts at repair, I couldn’t get them to open. If I had not created that backup drive I would have lost years of work on all my most popular images. I still had all the original RAW files, but

4000 files, 21 hours, priceless

the number of hours of lost image editing would have been incalculable and devastating. After all my sloppy delay in creating a backup, I definitely did not deserved to be saved, but saved I was. I retrieved my drive and reloaded the 1.8 TB of irreplaceable images. All I can say is that there are a lot worse ways to learn a few lessons. The lesions are nothing that we don’t already know but they are worth repeating, endlessly.

Lesions

• Every machine fails eventually, and that includes the best hard drive or array of drives.
• Images stored only on one drive will eventually be lost.
• Memory is cheap and getting cheaper all the time.
  
• An image does not exist until it is in two places, preferably three, with one off site.
• An automatic solution is better than one that requires regular thought.
  
• Regular thought NEVER happens.

My backup strategy is not perfect. I'm still working on the cloud solution, and the automation of my backup process, but the important thing is to have a plan and follow through. Many of you must have better solutions, but for those who haven't established a redundant backup plan, do it today.  It takes time, but no amount of time or effort can retrieve your precious lost images. You can't always rely on dumb luck. 

Oh, and the correct order is 1) Drobos, 2) Dog, 3) iPad and then 4) Wife.