I've been made aware over the years that many of you are very, very organized and that your back-up strategy for your digital files is well nigh flawless. I'm proud of you all and wish I could hand out gold stars personally. I am embarrassed to admit that my back-up strategy has more or less fallen apart and I depend more on endless duplication across an aging brigade of weathered, external hard drives that range from ancient, one terabyte (original) firewire and USB-1 drives, to little bus-powered "pocket" drives, all the way up to nearly modern, USB-3, four Terabyte drives.
Nearly all of them have names and most of them have swatches of white tape on them to label them both by name and by the year they were first introduced to service on my jangly and jumble-wired desktop. Some of the dates on them are starting to become....embarrassing.
In photographic caveman days my back-up strategy for client photographic files was pretty straightforward; at the beginning of the year I'd head to Office Depot and get a big binder that held a couple hundred CDs or DVDs. When I finished a job I'd make two copies of the material (raw files and finished files X2) on DVD's and put them into that year's binder. The binder would sit on the Metro shelving in the studio, along with last year's binder, and the year previous to that. I would also have a copy of the files on two different hard drives. But, when budgets were tight I'd only splurge on one new hard drive at a time, and only when one of the older ones either crapped out got too full.
At some point about six or seven years ago camera files got much bigger, I got more indiscriminate in my shooting style (more is better?), and I tossed fewer non-selects (because editing takes time....). Since a job shot with a camera generating 42 or 50 megapixel raw files takes a bunch more space than the old 12 megapixel, steam-powered camera files it became more and more impractical to try sitting around burning multiple multiples of DVDs for each project. And hard drives started filling up quicker as well.
Now I just keep a DVD drive around for the times when I need or want to access files stored and filed on DVDs. Or to impress children who have only ever known about streaming media. Nothing gets burned anymore because the first hour of shooting, in many projects, would fill up a couple of disks and time is money. Or, at least, billable time is money.
My current methods are these: Before I shoot a project I let the client know what my workflow is. I tell them that they'll receive final files and that they are responsible for storing, archiving and protecting these files for future use. Not me. I'll let them know that I'll try to keep the files safe but since I don't personally own a hard drive company there's no way I can guarantee with any certainty that the files will always survive and be available a decade later. In the last year I started letting clients know, in advance, that we only agree to try to keep the files safely stored for one calendar year. After that the availability of the files is totally up to them. This isn't a film world anymore. Everything moves faster. Images have a shorter shelf life and I have fewer opportunities to re-sell or re-license images.
Once I do the job for a client I ingest the images into my system via Lightroom and have Lightroom write the files (pared down keepers) to two different hard drives. I work on the files mostly in Lightroom and output them as highest resolution, full size Jpegs. If something needs work in PhotoShop I do that and then put the resulting files (as Jpegs) in to the Jpeg folder. Then, all of these Jpeg files get uploaded to Smugmug.com and put into individual galleries. The galleries are nested into logically named and sorted folders.
The clients get access to the galleries, and, if I've contracted to allow them to use the images for a long time, across many media, then they also get a download password that allows them to endlessly pull big, nice, color corrected Jpegs right off their gallery. I could be out of the office for a year and as long as I keep paying Smugmug.com for their service the clients will be able to access their galleries. I currently have about half a million images in folders and galleries on the site and the oldest folder is from 2004. It still works. It's still active and accessible.
The real reason I adopted this strategy was the dropping price and increasing speed of broadband web access. I can finish a job with 10 or 20 Gb of files, hit the upload screen, go for coffee, and chances are the entire gallery will be up and ready by the time I've finished bullshitting with friends and acquaintances at the coffee shop. It's nice to know that the files are stored off site and, in all probability, a local lightning storm or a random meteor strike that wipes out my desktop system won't wipe out my image inventory of client work. I also keep many galleries of family stuff there as well...
But....but...I still WANT to have those files saved as raws and Jpegs in a local storage device as well. Because....I think that's what we've been led to believe... So, I have a current collection of eight hard drives on the desk and a wide filing cabinet drawer with another dozen or so older drives sitting right next to the desk (don't know how I'm going to access a couple of the oldest SCSI drives....but three or four times a year I take a day to spin everything up).
When I tallied the active desktop storage selection recently most of them were filling up fast and many of them were getting up there in HD years (which are even quicker than dog years). My oldest current desktop veteran is a 1 TB drive that was put into service one year short of a decade ago. Its demise is inevitable. At least that's what the digi-savants tell me.
So, when I upgraded to the new iMac Pro I decided to also start the year out fresh by bringing in two, new hard drives; almost the same way in which would go out and buy a new binder each year. Since the files from the two Panasonic Lumix S1R cameras aren't going on a diet any time soon I decided I needed to get bigger drives this time. I bought two 10 terabyte drives which I have earmarked for all 2020 files on the desktop. Each folder of files is duplicated across both hard drives. Seems like a good plan, especially in concert with my ongoing uploads to Smugmug.com.
I formatted both new drives today and then went in searching for all the quasi mission critical files that sat on the oldest drive. I thought I could copy them over to the new drives, just in case... you know....entropy. When I plowed through the content on the oldest disk I realized how much sheer crap I save on my disks. There were folders filled with raw files of obsolete products from companies that had long since gone bankrupt and exited the market. Those didn't need to be transferred, they needed to be dragged to the trash. When I finally edited down to the "must have" keepers on that drive I ended up transferring less than 50 GBs of work files.
Now I'm plowing through each additional legacy drive to see just how much absolutely worthless crap I've been paying to keep around on my desktop for all these years. I'm learning that so much of what I've shot over the last decade is filler and exploratory stuff but not much that's valuable and dear. I should have learned this a year and a half ago when I had to clean out my parents' house of 40 years worth of stuff that needed to make the trip to the trash can long ago.
Obviously, the two new 10 terabyte drives are NOT SSDs. Not even Warren Buffet can justify the cost of multiple 10 terabyte SSD drives used just for storage and occasional access. But they are USB 3 drives and those are much faster and less doggy than the USB-2 drives and the 2's and the 3's run circles around the very old, original USB-1 drive.
I do have a couple of external 256 GB SSDs and I use them a lot now to load full of folders and work on files in concert with the SSD drive in the new computer. It's like working at hyperspeed compared to my old ways. I guess keeping up with some technology is prudent. It sure is nice to finish projects in one quarter the time we spent just last year. Go 2020!!!
(just above) a mission critical part of our workflow philosophy revolves around the access to and utilization of coffee 3.0. Made fresh with steamed milk and delivered in an enclosure that retards thermal decay. It doesn't make the machines run faster, it only enhances my typing speed and accuracy...
Ben and Studio Dog visiting relatives at Christmas. The uploading of family images to our online storage sure makes it easier to find photos I want to share with family and friends. Can't imagine now having to dig into a cardboard box to find a folder full of black and white negatives and then heading into the darkroom to print up a couple 5x7's for someone. Not for free, at any rate. And, I'd have to build a new darkroom. Never going to happen. Not while these disks keep working.
Next up, on my agenda, is to revisit printing my own stuff here. Stay tuned? Or largely ignore?
A. Molitor: How's the LED Light search going?
a caution to readers: Please don't tell me how I can save thousands and thousands of dollars by building my own hard drives or by building my own computers from scratch. I tried that with a Ferrari kit once and it was a f---ing disaster and blew up in the driveway...... YMMV.
25 comments:
What about a raid array? Like you I have a number of dusty cd folders with old client work. But a few years ago the studio moved over to dial hard drive raid arrays that work wirelessly over the network. So we back up and they duplicate each other.
Have you considered or used them?
Not yet. I'm more interested in encouraging clients to think of their images as very short rentals which might not be available down the road. Raids are, to me, tricky business. Wireless ones even more so.
Kirk, please consider a more comprehensive backup plan. I won't go on about the virtues of a Raid system, but do consider having a backup drive for the other drives that you can keep offsite most of the time. I try to backup my Raid system monthly and keep its backup in another location. In addition, I use crashplan to maintain a backup of the Raid system in the Cloud. I backup to the Cloud automatically, each night.
As you said, storage is cheap.
David, tell me exactly why. Thanks! Kirk
Hi Kirk
I give the clients a year of storrage to. But on top of that i keep all fotos in google photos compressed with googles compression. that way i can always send a link to a clients photos even a decade later. its compressed 16mp files i know. But the bride still feels she gets a hell of a service mailing me to adresse se lost the pictures and 10 min later she receives a link with the pictures. Experienced it last week wit a wedding form 2012. and i bet she dont care about the compression.
Many times i even deliver the ´photos this way, unless the clients ask for better files.
All my family photos i backed up this way to.
I realise your clients demand better quality than mine most of the time. But i recken some of yours would be fine with this to. But 99,9% of the clients that come back a decade later would be happy with this offering
Its free and unlimited storage and its been flawless for over a decade. Compression changed from 12mp to 16mp, 6 years ago. but other than that i have not noticed any hickups.
cheers Mads
Every small businessman in the world is now their own data base administrator, systems administrator, disaster planner, software upgrader, physical plant maintainer. It's a miracle they have any time left to do any work.
I'd be interested in a survey of wedding photographers that compared the long-term costs of maintaining backups to the income derived from sales of photos one year after the weddings. And I'm willing to throw out all the couples who got divorced, just add up the downstream profits from the couples who are still married.
Doesn't this, in part, make the case for handing over all the raw files to the client? I know this rubs everyone the wrong way, but there are surely many cases where photos have very little future value, if any, while they represent obvious future liabilities.
Backup to "clouds" make me nervous. Basically there is no cloud, it's just someone else's computer. What's the world's backup to Google? Sooner or later they'll hire a wacko CEO who will wreck the place. It has happened to lots of other companies, none of them are immune.
It is recommended that source code, for instance, be backed up in two locations to prevent loss in case of fire or some other disaster. If some of the images warrant that kind of protection you could have one drive in your studio and one offsite. (your home)
I actually back up to two clouds for more security in direction an organization will take. Yes, my system is redundant, but I can sleep at night and also not worry so much when I am away from my home office.
I think that unless you are in the business of selling code that the idea of off site back up it way over the top. A modality "borrowed" from other industries which were totally dependent on data for the financial health of their companies. I would ask everyone, how did we do "redundant" back-ups in the days of film? How did we split original film and negatives up to get off site storage? How did we put the same film in two different places, for safety?
Yes, hard drives die. So you have two. Yes, even the second might die so you back up to the cloud. At a certain point you have to pause and realize that this behavior could go on ad infinitum and that all of your time and money would then be 100% dedicated to backing up files that almost all have rapidly deteriorating value....
When I drive to important, time sensitive jobs I don't tow a second car behind me just in case the first one dies. When would I stop? Would it be at towing one "back-up" car? or two? or three? Do your maintenance and get on with your lives...
This post made me smile.
I have a carton of old hard drives in my basement. As files got bigger, and both my wife and I saved more of them, the drives would fill up. So, I would buy new ones, usually several times larger, but no more expensive, copy a mirror image of the old one to the new one, label the old one, and stow it into the "archives," aka, cardboard carton. At some future time, I'll probably drill a couple of holes in each one to ruin the surface, pass a strong magnet over each, and send them to electronic recycling.
Backups occur nightly, automatically, to other drives, and every so often, when I think of it, I'll copy the backups to another drive, and store it elsewhere. Maybe in the future I'll do cloud backup instead, just for convenience, because it is a pain the butt to do this stuff, when other things are much more interesting to do. Also, I do not try to "clean up" the drives. Too much time, too little benefit. Drives are cheap. Time ain't.
You need to apply some common sense to your backup strategy, otherwise you can drive yourself nuts. I like your approach. It fits your situation.
Oh, and for context: before retirement, I was an IT manager.
Thanks MikeR, at last some vindication! Based on enjoyment rather than slavish I.T. regimens...
Your backup strategy sounds fine to me. As you noted, you can have backups for the backups for the backups, ad infinitum. Where do you draw the line? I think you have drawn it well.
I also like that your clients have to accept some responsibility for preserving the digital assets!
DavidB
I've learned, over the years, that if I keep my eyes glued to the second hand of my wristwatch I don't get much of anything done. Too much attention paid to details is just as wasteful as too little.
DavidB, Thanks for your thumbs up. And shouldn't we all be having that conversation with clients?
Looks like LLoyd Chambers just survived a data loss, his story:
https://diglloyd.com/index.html
Just read the Lloyd Chamber's blog you sent along. Too much complexity overwhelmed and destroyed his day. Old, slow, manual tech means never letting "multiple blades" do their own thing and then finding out about it only when the whole system (tightly interwoven) kisses chaos on the lips.
And that's why we don't use "0" raids for our critical archives.
Like everything in our business lives, our back up strategy should be working for us and not we for it. Yours seems to be fine Kirk.
The only thing I'm careful about is to avoid having my backups permanently attached. I think it was Seth Resnick who related how all the passwords on the drives connected to his Mac were reset, maybe by a power spike. Lost the lot. Fortunately there was one backup not attached.
Last year the PSU on my Windows machine failed and fried every attached drive, including the DVD writer. I lost about 4 hours work. My backup strategy seems good enough.
Love the SmugMug solution. They are the experts and have proven it over the years. I have 2 copies locally, a copy on Flickr + BackBlaze & I think I’m good, too.
Not to be lecturing you but: You don't tow another car because you can always get another car that is just about the same. Not true with your photographs. If you have any "irreplaceable" photographs on your system, then you should have an off-site backup. Your house is right next door to your studio; if one goes, for whatever reason, the other will probably go as well. Your clients are adults and they've been told they're on their own. Fine. But if you've got a group of portfolio photos and another group of personal photos that you'd hate - really hate - to lose, then you should keep those folders updated and every so often copy them to another drive and give it to someone else to store at her/his place. (Maybe the someone else is a fellow photographer for whom you do the same.) Every so often, you make another backup, have coffee together, and switch. Low cost, low stress, not overdoing it but you never know...
Good advice Greg Heins. Most of my favorite stuff still exists on film... But, as far as the digital stuff goes... yes, the ART and the FAMILY images are backed up in lots of places. The old shots of circuit boards for a Motorola divisions that's been sold three times and moved on from whatever technology is in the photos? That stuff goes in the virtual trash all the time...
My 100 favorite photos (by Kirk Tuck)? They exist not only as digital files, scans from film, etc. but also as nicely crafted prints disseminated across the universe...
Thanks.
Thanks Kirk I'm using also SmugMug for client files since one year, works great.
My image files are on one big hard disc in the workstation and get synchronized manually via the open source software FreeFileSync (Win, Mac, Linux) to an external hard disc (USB3 or eSata). The software has also a RealTimeSync function for automated synchronization to a second internal hard disc.
Link: https://freefilesync.org/manual.php
Might take a look at Diglloyd.com and see how much he has LOST using his Top of the Line Uber Expensive archiving just this past week or so.
He has stuff backed up elsewhere as well - and lucky for it as his latest and greatest MAC has wiped out all he had hooked up to it.
Some of us still save in a few places and the "Select" images we also put on BluRay diskc - just in case.
I use those WD Elements HDs too, great value.
Interesting comments, there are so many options.
For the solo user, if the backup strategy isn't easy to do, it won't happen
Enough has been said about your backup strategy, but I wanted to mention that your choice of the Hydro Flask coffee mug as a thermal retention strategy is an excellent one.
Is that a thumb/finger in front of the lens ?
I thought that was just me that did that....
Tom, I love the Hydro Flask stuff. People keep giving it to me as gifts. Some for the car, some of the office, etc. They are very nice.
Post a Comment
We Moderate Comments, Yours might not appear right after you hit return. Be patient; I'm usually pretty quick on getting comments up there. Try not to hit return again and again.... If you disagree with something I've written please do so civilly. Be nice or see your comments fly into the void. Anonymous posters are not given special privileges or dispensation. If technology alone requires you to be anonymous your comments will likely pass through moderation if you "sign" them. A new note: Don't tell me how to write or how to blog! I can't make you comment but I don't want to wade through spam!
Note: Only a member of this blog may post a comment.