Just received a great question from Danny regarding online data storage.
Danny writes:
Mike…
Always appreciate and enjoy your newsletters. Do you use and/or recommend/discourage any of the online backup/restore services like Carbonite or Mosy? I gotta do “something” beyond my on-site hard drive mirroring using external drives. Want to cover my assets in case of fire or theft.
I’d appreciate your thoughts when you get a few moments.
Thanks,
Danny
My response:
Hi Danny –
I don’t use the online backup places for one significant reason: the amount of time it takes to upload/download data. Even over the fastest internet connections, transferring 1TB of data can take multiple hours and days. If my on-site disk drive fails it usually happens when I need the data the most. These things always happen when you are finishing a project for a client or trying to meet a deadline. Remember, Murphy’s Law is always in effect! Because of this, I don’t want to wait days (or even hours) to download my backup information.
I think the best solution is to create an off-site drive that is a clone of your on-site hard drive. Bring your off-site drive into the office once per week for data synchronization. Then, move it off-site immediately for safe keeping.
If disaster strikes my on-site drive, then I can quickly go grab the off-site backup and be back up and running in a matter of one hour.
Best regards,
Mike
Mike: The scenario you paint is not actually the way most people experience online backup. Yes, if you have a lot of data the initial backup can take days or even weeks. But after that, backups are incremental and are constantly running in the background. So your backups are always much more up-to-date than any kind of batch backup, such as your suggestion of doing a weekly backup and then taking the external drive to some off-site location. Plus, what a pain in the neck that is. As for the intial backup period, most of the time we’re talking about people who haven’t backed up properly in months, if ever. So what’s one more week? After that, you never have to think about backup again — it’s just always working in the background. That’s why Carbonite and our competitors are growing like crazy. People just love the set-and-forget experience. It’s like buying an insurance policy for your data.
Regards,
Dave Friend, CEO
Carbonite, Inc.
http://www.carbonite.com
Hi David –
Thanks for writing in. I greatly appreciate it!
I fully agree with you that backing up data from my office computer to Carbonite isn’t a big issue. These files are typically small and I don’t have to worry about file transfer times since I can upload/recover in a matter of a few hours.
However, it is the other side of the equation that makes it hard for digital photographers to use an online backup. Many digital photographers require disk space in the 1 TB – 10 TB range and all of this data is typically stored on external hard drives, RAIDs, JBODs or similar products. These days, photographers are also incorporating HD video into our professional work and that creates data with massive file size very quickly. Carbonite’s website mentions that its service is good for backing up 1GB, 10GB or more. It also mentions that if you are trying to back up any more than a few dozen GB of data, then it might not make sense because of relatively slow DSL/Cable internet connection speeds (http://www.carbonite.com/how_it_works/).
If my hard drives fail, then recovering this massive amount of data over an internet connection can take multiple days or even weeks depending on what type of internet service you have. Recovering my data from a hard drive stored at an off-site storage location will take a few minutes for me to drive there and pick it up.
Another point is that a single one-day photo shoot for a client will generate 4 GB – 20 GB in data that needs to be saved, backed up and archived within a few hours of the shoot in order to maintain data integrity. It is very common for me to come back from a trip with 100GB of photo/video data that quickly needs to be saved/archived. Again, uploading 4 ~ 100 GB to an online backup service takes an amazingly long period of time. Alternatively, backing up this data using eSATA or Firewire 800 drives will take minutes to a few hours.
Next, a professional photographer’s digital asset management (DAM) system requires that we can confirm data integrity at all times. What I mean is, I want to be able to confirm that my files exist on my archive drives. The only way I can confirm they exist with an online storage product is by downloading them from the service. With my own drives, I can quickly do checksums and local (i.e. fast) integrity tests with my DAM software.
The last thing I’ll mention is professional photographers have had a few large, mainstream data backup companies go belly up on them. The most recent example was Digital Rail Road. Many professional photographers used DRR to store their photo archives. When DRR went bankrupt, thousands of photographers lost their files and weren’t given any recourse for recovery.
Because of all these issues, I still recommend storing all files on local hard drives and then archiving at an off-site location.
I’m extremely interested in hearing your response to the unique needs of professional photographers. We are all looking for good solutions for our DAM systems and can use a partner out there who is specifically tuned to our needs.
Best regards,
Mike Hagen
Mike: I agree with you completely. While we do have a few customers who have managed to back up over 1TB, it is a very slow process — too slow, IMO. There’s no way to get the bits down the wire any faster.
I have a professional photographer friend who does coffee table books. Like you, he has tens of thousands of pictures that he stores on an array of large external drives. However, out of all these photos, there are only a few hundred that make it into his books, and those he backs up online — a total of about 17GB. His studio is, in theory, fireproof, so he’s pretty comfortable keeping most of his inventory locally. And a couple of times per year he makes copies that he stores in a fireproof vault. But there’s no way he can afford to lose the photos that he uses in his books, and so he keeps those online. He also likes the fact that his publisher can access those photos through a web browser.
As for doing checksums, it’s a very good idea because all drives create a certain number of bit errors. You are probably not aware of this, but Carbonite does this task in the background automatically, about once a month. We actually do a checksum of every file in our backup against the actual file on your disk. Because we store your files on highly redundant RAID6 arrays, nearly all of the bit errors we find are on our customers’ computers, not in the backups.
Dave Friend, CEO
Carbonite, Inc.
http://www.carbonite.com
Why not use both!
I use Mozy.com and it does the checksums…sure it took 6 months to get 700 gigs up to them….but when the heck! They said it was 5 bucks a month…eat all you can…I am!
I only backed up small sets at a time and then would add another set and another set and another set. After all the sets were done…I marked the entire drive.
Then I use SyncBack Freeware to echo all the changes on the drive to an external drive. One day I will do 2 drives in rotation and keep the drive off site…right now I just power down the drive and let it sit.
I have by the way rolled back to a preview version using Mozy just to check it out…and it worked.
I really prefer a triple system of storage.
The corner stone is my Drobo filled up with 2 terabites capacity HD, the I have 2 external HD and a Jpg storage onto Photosheter. Is not a cheap way but it is really safe.
I really need security for my data.