How Many Bluray Disks to Back Up 8tb

NEVER Use A RAID As Your Backup System!

RAID1

NOTE: Over the past few years, I have received a lot of messages and comments about this post. It is possible I did not make clear what I was trying to convey about RAID.

ANOTHER NOTE: I have redesigned the site and for some reason, the comments are not showing. I am working to get them restored.

What I am trying to say is that many people use a RAID as their primary storage system, and have it configured either as a RAID 1 or 5, thinking that the redundancy of the RAID is a backup. THIS IS NOT A BACKUP SYSTEM. This was the point I was trying to make.

So in that vein, I am retitling the post. A RAID IS NOT A BACKUP. Yes it can be used as a backup destination, but the redundancy is not a backup in itself.

UPDATE: As for my current 2018 setup, I am running an iMAC with a Retina 5K Display and a 3.5GHz Processor with 1TB Storage. The daily work for the year is stored on an external 5TB drive. That drive is backed up daily using Time Machine. Every month or so (depending on workload) the backup is backed up to a WD My passport Drive and that copy is moved offsite to a safe deposit box.

Past years are stored on two WD My passport Drives. One in a waterproof Pelican case at home and the other in the safe deposit box.

All worked up images are also stored on PhotoShelter.

Photographers acquire A LOT of images, and that translates to terabytes of data that need to be saved and protected.

Like their shooting styles, their archive/backup schemes can be just as diverse. From RAID systems, like a Drobo, to triple redundant drives, to no backup at all.

One of the first things to remember is that an archive and a backup are NOT the same.

Backing Up Your Data

To borrow a quote from a fellow photographer's blog, "Repeat after me three times: RAID is not backup. Period."

For those who never heard of it, RAID stands for "Redundant Array of Independent Disks" or "Redundant Array of Inexpensive Disks." And for those who use the phrase "RAID array," thats redundant.

The concept of a RAID is to combine multiple, less-expensive drives into a single, higher-capacity and/or faster volume. It is designed for redundancy so that the array and its data remain usable WHEN (NOT IF) a drive fails. The terms 1-disk or 2-disk redundancy refers to the number of drives that can fail with the array remaining usable.

There are many different types of RAID configurations:

RAID 0: Its primary purpose: faster performance. RAID 0 spreads the data across multiple drives. For example, block A is on drive 1, block B is on drive 2), and this permits increased write and read speeds. This is called striping.
RAID 0 offers no protection against drive failure, since this mode does not write any duplicate or parity information.

RAID 1: This mode writes and reads the same data to pairs of drives which is called mirroring. If either drive fails, you can continue working with the other until you can replace the bad one.

RAID 5: This mode is about both speed and redundancy. RAID 5 writes and reads from multiple disks, and it distributes parity data across all the disks in the array. Parity data is a smaller amount of data derived mathematically from a larger set that can accurately describe that larger amount of data, and thus serves to restore it. Since parity information is distributed across all the drives, any drive can fail without causing the entire array to fail.

RAID 5 needs a minimum of three disks to implement. Since data is read from multiple disks, performance can improve under RAID 5. This makes RAID very good for video editing systems.

backup

Other options include RAID 6 or RAID 10, but they aren't often found in consumer-level RAID units. RAID levels 2, 3, and 4 are not commonly used anymore.

The problem with considering a RAID as your backup is that it doesn't help you with file deletion, corruption by applications, operating system or viruses.

So if you accidentally delete a file, it will instantly be removed from both mirrored copies. If your disk is corrupted by a software bug or virus, the corruption will be done to both mirrored copies simultaneously.

Having all the drives in one box that is being served by one power supply and controller has its problems too. A bad enough power surge will probably fry all disks in the RAID. If your house burns down… well, you get the point.

A RAID is still a single device and because of that, also a single point of failure.

None of this means you should not use a RAID. Many photographers I know love the DROBO system. This is fine. JUST BACK IT UP! (I have never used a DROBO, but for another photographer's opinion on DROBO see Scott Kelby's post here: scottkelby.com/2012/im-done-with-drobo

A BACKUP needs to be a complete and recoverable copy of your data that resides on a separate hard drive possibly even a RAID. Just DO NOT USE SOFTWARE THAT MIRRORS THE PRIMARY DRIVE TO THE BACKUP or you will run into the same problems as above with at RAID 1. Proper backup software will perform a full backup and then hourly or daily backups of changed files.

My operating system and work disk (containing the current year's photography) is backed up daily using Apple Time Machine software and a SEPARATE 3-terabyte drive. The drive is also plugged into its own surge protector. This software does not mirror the primary drives but backs up files and changed files. This gives you the opportunity to go back and recover something that may have been accidentally deleted.

The work disk contains ALL RAW files from the current year.

Images that are worked up for publication are exported from Adobe Lightroom and stored on my Photoshelter Archive. I trust Photoshelter and their geographically redundant archive to protect those images. If disaster were to strike, I could still export the images again from the backed up Lightroom archive.

My ARCHIVE of RAW images is stored on a separate drive that contains the last two year's work. These images are also backed up on the primary backup drive.

Untitled

Every year I rotate the oldest year off to a small portable drive. For these backups of the archives, I use Western Digital My Passport 2-terabyte drives. They are small and easily portable for off-site storage.

Basically everything exists in two or three places.

Whatever method you use for backing up and archiving, make sure that your data is stored redundantly and housed in more than one place. It will be the only way to guarantee its safety.

If anyone has any questions, feel free to ask!

107 Responses

very good info!
I use a Asustor Nas 2 disk Raid 1 and a external USB backup disk that backs up daily
plus two 6 TB disks in the PC that keep the data as well
You never cant have enough backups ….
Hi Mr. Marovich.

I am a chemistry graduate student in Michigan, USA (oakland university).

In your post on RAID drives, I get the impression you know well what you are talking about. You were generous to describe your setup so openly.

Would you be able to share what HD or SSD you chose for your work drive and daily backup drive? I think I would like the same setup because, you getting your livelihood from these files suggests to me your setup is good.
I see that WD My book Duo has a 28TB option for RAid 1 which would mean you have only one disk 14tb with a copy (Raid 1 by definition). The setup would cost me $1000. Is there a cheaper way to get massive amount of storage (two 14+, preferably 20TB) that I can do a Raid 1. Essentially the question is, I would like to have a cheaper version of the My book duo but with even more memory space, like two disks of 20TB each. and do a raid 1 configuration.
WHY? I use a RAID 5 10 TB with 2 hot spares on a NAS server. Not a bad setup with a GIGABIT LAN and GIGABIT to the Internet for Offsite backup. Use 25 year ago when I worked at John Space we used NAS server with RAID 5 for onsite backups then we would back up the NAS with a 10 tape DAT which would have tapes for the whole week used one tape a day. at the end of the week I would take the Sunday Tape home for offsite storage usually my safe that was suppose to with stand 1500 degrees fire. I would also so the the last day of the month tape in the same matter. Then after 6 months we had archival storage of the DAT tapes.
Without (closely) reading the post, I think the writer is making the point that RAID by itself is not a backup solution (RAID != backup). But it's awkwardly stated and could confuse some readers. While RAID is not a backup, a good backup system can use RAID. For example, a small business might use a NAS with RAID as their primary backup. Be careful though. A NAS + RAID is not a backup if it's used for primary storage. It's only a backup if primary data storage is elsewhere (e.g. server disks/drives) and NAS + RAID data is copy/duplicate. All good storage systems should have at least 3 instances of data: live/primary data, primary/online backup (copy 1), secondary/offline/offsite backup (copy 2).
I'm using a NAS with RAID 1 plus I installed a USB HDD docking station to it. I setup the NAS to do monthly backups to the external HDD drive. I also backup to some old HDD i had lying around over the years. My main goal for backup is pictures / video that I've accumulated over 20 years, which is ruffly 200GB worth as of now. Honestly that the only data that can't be replaced. I also gave a copy to my EX as an offsite backup. I really dont trust Cloud storage as it can be hacked plus I dont want to pay 10$ a month for the extra space I need.

I also have the NAS configured to my cell phone to auto upload recent pics to the NAS.

Hope this helps
Yes, it is true that you should never use a raid backup system, but if you read the whole article, you will get the required suggestions for this.
I think letting people understanding better to use more causal term, RAID is single version data forever all the time while backup is making versioning of data
For RAID 1,
it applies data changes to all copies on all disks at same time , which mean all copies are same version/identical.

For backup, is making as many data version as copies whenever it performed.
Depend how you perform your backup action : overwrite, delete, create another set…etc
If you want backup, use tape drive. Still the cheapest and safest option per terabyte per tape. Starting costs are high, but it will pay itself back in time.
Of course I have seen that Scott Kelby don't use a DROBO article. I really respect Kelby, but I have used DROBOs for 6 years and no issues. I also maintain a duplicate backup of premises.
Hi Pete.
Thank you for sharing your thoughts about backup solutions. so helpful.
I have two questions and I'd love to know your opinion:
1. Which would you recommend to use- Lightroom or Lightroom Classic?
2. Do you have any thoughts regarding using iCloud Drive (Desktop and Documents)?

Thank you in advance – Moshé
With a NAS that has RAID 1 is a great way to back up your data. Been doing it for years..
Hi Pete:

I'm a hobbyist photographer and VFX artist / studio owner by trade. Each of my 12 or so work projects later year average a terabyte. My primary photographty workflow is: shoot B&W 6×7, scan 4K dpi, edit in Photoshop, prepare 16×20 digital neg in photoshop, print platinum / platinum paladium. The point of which is I'm oft challenged to keep honest backups of my client's and hobby's data.

I've been working with RAIDs on a daily basis since 1994; first on UNIX, then Windows, now Mac and soon LINUX. Couldn't agree more with much but not all of your perspective on RAIDs; nothing like the feeling of a parity or multi-drive crash ruining a false sense of security in one's significant data security investments. Albeit nowhere close to the depression of losing data forever on one (if not both) discrete, single-drive backups… Which is the point of this feedback.

Given one correctly configures the right brand / model of RAID and disks therein, the probability of losing data forever is a tiny fraction of that other's will face using single, discrete drives… even if they make duplicate backups on separate, single drives; presuming they intend to online that data a few or more years later.

As you correctly state, it's about when, not if, a drive fails. An even more accurate statement is replacing drive with "system"; including: drives, enclosures, enclosure components (PSU, controllers, IO interfaces, etc.) and the actual integrity of any compressed backup data format (oft overlooked).

Reasonably minimizing points of failure starts with omitting risky system bottlenecks where possible, like removing compressed backup formats from your workflow. Hardware storage is affordable enough for professional artists to forego making a compressed backup while hoping (if not ignorantly) that the compressed archive won't be corrupt on their otherwise healthy drive(s) years later.

The next step is redundancy. Drive enclosures system components are a common point of failure. Depending which enclosure and disks you compare, sometimes even more likely to fail than the disk itself. However, an enclosure with dual power supplies, drive controllers, IO interfaces, etc. is far less likely to experience an overall system failure than any disk.

As far as I know, there is no readily available drive enclosure with redundant components outside of a RAID. Which brings us back to my point of a viable brand-model of RAID that is properly configured.

The most effecient and secure practice I can reccommend to the working artist, of which a photographer will enjoy the highest success rate is a RAID system with:

1) Dual redundant power supplies, RAID controllers and IO interfaces (i.e. Thunderbolt BUSES, not just ports). Notice the mention of controller here explicitly implies against even considering a workstation-based software or motherboard-integrated RAID controller.

2) 8 or more, enterprise-class, 7200rpm or greater, 6GB SATA or better hard disk drives. The point here is either taking advantage of RAID level 50 (can fail up to two disks without data-loss), fast disk-rebuilds, the most reliable disks built to constantly run (consumer class, 5400rpm, low-throughput drives will keep you busy with IT instead of post), fast read and writes, etc.

3) Directly attaching the RAID to a workstation. Network connected storage, while unavaoidable the larger your outfit is, significantly increases the failure rates of any storage system.

4) Configuring the RAID to use RAID cache, BUT NOT DISK CACHE. Per disk cache in a raid offers a near guarantee of either a drive failure or worse when your power goes out, your kid turns off the RAID while in use or your UPS goes out (oh yeah it happens). RAID cache is reliable enough to enjoy the performance benefits, presuming you have a redundant backup of the component in your RAID.

5) Keeping the RAID utility and firmware updated, but only after doing some homework on users feedback after upgrading. The latter is just as important, as many Promise Technology users may know.

6) Keeping the RAID in a clean, temperature controlled environment. If you're already adhereing to this list; a well-cleaned room that doesn't face major dust challenges with AC religiously keeping the temp under 72F will do just fine. BTW, if you have dust issues, an IQ Air Health Pro does wonders.

7) Buying a reputable model RAID, preferably from a reputable brand. I distinguish the two because in my opinion, at least as far as the visual effects industry goes, LaCie has a horrible track record… however my LaCie 8big rack mount RAID is one of the two most reliable, well-performing RAIDs I've owned / used in 23 years. Since early 2016 this RAID is barely off (maybe twice a year), not for more than a few hours when so, I'm constantly grinding on it at over 1,000MB/s in RAID 5, it's never "acted funny", the software utility is easy to use, my first and only drive failure was two days ago (04.29.18), it took less than five minutes to replace the drive (while the RAID was on) and it was surprisingly inexpensive. I bought it with 24TB for $2,500. You'll find it for at least $300 less today, but I've seen way cheaper price drops, especially on the larger capacity versions. Given my antithetical experience with every other LaCie product used since the year the began selling external storage to general consumers, many of which I still frequently have to use; I'm too nervous to even presume their non-rack mountable RAIDs may offer such stellar performance and reliability. Sadly this means those of you without the space / a rack or tolerance for a bit of fan noise need to hunt for alternatives.

8) Don't buy or use anything made by Promise Techbnologies. These are the worst RAIDS I've ever used; worse than even cheap, software controlled offerings you'd only find in a Best Buy from companies lasting a few years or less. I have two Promise Pegasus R6 RAIDS. Every drive has failed at least twice in each, even enterprise-class drive replacements fail at a similar rate, they've prematurely released at least two firmware versions that created disk and / or total RAID failures on my and countless other customers RAIDs, their support flat out lied to me when I noticed they temporarily "fixed" one of the bad firmware releases with an update that intentionally crippled performance with a significant limit on throughput, their average tech support became poor upon Promise selling to an Indian company, their RAIDs are overpriced, they rarely deliver advertised throughput, a colleague just bought an SSD raid from them and the problems were so bad he moved the drives into a generic enclosure that's since been far better and the stories go on and on and on… enough to make this a point on the list.

9) Drives are overly criticized when it comes to failures. Those obsessive as me may follow a specific model HDD is notorious for failures while another isn't. I've come to realize no matter how many thousands of votes there might be against a specific drive, this feedback is barely worth considering. As mentioned my Promise RAIDS are failed products. Every drive in them has failed at least once, even highly regarded, manufactuer reccommended, enterprise-class models… a couple within a day of installing them! On the other hand, my LaCie 8-Big rack shipped with notoriously hated, consumer grade, Seagate model ST3000DM001 disks; and I mean the notorious version of this model. But also as mentioned, the first and only failed yet happened after grinding non-stop on heavy tasks nearly 3 years… as in exponentially more use than a busy post-production division of a high-end commercial photo studio. So the truth is while the drives fail, what makes them do so is often, if not usually something else; in that case my failed UPS. Keep that in mind when reading feedback on the most widely used models of disks.

10) Stripe your RAID for it's role. I'll use at least a 256K stripe stripe because I'm usually working with multiple streams of HD / UHD / 4K / 8K, 32-float, linear video. This means the rare task of moving a bunch of business documents happens on a geological time scale. If you're working with multi-layer PSDs averaging 500MB – 2GB; consider a 128K or 256K stripe.

11) Maximize your RAID within budget (relatively speaking). If you're…

a) A gambler; go RAID 0 on a run of the mill two-drive external from your local electronics store. If you got nerves of steel, just go external without RAID. Nerves of steel but using the pullout method? Then use an external SSD and hope it doesn't do what those in your cameras have before.

b) A busy pro photographer flying solo in a respectable home studio; use a Thunderbolt 2/3 or USB-C, dual-rendundant hardware RAID in level 5. This is your primary job server and depnding on capacity, is where your primary backups will live for at least a couple of years. Yes, I said it, but that's because you also need a DLT / SDLT tape backup drive… the most reliable and now cheap backup format in sizes ideal to photographers. A 24TB LaCie 8-big rack will be less than $2,500 and the DLT drive with your first tape or two will run under $500 if not half that. If you lose a drive, power supply, fan, controller or cache; you can replace it in minutes without turning off and keep working during a light performance hit until rebuilt. If you simultaneously lose more than one drive, component or a parity drive; rebuild the whole RAID and online whatever backups you need from tape. In 23 years of RAIDs on DOS, Windows, OS X and even NeXt Step – this happened to me three times; twice on Promise and once on the god awful money pit of an Apple X-Serve RAID. Your raid must be connected to either a very high quality UPS or very high quality power conditioner. UPS suck. They all under deliver and if you don't use a really good one on your RAID, it may be riskier than spending less on a high-quality power conditioner. Either way, anyone using a RAID must provide it a clean, stable power source to avoid major drops in data security and performance. Another oft ignored critical practice is, if possible, keep your systems on the same power circuit. As many audio mixers / producers will tell ya, the same system on more than one power circuit is a recipe for failure. Those who don't know it blame it on disk failure. So if you need more than one outlet to run your system(s), find out which useable outlets are on the same breaker circuit.

c) A busy pro photographer in a dedicated studio with 2-3 graphic artists; run 4-drive, SSD hardware RAIDs at level 0 at each workstation. Pull the days work onto them from a network attached RAID 5 server. Backup to that server from each whenever work on a file is done for the day and upon each break. Archive project backups to DLT or if that doesn't make sense from a capacity standpoint, configure the RAID in level 50. This way only a multi-drive / component failure or parity drive loss will be the end. The RAID should be attached to your switch via dual fiber or 1000 GB Ethernet.

d) A busy pro photographer with a dedicated studio and more than 3 artists; invest in a large capacity, network attached RAID at level 50 – one that even has a dedicated CPU, as in a true storage server. Use an additional network attached RAID (doesn't have to be CPU based server) in level 50 or even mirrored exclusively for project archives / backups. Everyone pulls from the project server without working locally, with daily backups to the backup RAID.

Well, hopefully this diatribe was long enogh to pass the disk rebuild on my RAID (hah!) so if you made it this far, by a RAID and get a life!

P.S. – if you need a high-capacity, high-performance, reliable RAID, custom configured at a solid price with great support; especially if it has to be network attached; I can't say enough good things about American Computers & Engineers (aceca.com). Worked with them since 2010. Whether it's been a RAID purchase, workstation rental or many multi-month rentals of up to 70 render nodes at a time; the gear is well-integrated, uses the right components from solid manufacturers, sold / rented at the lowest price I can find, never had a hardware failure, service is fast, support is perfect. BTW, I bought my LaCie 8-Big rack RAID from them… all the more testament to LaCie's atypically great product – considering it was used for rental before I acquired it.
I think it would help to be up front with something you have said in reply to comments:

A RAID is itself not a backup. It is intended to be more reliable disk storage.
There is nothing wrong — and a lot to be commended — for using a RAID as (part of) your backup storage system.

We have a lot of data to be backed up, for a long time. So we have two NAS boxes, each set up as RAID 6. That means the drives will retain their integrity even if two drives fail before we get the replacement drives in place.

And yes, since we have some concern about disaster, we have an offsite process as well.

Most people don't need all of that. HOWEVER: today's hard drives are not all that much more reliable than the floppy disks in days of yore. I recommend using at least RAID 1 for almost any disk drive situation.

[I've been doing disk drives since the early 1980's…]
Pete,

This post is crying out for an update! I'd love to hear what changes you've made to your workflow.

Thanks
Nav
Backing up data…
It's simple; there are two things to consider, file integrity and power redundancy, that's it.

Using two separate RAID 1 arrays together act as RAID 10, simple solution, I have two different 20TB disk arrays, setting backup interval per minute mirroring, I also have disk fails on me average every two years for high usage as daily server storage, never had a single data loss at all.

Guys, it's not as hard as people think, just the initial startup cost is high, that's all…
1. Scotty,
  Your setup should give highly reliable recovery in case of equipment failure.
  However, if I'm understanding correctly that you do "per minute mirroring"… then any time any of the following (or many other scenarios) occurs, you're in trouble:
  – Accidental file delete
  – Bit rot on files not noticed for three months… or a year… or two.
  – Successful ransomware. Files encrypted silently in the background for a week or two, and the encrypted versions get mirrored to your "backup."
  
  These are very real scenarios.
Why not use a router like the Asus AC68U ( which I have ) along with a 2 -4 disk usb 3.0 raid enclosure? The AC68U allows you to use a usb3 drive as a time machine backup plugged directly into the router…combining both the redundancy and backup simultaneously. I currently have a WD 750Gb Passport plugged into the port and it works great, although there is no redundancy currently. I have been looking at 2 and 4 bay usb raid enclosures and will likely be dropping 2x4tb disks in raid 1 for archival / backup purposes with time machine for both the macbook pro and my air.
1. I should mention that this solution is only costing a total $150 / 4-bay usb ( or $50 for a 2 bay ), the cost of the drives ( $99 per 4tb drive currently ) and the router, which i already had. it also allows you to access your files on the fly remotely with asus personal could software ( allowing ftp / smb as well ) , and also built in VPN services.
  
  I will likely just do a 2 bay setup and pay a total of $250 for a full redundant raid 1 setup with time machine accessibility. THATS PRETTY DARN CHEAP!
Hello,
Great advice. What are you thoughts on online back up services like CrashPlan or BackBlaze? Would that be safe enough for you to use as your off site backup?
I'm very late to the party…
I've spent the last seven years with a pair of RAIDs – one for my main working files and a second for backup. Saying unilaterally that a RAID should never be used for backup doesn't make sense if the array itself is used as for a dedicated backup. Large companies depend on RAIDs for backing up as a crucial tool for data security. The other non-array system is tape, but this takes eons to complete a backup and I'm not aware of any company beyond a large dusty mom-and-pop that still uses this.
Waiting for that answer myself. Thanks
Yes, you could back it up to another G-Raid system – but I would suggest (or even recommend) that you back it all up to single Hard Drives instead.

So that way, you have a backup, that doesn't depend on a Raid system.

I use the G-Riad (Raid 1) to backup 'my backup' single Hard Drives.
Hello,

This thread and article are very helpful. Could I use a G-Technology 20T Raid 0 and back it up to another G-Technology 20T Raid or single 10T drive? The reason I ask is that you get the write speed and the safety of a back-up to another drive?

Thank you,
Pete

Fantastic post at exactly the level of detail I needed! Thank you for sharing!

I realise that your setup would have changed in the 4 years since this post was published, so I'm keen to know when you might post an update?

Thanks,
Nav
And I'm the idiot who did just that! (Had no clue)

I have a G-RAID 4tb (4th gen) RAID 0 drive. I think the drives are fine but the power cut off as I was uploading files to the cloud as another back up source.

So it simply won't power on. I know if I take each drive out of the enclosure they would be worthless in and of themselves. But could I put them in an external dual docking system that supports RAID 0?

Grateful for any info you might be able to provide….

-Chris
By the way (for all) if you DO use RAID (as I am), make sure it is RAID 5, RAID 6, or RAID 1 and not RAID 0 which offers no protection whatsoever. Remember – RAID 0, zero protection. Also in any case this should not be your only copy of the data and I think that was the author's main point.
Interesting approach. My system is a little more paranoid.

I have (1) local storage which is very fast; (2) drobo 5N which is my backup of the local storage (2nd copy); (3) several small external drives which each have a couple of years backed up; (4 & 5) : TWO COPIES of everything on bluray data disks. I don't generally burn blurays more than once every 6 months, but it allows be to be less careful with the older stuff on other storage. What is NOT on the bluray but is on everything else is backups of my edits (lightroom, etc).
Hi Pete,

Thanks for writing this! It's really made me rethink my non-existant backup strategy…

So a few questions.. can I backup 3 external hard drives (each being 1-2TB) to a 5TB SATA drive (connected with a Plugable docking station) using Time Machine on an iMac? Would I always need all external drives connected for it to start backing up or can I have any combination of drives plugged in at any time? Or is it best to partition the 5TB SATA and have each external drive backup to its own partition of about 1TB?

Also with that 5TB SATA backup drive, can I then run Time Machine to backup that drive to another 5TB SATA drive in order to backup my backups? I worry that's complicating the process too much!

Thanks for your time,
Emily
For a nonprofit that is creating a digital library of video and images, would you recommend a RAID 10? Individual files are TIF between 20,000 and 300,000KB.
1. You can certainly use a raid for this, BUT the raid needs to be backed up. The point of the post is that you can't count on the redundancy of the RAID to be the backup. You need to back the raid up on another drive, and everything should also be archived someplace else.
Why exactly do you advise against doing an image backup? If it is truly an image, then it contains a navigable file system from which you can copy the backed up files. Maybe this is a difference between Mac and PC but (and I don't want to claim I know Macs) can't Mac users use dd via the terminal, just like Linux users? a one line command and you can get a TRUE image from a disk. One with a navigable file system, just like your backup disk as it is now. You can even schedule a dd without any special software like Time Machine. I guess the only downside is that you write much more data, more often to the disk so the disk might not last as long under heavy use, however, it might be worth it for the ability to restore the underlying operating system in the event of a system disk failure….
thank you all for the ongoing learning(and sharing) discussion. storage, backups, archiving is so critical to our lives now, and new technology continues to try to adapt to offer usable options to address these growing complexities. remember when 1MB sounded like an enormously huge file?

my question is related to Hard Disk Drive size, iCloud Drive, DropBox(Google Drive etc etc…) (assume I'm backing up with Time Machine on several separate local ext hard drives of adequate sizes). even taking home several other HDD home with me for offsite backup storage).

in my case, i use both iCloud drive, and DropBox. i store my entire Photos Library on iCloud, with pretty much every document file I've created, stored/filed on DropBox(i may make an alias of a folder on my desktop for folders i access most frequently). i know DB can be arranged to 'selective sync' folders that i don't need on my local drives like laptop: MBPro,( but could still access via dropbox.com account as long as i have internet service). my growing concern and problem running out of Disk Space. iCloud Drive stores all files on local HD; DB can upload files to server. with 1TB storage on my Mac Pro(tower), I'm essentially running out of internal disk storage space to continue to hold my growing Photos library,( and LR Catalog folder). my Photos library(downloaded originals) alone is getting close to 800GB( BTW, i don't store any video of professional web content we have produced, on any local disk, only on external Drives). if i attempt to store a backup of the Masters folder (Mac HD/Users/HomeFolder/Pictures/.Photoslibrary) on DropBox it stores redundant files on my local HD… my MacPro has (max) 1TB SSD. what happens when my .photoslibrary gets bigger than 1TB?

where does anyone store their original .Photoslibrary or LR catalog folders, if not on the computer's HD? external HDD are bigger, and cheaper(?) than internal disks; can i place my .photoslibrary on an external drive? will it still sync with iCloud? Can it still be used with Time Machine to back up somewhere? whats the probability of corruption with any set up like I'm talking about? do you remove all images from many many years past from the photos library, and store separately somewhere outside of the computer? put older content into a separate .photoslibrary? one library for each 5-year period or something? can i daisy chain Mac Mini's to add additional OS HDs to the environment to allow Time Machine to store backups on separate Hard disks?

I'm not even going to ask where a RAID system would try to fit into anything like the options above…(unless there is a suitable option?)

thanks, and thank you for continuing to share your experiences and knowledge with the rest of us…it beats having to go back to school to try to 'get learned' any of this new technology stuff. give me back my darkroom and Negs! ( actually, i don't really miss those either, but they were analog in comparison, at least.)
1. Hi Scott/Peter,
  
  I've been using computers for many years and in my experience, I've only ever lost data on two occasions. One was when a hard drive filled with personal data and photos had broken due to very rough handling by a carrier when I was emigrating from one country to another (the disk inside had shattered) and two was when I had accidentally plugged in a newer Lexar USB 2.0 stick into an older HP laptop with a bigger operating voltage which somehow fried it. Aside from that I've had no losses of data in general.
  
  In my experience as a user, RAID drives are extremely useful, especially the ones from Drobo. With any system of storage, proprietary or otherwise, failure can be expected – even with online cloud systems. A friend of mine who is a well-known German photographer, somehow had lost over 1TB of data with his Dropbox account when a syncing malfunction occurred with his Mac desktop. It took him a very long time to retrieve the files. The key point to a successful and safe backup system is not just quality backups, but fast systems retrieval and restoration. People are suggesting to use Raid10 but in my experience Raid10 is generally not needed. Also with such high rates of transfer it can cause bitrot and dataloss on drives due to wear which is another issue. Raid5 in this respect is better, as it brings a balance between performance and data integrity.
  
  As to your question Scott on your workflow, the system you have is not bad, but I find it is not allowing you for growth (Mac Pro 1TB) nor extra protection (especially with external drives that you've saved data to). In the likely event an external drive dies, or your Mac Pro drive fries so would your work. I would therefore likely do the following:
  
  1) As you use Macs, I would suggest you invest into at least 2-3 Drobo units, and populate them with Toshiba or HGST drives (which have been tested by Backblaze as some of the most reliable drives around) so I would use those. I have used Drobos and Areca RAIDs for more than 10 years in my business as well as home and have never lost anything with them, compared to other brands like Promise, Lacie, Highpoint and Buffalo which I found to be shoddy. Lacie in particular as I found it used cheap parts to create a RAID in their Firewire drives (how does two IDE spec drives stuck together to make a RAID sound like to you?).
  
  And to answer your question yes you can move your iPhoto library to external drives, as well as your Lightroom library. You can follow this guide here > http://appleinsider.com/articles/15/01/25/how-to-safely-move-your-macs-iphoto-library-onto-an-external-drive and here > https://www.google.com.hk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=14&ved=0ahUKEwj-1v26irPYAhVBnJQKHeqfAQYQtwIIdDAN&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DEnfGAwcOT5k&usg=AOvVaw1wvsm3XWwTB_8PQfbnTTu9
  
  So I would suggest using two Drobo 5D3s as your main work and backup units (connected via Thunderbolt3 daisychain), and then a Drobo 5N2 (or more) on your network as a tertiary backup. I would also have a 1TB USB 3.0 or TB drive attached to the Mac Pro as system drive backup (basically as backup against drive failure on the main Mac Pro drive). I would also employ the Superduper drive cloning app from Shirt-Pocket as well as Chronosync and iPhoto Manager from Fat Cat Software.
  
  With this setup, this is how I do it:
  
  1) As your main work unit, you would work and save all your most important files on the 5D3. Since it has up to 64TB of potential storage and is on Thunderbolt 3, you are not limited by size or speed limitations set with other RAID systems.
  
  I would first migrate the entire iPhoto and Lightroom libraries to the first Drobo 5D3, by creating new folders for it. This leaves your Mac Pro's 1TB SSD free for systems access and performance (which is crucial). I would also save or copy any video files or works done on the external drives and save a copy on the 1st Drobo, which would be populated with minimum 15TB (5 x 3TB drives).
  
  I would then create a scheduled sync copy of the 1st Drobo on Superduper that would replicate everything on the 1st Drobo to the 2nd. The schedule would set the replication of the Drobos at the end of the working day.
  
  2) Dropbox and iCloud would be used to save all my important personal files for my business and whatnot. iCloud would be used for backing up smartphone data. I may keep photos on Dropbox but only a small amount. Aside from keeping a copy on the Mac Pro, this would also be synced over to the 5N2 using Chronosync.
  
  3) At the end of the working day, I would backup the files worked on during the day to external drives and take those home with me (in case of fire etc).
  
  4) I would set TM on the Mac Pro, and have that sync with a Drobo. Drobos can be set with a Time Machine volume, so you have a choice to either have TM enabled on the 5D3 or on the 5N2. In my experience I would probably put it on the 5N2, since I already have a backup of the Mac Pro's system via the attached USB 3.0 or 1TB external SSD TB drive, the TM backup is just precautionary measure against failure. I would also likely take the TB backup drive of the Mac Pro with me.
  
  5) As 3rd copy of data offsite I would set up an account with the likes of Crashplan or Backblaze and have a copy of the 1st Drobo uploaded to their server.
  
  6) If adding a 4th copy of the data, I would likely either setup a VPN network both ends and have a 2nd Drobo 5N2 transfer the data by scheduled Chronosync directly from the office to the home OR have a 3rd Drobo 5D3 setup at home where data I have from work via the external drives is copied over to manually at the end of each working day. Being on the network you can also copy what is on the 1st 5N2, so you have a secondary copy of the TM backups.
  
  This setup seems costly at first, but compared to the cost of paying for more Dropbox or iCloud storage or more Mac Minis or loss of data because of poor planning it will pay for itself.
hello – how often do you replace your onsite/offsite backup drives for each year? or is the assumption that those are ok indefinitely since they're not used very often?

thank you in advance for your advice!
Just a short question. I have a 12 TB Raid 0 on a Lacie 2Big. When I want to move on, could I just take out the 2 disks and store them as backup and use two new disks for new work? Would the 2 old disk work as backup meaning when I put them back into the LaCie, would they still work of will the Raid be lost or at least risky? I hope my question is clear .
1. Hopefully, someone may be able to help you better. My concern would be if the RAID is being administered by hardware in the unit or by software. If you pull the drives out and then replace them in the same enclosure and the RAID is being administered by hardware in the enclosure, all should be OK. Of course, if something ever goes wrong in the hardware, you probably will not be able to use those drives as a backup unless you find another Lacie 2Big.
  
  Since I don't know much about the LaCie drives and RAID implementation, I can't be much help. But I think it is a risky proposition.
I have a IMac deskop 27 x 3 years old – have 2 8TB external hard drives which contain about 14TB of storage – really scared to lose my images – I was about to buy a QNAP raid simply for back up – have no other purpose for the Raid other than to have a secure back up – I am very unsure what to do now having read your excellent report I have loads of hard drives in my cupboard which is the reason why I was thinking about the Raid
1. Hi Gavan
  
  I am assuming from what you are saying that you have 14TB of images on those two drives. My first question is do you need to have all 14TB of images online (I use the term online here as meaning connected to the computer and accessible instantly) at once. Sounds to me like much of that could be archived, i.e. placed on offline drives and stored (at least 2 copies in different locations). IF that is the case I would archive as much as possible and then use 1 8TB drive for storage and one for backup, moving older images to the archive as you need more space on the first drive.
  
  I have received many emails saying that you can use a RAID configured drive as a backup drive. Which is true. My headline is not as clear as it should be. My point is that having a system where a Mirrored RAID (RAID 1) is the primary drive AND the mirror is the backup is a very bad idea. But you could use a RAID (4,5 or 6) as the backup drive if you really have to have both 8TB drives online and backed up. The problem, of course, is that eventually having all these images online at once will be unmanageable. And most likely there is no need for it.
  
  I recommend archiving the older images and keeping your online storage and backup at manageable sizes.
  
  Hope that helps.
regarding your two separate drives (by year) – one onsite, one offsite: do you ever replace these drives or is the assumption that they should work indefinitely since you're not actually using them on a regular basis? thanks in advance for the info!
Nice article. Well written! Informative! Thanks. I used to but have not recently, switched hard drives every 6 months to a safe deposit. So your two years 2012 and 2011 would be off site, and most you could lose in fire, etc would be 6 months instead of 24 months! Thoughts??

Greg T.
I've used the same backup strategy. using time machine instead of RAID. However on several occasions it has happened that all of a sudden, time machine completely erases itself and starts over from scratch. This terrifies me. Have you run into this as well? I'm wondering if time machine does this when it runs out of space, or if it's a maintenance thing, but I'm terrified it would start doing this when I really need to retrieve something.
1. Time Machine should only delete the oldest backups first.
  1. and usually it does, but once in a while it starts over from scratch. erases everything and starts over.
I think the article is not clear enough. RAID is NOT a backup strategy, so it has no sense at all. RAID purpose is to offer some sort of redundancy in order to provide more reliability on the data availability. Another point: using Hw based RAID will have more reliability than Sw based RAID (it's more expensive, but is "safer" also). Using RAID6 with at least a Host Spare will offer the most reliability at a RAID support level.
Now, backup. A backup consist in a policy and strategy to save (backup) your data in the best adapter manner for your use. What does it mean? That using a good backup strategy will be based in the policy you needed to backup your data, according to some parameters (Google it, Time To Recover Backup; RTO, RPO). And… YES, you can use RAID based enclosures to perform this operations! Of course! that's what the industry in fact currently recommends as it reduces both RPO and RTO and help you have your data available almost every time. It's also recommended, in a case it is really critical to have your solution synchronized with another storage enclosure. But… just saying "don't use RAID to backup your data" it's a fallacy 😉 Just, read more, learn more.
1. The title might be misleading and I am due to update this since my configuation has changed. But the point of the post was to address the people that were using a RAID (such as a DROBO) as their main work storage and assuming that since it was mirrored, that it was backed up.
Thanks for this wealth of information. +1 on avoiding a Dobro. I worked at a radio station. We used a big Dobro to backup our 10 Pro Tools workstations with Retrospect. The Dobro was always causing problems so backups often didn't happen. Retrospect was problematic too. The entire setup was a big kluge. We finally dumped the network solution and put an additional Time Machine drive connected locally to each workstation. That was fine, but we really should have used dual drives configured for Raid 1. A single drive can (will) fail. Now the chances of your backup, or should I say archive drive and your system drives failing at the same time are small, and we never had it happen. But for mission critical work, you should have 2 local copies and an off site solution.
One more point. Surge protectors offer little to no protection. A power failure can screw you and surge protectors don't suppress all surges. Once they get hit hard, most fail and offer no protection and you probably won't know that until it is too late. The best solution is a UPS where the line input charges the batteries and the output power is always supplied by the batteries. That is probably the most reliable and affordable surge protection. Plus, most UPS units can shut down your equipment before the batteries run out of power during an extended power failure.
This is how I have my home/studio set up. All computers back up with Time Machine to a 10TB mirrored raid connected to my studio computer on a UPS that will shut down the computer safely if there is a long power failure. I also backup my really important work to a cloud. It's slow and a pain, but it's cheap insurance for the stuff I can't live without. Been working this way a long time and never lost any data. Had local drives fail, but the Raid saved my ass every time. Never had to go to the cloud.
1. Hi Steve and others, Your configuration sounds worth investigating further for our requirements. I'm in the same situation. Could I ask what cloud service you are using? was it easy to configure? and what mirrored raid level.
  Glad it's working and I'm looking to avoid an eventual single drive failure. I have a number QNAP NAS systems 8TB, 24TB and 96Tb for large HD video files. I don't look forward to backing up our mission critical data!
Hey Pete,
I hope you can help me out.
I use Time Machine to backup up system and currently have a Lacie with 2 qty 3TB drives RAID 1. My drives are full and my data from 2012 is being overwritten, this is okay.
I would like to get two new drives at a higher capacity to replace these; how do I transfer this over and after reading these comments, is this the correct thing to do?
I have a new Mercury elite pro thunderbolt enclosure I was going to use and take the Lacie home. Do I use this as my second backup? What size HDD do you recommend, knowing I have 3TB (6TB RAID) and any brand that works best in your opinion?
How do I connect these and follow your rule based on two separate drives rather than one?

Your expertise is most appreciated.
Thank you. Nice article
Raid 1 should only be used for servers to duplicate the boot drive. No other situation calls for Raid 1 because it's usually a waste of space and a lot of wear and tear. Production servers are the only ones that need that kind of redundancy.
I also agree: "RAID is not backup"
But if you have a backup, I suggest to use RAID in the backup repository.
Here is a performance comparison about different RAID levels:
http://goo.gl/wVca2k
Still don't get why RAID can't be used for backup. If your external drive fails and you don't have other copies of the data its gone. On the other hand if the external drive was run via a 2-hub RAID1, one drive could die and you still have your backup save on the other.
Pete, I'd appreciate your assistance with current and planed workflow:

Here is my scenario. I have a 2014 Macbook Pro with 3 external Dell 24″ monitors connected to it (2 via display port and 1 via HDMI). I come in from a shoot and take my SD card, dump it to a new folder on my MBP. Edit, and then move to a primary external drive (passport 2TB). CCC then copies any new changes to another 2TB passport AND a 4TB G-Drive. All new data is also uploaded to Crashplan and Time Machine is also running. I lost 177gb of data 3 nights due to a corrupt drive. I was able to recover it using Data Rescue. What's funny is that the only reason I had to recover anything was because I hadn't moved data from my laptop to my external drive in about a week. My backup process was there, I just got lazy.

So….I was about to updgrade to some larger drives anyway and since I just saved $1000+ since I didn't have to send my drive off for recovery, I want to up my game. I like the portability of my MBP BUT part of the reason I didn't back it up over the past week was that it wasn't at my workstation…it was with me. I am considering adding another iMac 5k to my house but this one to be used as my primary editing machine…still keeping my laptop but really only for portability. Since I have three monitors and they take up my two display ports (assume at least one of those is the Thunderbolt port) and HDMI port, I picked up a Thunderbolt hub to connect the monitors. Tomorrow I'll get my iMac 5k and two OWC ThunderBay 4 RAID 5 in 16TB and 20TB.

The plan was to setup the 16TB as primary and 20TB as backup. Even OWC suggested that. What am I missing?
1. Seems reasonable to me. The weakest link for me also is the time span between the shoot and the movement of the new files to the desktop system. I try to be diligent about the transfer so that if the macbook crashes, I have a backup. I think I am going to get a wi-fi enabled WD My Passport to keep nearby to backup without me having to think about it.
  
  Since this post, the setup has changed since the tower has been replaced with a 5K iMac. Update to the post is forthcoming.
  1. Hi Peter,
    I am not a pro photographer, simply an enthusiast taking classes and learning all the time. I am, however, a mom and that means my images are very precious to me:-) What are your thoughts since your last article and since your equipment upgrade? I have a 2013 MBP, no desktop, a LaCie Rugged 2TB disc and a 1TB WD MyBook (totally full). I need a 2nd drive and need to build in more redundancy. Also, it would be nice for back up to be faster so that I will do it more frequently. We lost 5 years of photos from all of our travels when my 3 yr old 1TB LaCie Rugged died early this year and I would be so sad to repeat that with my family photos. THanks for your help!
  2. Interesting article.
    It has been a few years since this was written, and because you have upgraded to the 5K iMac, will the update be coming soon? It would be interesting to see how your workflow may have changed.
    (or I may have missed the update?)
What happens when the 3TB fails? Shouldn't it be RAIDed to replace the disk to preserve the data? I don't know if you can have a RAID back up with a separate non-RAID OS disk setup.
1. The 3TB IS the backup. On it is the backup of the system from the tower, the work disk backup and the backup of the last two years disk. The last two years are also on separate drives that are stored in a safe deposit box.
  
  This setup has been changed recently since the tower has been replaced with a 5K iMac. Post update is forthcoming.
I get what you say about RAID as a back-up system but I would argue there's nothing wrong with it as an external drive storage option if your primary machine is a Macbook and you dont have the option of multiple internal drives.

One related question: I have a Mabook with about 700 gigs of used storage and an external drive holding 648 gigs (mostly photos) which I'm backing up on a 3 TB Time Capsule. That feels a bit small and its going to feel tighter – I add about 125 gigs a year (shooting a D800). Time Machine doesnt offer the option of spanning drives. Any solution you can suggest?

Second question: Is there any point in using SuperDuper if I'm already using Time Machine?
1. I never really considered options for backing up if my only machine was a laptop.
  
  As for the Super Duper question, I use Carbon Copy Cloner to back up my Macbook since I need to have a quick recovery ability if traveling. I also have a copy running on the desktop to create a bootable copy of the system that I can use quickly.
I believe I may be asking a similar question to Doug on the 25th Nov 2015. I have a new iMac and an external Pegasus R8 array which comprises 8 x 4Tb drives (= 32Tb) set up in RAID 5.
I also want to have all the image files contained on the Pegasus backed up twice. I already own some older Sonnet Fusion enclosures (2 x 4 drives = 8 total) and can max those out at 24Tb which is more than enough) but my question is what is the best way to 'mirror' what is on the Pegasus? Do I separate the info on it into a series of folders say 1-8 and simply do a Raid 1 set up ie mirror each folder to each corresponding drive on the Sonnet? Or is there a much smarter way, ie RAID 5 on both? I would intend to take the Sonnet drives off site periodically and swap them over so hopefully everything is backed up in triplicate.
Many thanks for any enlightenment! 😉
Chris
1. The Raid 5 should be considered at one big drive as far a backup software in concerned. But this is where you have created an issue for yourself. Since you feel you need to have 32 TB of data readily accessible, you need a 32 TB drive to back it up. (or should have anyway)
  
  Unless you are doing high end video work and need that much space to work, you would be much better off and safer using smaller separate drives which can easily be backed up.
  
  In my work there is no need for me to have raw files, from 3 years ago, online and accessible instantly. I have those on separate drives (by year) in a case near my computer if I need them. Their backups are in a safe deposit box. All of my "worked up" files that are ready for publication are on PhotoShelter.
I agree and also disagree with "Never use raid as your backup system"
1. Agree that raid is not a substitute for a backup.
2. Disagree in using raid on a backup system.

I use NAS for backups and all of my NAS are raid 1 or raid 10.

So you can and should use raid for storing backups.

I think the confusion comes with people using raid and thinking its as good as a backup. Its not. But if you want to make sure your backups are viable I would store them on a raid 1. Then it is less likely when you need to go to that backup that its bad.
I use ZFS with scrubs + ecc to ensure no bit rot creeps into my backups too.

If you can't use raid for backups then I would suggest keeping multiple backups on cheap usb drives.
1. I agree the title is misleading.
Pete,

Thanks for this superb article.

When responding to PJ you mentioned two internal scratch disks. I'm assuming the first scratch disk is listed on the diagram as "Last Two Years Work" that also contains the raw files from the last two years. What does the second scratch disk contain?

Best wishes,
Jen
1. Hi Jen. This post needs to be updated since I have since replaced the tower with an iMac 5k Retina machine. So there are no longer internal drives (except for the 500GB Flash drive). I plan to update this post soon.
  
  But to answer your question, the extra scratch disk was just a temporary storage place for misc. stuff.
I must say for a Non tech (I am assuming that) or in your case a photographer you are quite adept and your methods are solid. You can recover data from Raid drives but it will be expensive. Have had to do it several times in my career for clients that simply had no backups or never tested the backups. Never lost data in 25 years in IT doing storage and Server management. Not that I am sure it is possible data is irrecoverable but I have been lucky enough to always be able to recover though it has cost some big dollars a few times for the client that did not listen.
I have trained many Photographers and Photo Booth Operators and know that you are a very smart group in general. Rarely am I asked more questions after the first 6 months unless its things outside the scope of the shop they are running.
As to Raid I would highly suggest Synology devices for your backups. They are extremely well made, very versatile and I have heard so many positive experience reports and few bad ones. I do agree 100% in the policy of having 2 backups of your data. One is simply not enough. Plus an offsite cloud copy is a great idea as it is so inexpensive. BackBlaze is $50 per year unlimited for one machine. Other deals offer 1 TB at $50 a year unlimited machines and there are offers with Synology now for some cloud backup that is very reasonable such as $10 TB per month. Amazon has a new low cost offering as well. JungleDisk is another excellent choice. For Windows software SyncBackPro is excellent as its Touch add on software can backup even Android phones. Quite amazing,

So if it was my choice I would have Raid 1 on the OS drive and I agree as you stated it is only to avoid being down if a hard drive fails as it serves no other purpose. But it does that very well and drives fail. I just built a Raid 1 SSD machine for a client. He loves it.
Next a Raid backup unit and as large or as small as you need are available and Synology has been extremely solid from low end to high end.
Lastly a USB hard drive as secondary backup. An 8TB USB Drive from Seagate is $250. So you really could buy several if needed or smaller drives if not including the Passports you are using which are just fine or the smaller Seagate backup drives.
Great writeup but I believe the most important thing is off site backup as well as on site. If you use only 1 drive for OS and data, then 1 USB drive for local daily backup (using TM is great) and then something in the Cloud with a copy of the local backup drive should be the minimum. That way it would be very hard to lose everything especially if you use a good cloud backup with differential backups allowing you to go back as many days as needed to get before possible corruption or the Spyware that Encypts your data and holds it hostage till you pay. The FBI says you should pay. Seems like they must be behind it to say that. Why not say HAVE DAILY BACKUPS. JungleDisk is a perfect cloud backup system. I sell it but I am not giving my links here as I mention it only because it is great. Go to their website and try it..
Wow, amazing wealth of info here. I may have missed it, but where do I learn how to setup a RAID 5? I do tons of HD video editing on my MacBook Pro and have almost filled all THREE of my external 5TB USB 3.0 drives (yes, 15TB is a lot!). I think what I want to do is connect them together as one (a RAID 5) and then have another setup as a backup. This would avoid the nightmare of plugging drives in and out and trying to figure out which things have been backed up.

Sound logical / correct ?

If so , how do I create the RAID 5?
Just FYI, Drobo is NOT a RAID array. It performs in a similar way to RAID, but with a lot fewer restrictions (i.e., maintaining disk order, same disk sizes, etc.) Having said that, you are correct about disaster recovery vs. archiving.
1. This post was written originally before DROBO deployed their "BeyondRaid" technology. But truthfully, the only thing scarier to me than the traditional hardware based RAID would be a proprietary software based RAID.
Yeah??? But Raid1 is my backup! What happens if your 3 tb drive crashes? Everything older than two years is gone unless you got another backup some wheres. Get another 3TB drive and make a 3TB raid 1 array. Don't use Raid 1 for your main drive. Instead, use it for you 3TB backup system. Make incremental backups to the Raid 1 backup system Any file you accidentally delete from your main drive will still be available on your Raid1 backup device. If one of the 3 TB drive crash, you will still have your data on the other one. Never delete data from the Raid 1 array. Let your backup software do that. I use raid1 for a backup up array. It works fine without all the difficulty trying to use Raid1 for your main drive entails. Yes, a raid1 main drive is not good idea for data preservation. Instead, it's better used for main drive to allow a system to keep running if one of the hard drives fail. Put data somewhere else. I do incremental backups to my Raid1 array all the time. It works fine. It doesn't delete a file from the backup when that file is no longer present on the main drive. I don't let it delete anything I don't want deleted. Deleting something on my drain drive doesn't cause the file to be deleted from my backup raid1 array. I use Ubuntu's Backup utility to backup my data to a software Raid1 on another machine. The incremental backups take no time at all. Use the imagination! Main drive raid1?? Nah, not for data preservation! Instead, backup your data somewhere else. That way, if you accidentally delete a file from your main drive, you can go get it from your raid1 backup device. You can even get older versions of the file from your raid1 backup device if you changed it at some point.
1. You apparently did not read the post closely. If the 3 TB drive (this is actually now a 6TB drive) crashes I get a new one and back up the system again. It is the BACKUP not the work disk or the system disk (these are on two separate drives). As for the drive with the last two years of work, if you look at the bottom of the diagram, there is a separate drive with each years work stored in a watertight case on site. There is also another copy of each of those drives in a safe deposit box.
  
  Now when talking about your RAID1, you say "If one of the 3 TB drive crash, you will still have your data on the other one." Yes that is true, UNLESS there is a controller error (which effect BOTH drives), or if the data on the first drive corrupts (and this happens), it will automatically be mirrored to the other. If the RAID software reads garbage from the first drive, it WILL write it to the second drive. It does not know it is garbage. The software does not know if a file is corrupt, it is all 0's and 1's and they all get copied. RAID doesn't protect you against a file being deleted or overwritten.
  
  One last thing, there is also a difference between a backup and an archive. Backups are for a recovery process and archives are for the retrieval of information. Backups are copies and archives are originals. Backups are usually done with proprietary technology and archive technology is usually open standards, so material can better be retrieved years from now.
  
  The idea behind archiving is to take that material out of your system and putting it somewhere safe.
  1. I can't argue your points. They all seem valid to me. I originally didn't understand that your approach had backups offline. I've heard that I don't need to be able to read fast, I just go to be able to read! However, I don't differentiate between archives and backups since my system contains every data file I ever created and which I want to keep.
    To prevent loss of data, I back up my system routinely to a raid 1 array. I recovered a couple files I accidentally deleted from my system. Otherwise, I've never had to use it. After the initial backup, I make only incremental backups since it seems unnecessary to backup something I already backed up before. If the array completely fails, where it isn't recoverable, then I still I have my original data and can simply create a new array after replacing the failed components on the backup device. I was only suggesting that Raid 1 is a perfectly suitable medium for backup data. The idea is not to run the system on a raid 1 array and expect that you don't have to backup. Instead, run a raid 1 array for on another device for backups. To loose data, I need to have my mains system drive crash as well as both drives in the Raid 1 array. I feel lucky!
Mike Chambers is exactly right.
I would like to point out that there are software and hardware RAID solutions with hardware being the preferred and safer option.
My workstation has a single RAID 5 array consisting of three 2TB SAS drives plus one hot spare. If one drive fails, the hot spare is automatically used to rebuild the volume. The RAID controller's software would have notified me of the impending failure, actual failure, hot spare rebuild…etc. I also use a RAID 5 NAS device with the same configuration. Unfortunately, off-site/cloud backups are not an option due to my bandwidth limitations. Any single hardware RAID option is superior to a single external drive and I would feel much better having a good RAID 5 setup over multiple external USB drives. The only benefit to external USB is its portability.
1. Anyone who just uses a RAID as a backup is asking for trouble. As Mike Chambers said…. "RAID mirroring protects against a single disaster; hard disk failure." Also, if you delete a file, it is deleted on the mirror. If a file corrupts, there is a excellent chance it will corrupt on the mirror. Saying that off-site backups is not an option is ridiculous. You may not be able to use a cloud option because of the bandwidth, but you still need off-site backups. of the RAID. RAID IS NOT A BACKUP.
  
  For more information please read: http://www.2brightsparks.com/resources/articles/RAID-is-not-a-backup-solution.html
RAID, especially RAID 1 (AKA Mirroring, is indeed a viable backup solution. However, it is NOT a whole backup solution. In the heyday of the mainframe, before PCs and the Internet, mirroring data was highly desirable way to backup your data.
The reality of how safe your data is using TAID is contingent upon severak factors::
1. What level of redundancy do you really need?
2, How much do you Have to spend?
3, How well protected is your data>?
Most home users do not require 24/7 by 365 uptime. Some businesses might. This will determine how deep your redundancy needs to be…. RAID, Mirrored disks with dual controllers, Mirrored systems ae all options depending on specific needs. The bottom line is that RAID mirroring protects against a single disaster; hard disk failure. It does that very well.
However if you get hit by a virus, accidentally delete data or have corruptions not causd by hardware failure, RAID will most likely not help you. Remember that hard disks are mechanical devices with are expected to fail eventually. This is the specific purpose of mirroring.
Hello everyone! When I recently upgraded my internal drive on my iMac 2009 27″, with a new HDD combined with an SSD for the OS and applications and found out that although we technically have a infinite amount of knowledge and information about back-up systems we seem to omit talking about the actual experience of having to restore your system within our blogs, which in my opinion is just as crucial. The user thus has to search about restoring a computer, which to my opinion requires just as must research than the time spent on understanding back-ups. Why are both subjects so appart from one another. For months I spent time looking up solutions (softwares and hardwares) and they all sell how good they are with backing up and how easy it is restoring, which to my surprise IS NOT as simple as one might think. From a blogger (and god thank him) I took the following precaution which saved me a huge headache. Even before I started, I created a bootable drive on a USB key in order to install my OS back onto the new SSD. From what I read and experienced, this saved me huge problems not mentioning a lot of time!

I also found out that Time Machine has issues with backing up huge files and often stops the restore process when the size of files are too big. If you use softwares such as iPhoto and Aperture and your libraries are consolidated, then that creates an issue you would not have with a "Referenced" library. Look this subject up.

This being said, I feel that regardless of your approach with back up systems, you need to test and document restoring your back ups before you system does crash. This will ensure the user has a plan and knows how to proceed when all fails!!!…..

G.
1. Nicely spoken! I have crashed so many times in the last 20 years and learn something from each. But wisdom is learning from other's mistakes too.
another problem with RAID: even the most elegant pre-configured solution in the market, Drobo, has tremendous risks, as famous photographer/author Scott Kelby complains in 2012-06 http://scottkelby.com/2012/im-done-with-drobo: there's no point in raid that constantly monitors the health of HDs & auto-shifts data to healthy drives if the drives keep failing due to software/os/virus/shortcircuit etc. + if you use proprietary system as Drobo you can't plug drives into different system to recuperate files
Very helpful info – thanks.
Just as a side note ~ I came here as my Time Machine backup has failed. So I'm not sure it is trustworthy software. I have random folders & subfolders to the tune of 150GB missing from my 800GB TM backup. Apparently your OS X system records of what has been modified can get errors & then that stuff is ignored by the TM backup over time. Luckily I noticed this before I actually needed the backups!
RAID 1 with 4×3 TB SATA drives, backed up through timemacine. Job done
Hi!

Your article is very learnable to read, as I the last four years used to save all my photos I´ve been taking the last 10 years on a RAID 0 using two 1 TB internal hard drives.

Last week, the PC lagged and there was some strange clicking noise from it, and after rebooting, I get an error message the RAID 0 was failure and it will take a long time to start Windows. And in the BIOS I can see only one of the disc´s can be reqognized.

I know I should blaim myself to be stupid for having no backup! But may it be some change to retrive some of the lost data, without leave to a professional costing up to $1000?
Are all files splitted to the two disc´s, or may some files be at only one of the disc to maybe be recovered if take the RAID away?
And if I try do so, can´t it be set as RAID 0 later if I will then try to start the bad disc again? Or maybe I should try the freezer method!?

Best regards: Mikael Markstrom
When I was looking to backup my RAW photos and files in general, first of course I do ue time machine. But, I wanted offsite data (home office) and looked at dropbox as a solution. I can backup the same data as I do on time machine and it has the added feature of being able to recover deleted files (extra fee). Now I'm also looking into Amazon's cloud storage, haven't looked into it much beyond around $70 a year for unlimited storage. If that's true, I could even mirror my time machine if I wanted.

Is using multiple cloud storage for backup like this come off as just inane or is it a practical solution for backup? If Amazon is basing their cloud on their S3, wouldn't it be both reliable and with unlimited storage, can both mirror time machine and store older data in conjunction with other cloud services which now includes Microsoft getting into the game with enormous storage at a very low price.

I don't know where and how secure dropbox is if they had a major failure, but certainly Amazon and Microsoft are likely to be secure even if major failures occur.

One last note is that while it's not a cloud solution yet, Google has nearly completed their system for "perfect" reliability. Impossible for their search engine and the data they collect to be lost.
Thanks for this! I found extremely helpful. I've inherited a small personal library of some pretty great footage. It all on lives one shared RAID drive. Editors work directly on this drive and once their down they create a new folder called, "exports" but it still lives on the shared RAID.

I'm trying to fix this, and feel a bit lost in the process.

Here's how I understand the workflow (please correct me if I'm wrong):

Original files get imported into the RAID as a new project name (with a batch rename of all files for future ease). This gets backed up through Time Machine at the end of everyday. Editors can then go in and create complete projects, which will ALSO get backed up through Time Machine.

HOWEVER, what about an off-site storage. This is where I'm getting lost. How often do you have to bring the off-site 3 TB drive to get updated? Does it automatically know which files are new and where they belong? How do you make sure projects stay complete and together when it backs up and updates files?

Thanks in advance!
Questions:
Q1 a&b: What is is the problem with Raid-1 using Time Machine? …Won't I end up with TWO Identical Time Machine on the Raid-1 external drive dedicated to back-ups using Time Machine program? … that isnt that superior to having just a single external drive back-up?

Maybe there's something I don't understand in your article "NEVER use Raid for a back-up". I will appreciate your answer as I plan to purchase a Western Digital "My Book Thunderbolt Duo 4TB Dual-Drive with RAID" to replace an older 2TB "Iomega Companion for Mac' that I've using… and use the new WD MyBook external drive in Raid-1 conficuration so I always have two back-up time-machine copies of everything.

My thought is that for additional back-up security, I could even switch out one of the removable/replaceable drives in WD "MyBook Thunderbolt Duo 4TB Dual-Drive with RAID" the on a weekly so I have also have an offsite copy thats no more than a week-old at any given time in case of fire other catrastophe at the physical site of my digital workstation.

Any answer or further explanation will be appreciated. Thanks

Becase the WD …. and a third off site if I follow-thru w/ the
1. RAID software instantly mirrors one drive to the other in the RAID. If your time machine backup software fails or corrupts during the backup, both sets of data will be corrupted.
  
  RAID gives redundancy, IT IS NOT a backup. Yes you can use a RAID array as your backup drive, BUT if the array fails you still need a backup that is stored on an entirely different physical disk. The second drive in a RAID 1 array IS NOT that different disk.
  
  When you configure the RAID 1, you will never see two drives mounted on the desktop. If one of the drives in the RAID fail, you will be notified by software that the drive needs to be replaced and once you do, the firmware will rebuild the mirror. RAID 1 is NOT an automatic backup!! But in situations where you cannot tolerate downtime, this will keep you running until you can make repairs.
  
  The best best option is to use TWO SEPARATE DRIVES for backup and let Time Machine do the work and eliminate the complexity of the RAID.
  
  Time Machine can be configured to work with multiple drives and alternate backups between them. This way if one of the backups were to fail, the other will be available.
  
  Hope this helps.
Great article, its january 4th and Im sitting here wondering if I need to change my system… a disc has just got filled up, so I need to buy new ones…. I use large storage discs and back them up with chronosync, which is excellent. Many thanks for this piece, it helps to know that Im not the only one with this eternal headache! Mark

http://www.markboltonphotography.co.uk (interior and lifestyle photography)
http://www.warblerdesigns.com (Fine leather camera straps, made in England)
I was about to impulse buy a Drobo and at the last minute I found your article! Thanks! It gave me the pause I needed to do some research first! I'm sticking to my three WD My Passport drives for now.
Thanks for this article – I just read one today suggesting to use RAID – for now, due to the small volume of photos, I use a couple of external drives to save my work.
Nice right up. Raid 5 is not for speed, its actually slow on rebuilds. Raid 0 is for speed then raid 1 then raid 10. Raid 0 if one drive fails you are dead. Raid 1 you can loose one drive and still keep working. Raid 6 and raid 10 can loose 2 drives with 6 being slower on rebuilds. Raid 10 is probably the best for speed with Raid 6 being better for getting more space per drive when you go past 4 drives. None are for backups but for preventing downtime and data loss.

But I think your backup drive should be raid 6 or 10. ZFS files system is also nice to prevent bit rot. Lets say you need your backup and its on just a solo drive and that drive has bad sectors. That backup is no good. That same backup on a good raid with zfs is very unlikely to ever go bad. Of course if the building they are in blows up then its all gone so offsite is smart.
1. I think RAID 1 can be used for redundancy, but not backup. As the author here write, if you delete one file, you delete them both – this is true also of some backup schemes, though also.
  
  I think RAID 1 will beg you back up and running faster than no raid at all. But I am inclined to agree, redundancy is not quite the same as a backup. For day to day recovery from mistakes or misplaced files, a regular backup is better. But I am not so convinced that it replaces RAID 1 for a catastrophic failure. I think both backing up files and RAIDing systems is a best approach.
2. You can't lose ANY two drives in RAID 10… it has to be juuuuuust the "right" two.
  
  And yes – ZFS .. with RAID 60 or 61 … is interesting.
  
  I'd say a mirrored set of the entire 60, and then multiple stripes for the 60 (maybe 3 or 4) — but then! You're already talking the cost per TB of getting NVMe Drives, which have a URE rate substantially lower (1/100th as often) …
  
  However — we'll have to wait about 6 years for NVMe fabric to become affordably available, as NO individual is going to shell out 30k for a home switch and pool, let alone, whatever the approved ACL is for the initial drives… which will be arbitrarily expensive.
  
  At that, Intel is releasing their 3d-Xpoint … which includes Optane, which is about 1500x as reliable as a current SSD drive — IF that statement can be believed! If.
Use a Seagate 1 TB USB drive, built in software allows you to choose the time of day and daily schedule of the backups to the drive. I will be using a 6 drive Raid 10 solution for a t620 enterprise server. I will be using the Seagate for a backup of the sql database. The seagate will be left in a fireproof safe with a hole in the back for wiring.
Pete,

I have just spent the best part of the day looking for a longer term solution to back up my photography. I have learnt a lot about the pros and cons of RAID and proprietary systems to then find your article, the best on the net for my purpose. It also turns out that it is pretty much what I am already doing, though I am doing it manually. What software do you us for your differential backups? (possibly time machine? I am PC so if that is the case I will need to explore some more).
1. Yes, I am using time machine. But there are a lot of good programs out there. Not familiar with the PC world though.
Thanks for this article Pete – PERFECT timing for me and I plan to adapt this myself. I've had a Drobo, it was a nightmare, finally turning into a brick. As I rethink my backup system, I've gotten myself totally confused with RAID 0, 1, 5, etc. I embrace the concept of backup being more than drive failure replacement.

If I use Time Machine to for my main backup, what [software] do you use for your on-site and off-site backups? I thought that TM could only backup a working computer – might you elaborate a bit if you drag and drop, or maybe use another program front the archived backups?

Thanks,

Greg
1. Hi Greg. I do not have an off site backup for the general system. The image files from each year are exported from Lightroom as a catalog to separate externals for off site. And of course all of the "worked up" images are stored on PhotoShelter.
Looking at your diagram, am I correct that you have four separate drives coming off your desktop: one for the system software, one for work files, a third for images for the last two years and then a large 3TB drive holding everything? If so, mind if I ask what type of drives, sizes and enclosures you're using for each of those. It also looks like you've foregone RAID. Are you using Time Machine for backups or making bootable backups with something like SuperDuper?
1. Hi PJ
  
  There are actually 5 drives attached to the Desktop machine, 4 of them are internal to the tower. (The system disk 500MB, work disk 2TB and 2 scratch disks 1TB each). The 3TB backup is external. All drives are Western Digital.
  
  I have a bootable backup of the system disk which is simply another internal drive that is not installed. I can pop it into a docking station that is generally used to diagnose disks if I need to boot from it. Of course since the system disk has nothing on it but the system and the applications, I can also do a pretty fast recover from TIME MACHINE of that drive if needed.
  
  Yes I use TIME MACHINE.
  
  The problem with the RAID 1 as your main drive is this. If you delete a file, it is deleted on the mirrored copy as well. Same issue with corruption. The drives in the enclosure mirror themselves in real time. The only thing a RAID 1 protects against is a crashed drive.
  
  RAID 5 would offer better protection and the RAID 5 draw is speed. But truly, unless you are doing a lot of video or sound work, I don't see the need. AND even though you can recover from a crash, you should talk to people that have had to do so, IT CAN BE PAINFULLY SLOW to rebuild the failed drive. It is doing so by using parity data. It is not a simple copy to reload the data. Of course this drive would also have to be backed up. I am unclear as to whether you can recover a single deleted file from a RAID 5. Its main purpose is speed and ability to rebuild a lost drive.
  
  My laptop is backed up with TIME MACHINE to a portable drive. Here I am mainly concerned about the SYSTEM files and other things like accounting that I have on the laptop. I transfer daily work off each night, so I am not too worried about that. Usually by that time, worked up images are on the way to the client and have been sent to Photoshelter.
  
  Yes the small WD My Passport Drives are great for long term archive storage. I simply buy 2 at the end of every year. If you look for deals at Staples or Office Depot you can usually get one for about $120.00.

How Many Bluray Disks to Back Up 8tb

Source: https://www.petemarovichimages.com/never-use-a-raid-as-your-backup-system/

How Many Bluray Disks to Back Up 8tb

107 Responses

More
articles

0 Response to "How Many Bluray Disks to Back Up 8tb"

Post a Comment

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel

How Many Bluray Disks to Back Up 8tb

107 Responses

More articles

0 Response to "How Many Bluray Disks to Back Up 8tb"

Post a Comment

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel

More
articles