Purpose Built System - Thoughts?

[Gary Bettan]

Guys, an external RAID device using eSata will do a great job for compressed HD video editing. We've got so any customers running G-RAIDs, G SPEEDQ, ForteRAID and Glyph RAIDS every day. They edit ProRes, AVCHD, XD Cam, P2, HDV, Cineform, DNxHD. Long format and pretty complex timelines using FCp, Avid, Premiere Pro, Edius, Vegas you name it.

Some folks need faster throughput. For those we have the G-Speed eS and eS Pro. You pay a premium for a dedicated RAID card, but you get much higher throughput.

Gary

[Steve Kalle]

Those external eSata raid boxes are designed for Mac Pros because they are limited on the number of internal drives.

I would ditch any Hitachi for Seagate 7200.12 or Barracuda XT or WD Caviar Black because they are 10-15% faster.

I would also ditch the Velociraptor for a SSD for many reasons, one of which is speed. A single Intel X25 80GB SSD makes 4 of those V-Raptors in Raid 0 look slow (from personal experience). In addition, the reliability of WD's 10k rpm drives is poor. I have had 4 Raptors die within 3 years of use and 3 V-Raptors die within 2 years of use. There is a HUGE thread over on storagereview.com about reliability issues. Many of those guys over there with problems have used them in large server arrays so its not just a couple computer geeks with one or two drives in their PCs. I used to love these 10k drives and now have 4 Raptors sitting on the shelf and 2 V-Raptors as media cache drives - nothing important will ever reside on them again.

Also, Intel's SSDs have proven to be the most reliable with the largest web servers now using them for a couple years with a much lower rate of failure than the best 15k drives.

I have a total of 5 PCs with an Intel X25 80GB in them (2 are work and one is my mom's laptop). How would you like Premiere and AE and Photoshop and Windows Explorer all opening in under 10 seconds (about 4 seconds for Premiere by itself, <3 seconds for PS by itself, ~4 seconds for AE).

It looks like you have a large budget so I would seriously consider a Lian-Li case - some of them can hold 12 drives. The extra $100 or so spent on the case will be saved by not having to buy an external drive case. Also, Lian-Li's higher end cases have great airflow AND are very quiet. I have one of their older models (a $350 case 5 yrs ago) with 10 drives inside and cannot hear any drives including an 8 drive Raid 5 array (1TB 7200.12s). If you want a case that is loud, look for the ones with large fans on the side. For quiet cases, look for insulation on the sides.

My last recommendation: if you rely on this PC to make a living and you have deadlines and possibly client sessions, then I wouldn't use anything but a hardware Raid controller from 3ware. Intel's software raid like what is built into the motherboard is not as reliable as a hardware raid controller and it cannot notify you if a drive is having problems with alarms and email. I have an Areca 1680ix in one PC and a 3ware 9750 in another (and a 3ware 9690 in a server). Only 3ware's software is able to send me emails with notification of problems (the Areca requires an email server on the computer it is installed in). I recently had a drive starting to die and the 3ware software notified me and I had a new drive by the time it died; thus, I had zero downtime and very little chance of losing data.

Another reason for hardware raid controllers is their ability to fix read and write errors which happen from a computer glitch or a hard drive error like a bad sector.

The last reason is redundancy which you get from Raid 1, 5, 6 and 10. Most likely, you would use Raid 5 as it gives you great read speed and allows a drive to die without losing any data and having any downtime. However, many like to use Raid 0 but you get NO redundancy and if one drive dies, you lose ALL data. 99% of people don't have a rigid backup schedule in place so imagine working on a project all day, and then BAM, a drive malfunctions or dies and all that work is gone. Or imagine being on a tight deadline and without redundancy, you will waste time reloading data and editing. Or if a client is in the room and they are being charged $75+ per hour - they won't like having to wait for you to fix it.

What about external backup drives? I prefer to use Vantec and Antec cases with fans so I can put in new drives as they fill up. I prefer Seagate 5900rpm drives in the retail box because they are the cheapest 2TB with 5 year warranty and are faster than the other green drives. Also, stay away from WD green drives because they spin down all of the time after just 7 seconds of inactivity. Personally, it is soo annoying having to wait several seconds for the drive to spin back up. I have 3 2TB Hitachi's full but one is in the mail to be replaced after just 6 months of use, and I have 4 2TB Seagate LP's on the shelf with 2 more on the way (they are called Seagate Green now).

Just a quick thought on saving money: if you do NOT use After Effects a fair amount, then going with the newer i7-2600k can save you some money, which I would spend on a 3ware 9750-8i and 4 2TB Seagate Barracuda XT, an Intel 510 series 120GB, and a 1TB drive to hold: Page File, Media Cache (from PPro & AE) and encoded video. Something to be aware of with WD drives, they are not designed to work in Raid 1, 3, 5, 6 or 10 and they can randomly disappear and cause data loss. For single drives and Raid 0, they work fine.

Newegg.com - Intel 510 Series (Elm Crest) SSDSC2MH120A2K5 2.5" 120GB SATA III MLC Internal Solid State Drive (SSD)

[Andrew Smith]

Hi Steve,

Why would AE usage make a difference with the i7-2600k? It's still an i7 CPU running at 3.4GHz, isn't it?

Am I missing out on something here?

Andrew

[Blane Nelson]

OK. So if I ditched the external RAID system, and just went with my system drive, plus the 4 x 7200 1 tb drives for scratch, system files, etc, then add 4 more 1 tb drives for long term storage and backup, what would be the best way to interface all these drives with the mobo? I am using this motherboard: Newegg.com - ASUS P6X58D-E LGA 1366 Intel X58 SATA 6Gb/s USB 3.0 ATX Intel Motherboard

- Would I use a hardware RAID controller or can the motherboard handle the task quickly?
- Is this a PCI device?
- How do the drives connect to the RAID controller?
- Can anyone recommend a hardware RAID controller?

I have my eye on a Lian Li PC-P80N. With this case, I could include my archive/backup drives in the case.

- Would the 850W PSU still be usuable?

I should have created this thread before I went out a bought a bunch of stuff. : /

Thank you all for sharing your vast knowledge! It is helping me build the best possible system for sure.

[Andrew Smith]

Blane,

If you are going to have all those hard drives inside your case, then there will be corresponding heat issues. Best to use an external enclosure. The SOHORAID SR4 Enclosure with Hardware RAID 0,5 from VideoGuys is the one that I am getting for my system.

This unit has its own circuitry that does the RAID5 parity calculations and the only bottleneck on the system will be the speed of the hard drives. Just hook it in to a RAID "lane controller" card in your computer ... this is one that I have noticed lately and (subject to double-checking) should do the job. It also comes empty so you can fill it with the hard drives of your choice.

I can't remember the reasoning, but people seem to be not so relaxed with motherboard-based RAID. But you really should have those drives in an external casing outside the computer.

Andrew

[Blane Nelson]

Originally Posted by Andrew Smith

Blane,

If you are going to have all those hard drives inside your case, then there will be corresponding heat issues. Best to use an external enclosure. The SOHORAID SR4 Enclosure with Hardware RAID 0,5 from VideoGuys is the one that I am getting for my system.

This unit has its own circuitry that does the RAID5 parity calculations and the only bottleneck on the system will be the speed of the hard drives. Just hook it in to a RAID "lane controller" card in your computer ... this is one that I have noticed lately and (subject to double-checking) should do the job. It also comes empty so you can fill it with the hard drives of your choice.

I can't remember the reasoning, but people seem to be not so relaxed with motherboard-based RAID. But you really should have those drives in an external casing outside the computer.

Andrew

Thank you, Andrew!

I have already ordered the Glyph ForteRAID external system from Video Guys: Videoguys.com - Glyph ForteRAID Production Quality RAID Array 4TB
Using the Glyph external storage unit has been part of my original plan from the planning stages of this system, but others had recommended against using it for reasons stated in previous posts in this thread .

I think that I'll give it a shot and see how it works.

BTW, the Glyph comes with drives already installed, but can they be replaced with any standard format drive? Thanks!

[Steve Kalle]

Sorry, but that single 4-bay case is not ideal for real everyday work. I would never rely on a no-name company for something as critical as my working data. Also, this case is limited by the eSata connection AND the cheap IOP chip processing raid 5. In addition, with Raid 5, the quality of the raid controller is even more important due to how complex calculating the parity is. When a drive dies or just malfunctions causing it to be dropped from the array, you put in a new drive to begin rebuilding. With quality raid controllers, this will take a few hours per TB but with cheap raid, this rebuilding process can take days. In the meanwhile, a simple read error from a drive will cause ALL data to be lost. With consumer drives, there is a read error rate of once every 10TB - with a 4TB raid 5 array, far more than 4TB will be read during the rebuild process due to the way parity is calculated.

Saying that having many drives in a case will cause heat issues is completely untrue. Five years ago, maybe, but with today's 2 platter 1TB drives, there is no problem whatsoever.

Another reason why I like drives inside one case is due to all the extra parts in an external case: backplane, cables, Power Supply (always of the cheap kind), cheap fans, cables going from the drive backplane to the outside connectors (eSata, SAS, etc). I have purchased several external cases and have had everything from the fans to the drive backplane to the outside SAS connectors die on me. Aside from the irritating sound from bad fans, the other bad parts cause raid arrays to DEGRADE and even FAIL. Unless you have tons of spare parts like me, troubleshooting the exact problem is a nightmare.

About the i7-2600k and AE: AE likes more cores so the i7-970/980/990x with 6 cores is much faster than the 4 core 2600k.

Last thing I will say about storage is ALWAYS have your backup drives in external cases separate from the PC. Also, ALWAYS use a quality UPS to prevent damage to your computer from brownouts, surges, dropouts, etc. Several years ago, I made many mistakes due to thinking that none of these problems would happen to me. Then a brownout occurs and with no backup, I spent roughly 30 hours fixing my 1TB Raid 5 array and getting back 90% of the data. I ended up having to create a Virtual Machine and use a recovery program within the VM - at that time, this was a very difficult task due to how young the VM software was.

The catch-22 about high efficiency PSUs is that they need a very expensive UPS to run them which uses 'Pure Sine Wave'. At work, I have almost $2k in Pure Sine Wave UPS's for the computers. At home, I found out that my dual-battery APC is not PSW because it caused my LCD TV to make a horrible whizzing sound when I lost power for a few hours a while back. Also, HP issued a notice to customers a few years ago that most of their computers with high efficiency PSUs require a PSW UPS otherwise the PC would shut off during a power loss.

[Steve Kalle]

Originally Posted by Blane Nelson

OK. So if I ditched the external RAID system, and just went with my system drive, plus the 4 x 7200 1 tb drives for scratch, system files, etc, then add 4 more 1 tb drives for long term storage and backup, what would be the best way to interface all these drives with the mobo? I am using this motherboard: Newegg.com - ASUS P6X58D-E LGA 1366 Intel X58 SATA 6Gb/s USB 3.0 ATX Intel Motherboard

- Would I use a hardware RAID controller or can the motherboard handle the task quickly?
- Is this a PCI device?
- How do the drives connect to the RAID controller?
- Can anyone recommend a hardware RAID controller?

I have my eye on a Lian Li PC-P80N. With this case, I could include my archive/backup drives in the case.

- Would the 850W PSU still be usuable?

I should have created this thread before I went out a bought a bunch of stuff. : /

Thank you all for sharing your vast knowledge! It is helping me build the best possible system for sure.

This is what I would get and from this place due to their excellent tech support and service. I get all raid equipment from them now for those reasons. I had purchased an external 4-bay Sata case and had problems with the backplanea year later. They let me return it for the more expensive SAS case and only pay the difference. They said that they almost stopped selling ALL sata backplanes due to poor reliability. Ever since then, I have pirchased only high quality SAS backplanes and have had zero issues.

3ware 9750-8i "+getMessage("iPrintVerKit")+"

and this for cables from the raid controller to the drives "+getMessage("iPrintVerKit")+"

IIRC, that Lian-Li case has rubber contacts for the hard drives or some way of isolating or absorbing drive vibrations, which is a great feature.

Watch this video to see why I like that feature.
Shouting in the Datacenter - YouTube
a guy yells at a 16 drive array and then shows you how much those drives slowed down due to the vibration of his yelling. He talks in geek, but the conclusion is vibration hurts the performance of drives (and hurts the reliability as well).

I have a 750w PC P&C running 10 drives, GTX260, P6T, i7 920, 3ware raid controller and a few other things. In my HP Z800, a 850w PSU runs 2 6-core CPUs, 24GB ECC ram (draws more power than regular ram), 4 internal drives & 1 SSD, FX3800, raid controller, BM Decklink Extreme 3D and a few other things.

So, yes 850w is enough.

[Andrew Smith]

(Sigh.) There's no end to the detail involved in building a decent edit computer.

Andrew

[Steve Kalle]

Yep. Even though I don't like spending extra money on something I can easily build, I prefer buying customized PCs and servers from reputable companies like HP and their Z line of workstations. When I am on a deadline and/or have a client in the edit bay and something goes wrong, I absolutely must have someone to call to help me fix the problem. Because of this, I am willing to pay extra for better support and HP has been excellent.

This same reasoning explains why a business pays $12k for an Autodesk 12-drive raid system when they can get almost the exact same thing for half the price, but it doesn't come with the support of Autodesk (which is one of the best).

Another reason I like a PC builder is I have only 1 person to call if I have a problem whereas you must call every single manufacturer whether its the mobo, GPU, ram, raid controller if you build your own PC. And then you have so much fun listening to these manufacturers blame other parts for the problem - like I did when ASUS blamed PNY and the GPU and then PNY blamed the PSU. Thus, none of them would fix anything. With HP, I call or go online for support and the next morning, I have someone there or something in the mail. I thought that I replaced something properly in my Z800 but I screwed it up so the mobo had to be replaced. They even let me diagnose the problem so I didn't have to sit there for hours with them telling me to make sure the power is on.

What it all comes down to is when you are making a living from editing, your perspective should change.

[Blane Nelson]

Needless to say, my head has just about spun off due to all of these replies. All of them have been great perspectives btw!
As of last weekend my build is complete(OP build) and all is working well with PP5.5. I have the 4 1tb drives working as 2 RAID 0's through Windows which I know is not the best way to do it, (the mobo wouldn't do two separate RAID configurations). And the Glyph external RAID is running in a RAID 5.

I am very happy with the system so far, but I know that I can do better in the hard drive department.

Should I be using a PCIe hardware RAID controller for the internal hdds? I think yes.
What else can I do with the external

Should I configure the external drives some other way for better reliability?

I have tried to follow all the advice that I could regarding this system so I figured a way to add an UPS to the budget. Now learning that it should be a PSW UPS, I'm pretty sure that the one that I chose doesn't fit into those guidelines.
I bought this one from Costco: http://www.costco.com/Browse/Product...=1&topnav=&s=1
Will this work OK for me if I'm not trying to run the PC for prolonged times off of the batery?

You guys are great!

[Steve Kalle]

Originally Posted by Blane Nelson

Needless to say, my head has just about spun off ...

Ah ha....that was my GOAL!

What size drives do you have in the external box? If 2TB, then I would go with Raid 10 which eliminates parity calculation AND gives you protection from 2 drives going bad without losing data (technically, due to the way Raid 10 works, which is taking 2 pairs of drives as Raid 1 and striping data between the 2 pairs, if you lose both drives as part of pair 1, then all data is lost. But if one drive from pair A dies and another from pair B, then you are fine. This happened to me a couple years ago with 10k rpm Raptors and I didn't lose any data). Also, if a drive dies with R10, the rebuild process is much faster because there is no parity calculation occurring - its simply copying data from one drive to the new one.

Also, R10's random access speed is much faster than R5. In addition, R10 is even faster when there are more than one read and write at the same time. With R5, a single write requires a read, then a write and then another read. With R10, a single write requires only one write to the disks.

Another reason I like hardware raid controllers and even Intel's software raid with Raid 1 and Raid 10 is because they are able to increase read speeds and lower random access. This works by reading data from all drives simultaneously rather than just one drive.

I find it very odd that you could not create multiple Raid 0 arrays. Did you use the Control + I during bootup to go into the Intel Matrix Storage manager and did you set the drives to raid within the BIOS? I have the almost exact same (P6T) mobo and have setup multiple arrays. Nothing has changed between my mobo and yours because they use the exact same ICH10R chipset. Also, did you plug all 4 drives into the correct Sata ports? I believe that 2 are Sata 6 which is controlled by a separate chipset (Marvell maybe?) and 6 are controlled by the Intel ICH10R.

I would definitely redo your 4 drives and create their Raid 0 arrays with the Intel Raid. You can download Intel's Matrix Raid manager and it will show you which drives and other devices are plugged into the Intel ICH10R Sata ports, but it won't show anything for Sata ports controlled by other chipsets (ie Marvell).

As I understand it, you have your video and project files on the external box. What are you using for the OS and how are you using the 2 Raid 0 arrays?

For me, there are two areas which are very important: the drives holding work data and the OS drive. With work data, I don't want to work on a project all day or across a couple days, forget to backup everything and then have a drive die or malfunction; thus, losing all that time. So, I always use redundancy which is why I use Raid 5 or Raid 10 on all of my PCs. My home workstation has 8 1TB drives in R5 - Seagate 7200.12 connected to a 3ware 9750; our broadcast server has Seagate Constellation in R5 with a 3ware 9690 controller - both are setup to verify data every week and email me with all status updates, warnings and errors. I was able to buy and replace a failing drive by the time it died because the 3ware raid controller emailed me about some sectors that went bad.

Finally, the OS drive is important because without a working OS drive, you cant WORK. I had always used either Raid 1 or Raid 10 for the OS drive, and then came along the Intel X25 SSD. This SSD makes my prior Raid 10 with 4 10k rpm Raptors feel utterly slow. Plus, SSDs tend to be more reliable than hard disks and Intel is at the top of SSD reliability.

On my main workstation (HP Z800), I have:
A) 7 2TB drives (enterprise class Seagate Constellation) in Raid 5 connected to an Areca 1680ix controller plus BBU (battery backup) with all documents, video, audio, stock images/audio/video, PPro project files, AE project files, Cinema 4D projects & renders, etc...
B) 4 1TB drives in Raid 10 via onboard Intel Raid for encoded files and Media Cache
C) an 80GB Intel X25 SSD for OS/Programs

FYI, another cool raid tidbit I like about the Intel Matrix raid is the ability to take 2 drives and create 2 DIFFERENT raid arrays on them. After 2 of my Raptors died, I changed the OS drives to Raid 1 and then created a Raid 0 array with the same drives for media cache. So, with 2 150GB drives, I had 80GB from each for Raid 1 & the OS and used the remaining ~160GB for a Raid 0 array. You can do the same thing with 4 drives and use Raid 10 for the OS and Raid 0 for the media cache & page file which is what I had before my Raptors began failing.

With your external raid box holding important data, I don't see a need for a hardware raid controller inside your PC because the onboard Intel raid works good enough.

Just one last suggestion from me: for backups, I buy only Seagate's 2TB 5900rpm drives now called "Green" (formerly "LP"), BUT you must make sure to get the Retail version in a box because it comes with a 5yr warranty whereas everyone else includes only a 2 or 3yr warranty. I just got 2 more today for $59.99 from Microcenter Micro Center - Seagate Barracuda LP 2TB 5,900 RPM SATA 3Gb/s 3.5" Internal Hard Drive ST320005N4A1AS-RK ST320005N4A1AS-

[Blane Nelson]

Steve - that was a wonderful reply! Thank you!

Now, to answer some of your questions and give you a little more info.

The external box contains 4 x 1tb Seagate 7200.12 drives. These are configured in a R5 through the Glyph manager software. They are connected to the computer through a PCIe card via an eSATA cable.
(An interesting tidbit: When the Glyph box is connected via USB, the Glyph Manager software reads vital info from the drives. When connected via eSATA this info is not available. I hope that this still means that all alarms and the R5 system will still work.)
I think that I'll configure these 4 drives as a R10 as you suggested.

Inside the tower I have the one 600G VRaptor for the system and apps that is hooked to one of the 6gb/s SATA ports.
The other 4 drives are Hitachi 7200 1tb drives connected to 4 of the 6 regular 3gb/s ports.
These are the drives: Newegg.com - HITACHI Deskstar 7K1000.C 0F10383 1TB 7200 RPM 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive

I tried to configure two internal raid 0 volumes via the Intel Matrix Manager, but I was only able to create 1 of the volumes. The options to create another were grayed out. Also, before I did this, I did set these 4 drives to RAID in the bios.
I ended up using a (workaround) through the MS disc manager where I selected drives 0 and 1 and striped them, and then did the same thing with drives 2 and 3.
I would really like to set them up properly through the Matrix Manager though, so any help with this would be appreciated. I'll also google this again and maybe go back into Matrix Manager a poke around a bit more.

"As I understand it, you have your video and project files on the external box. What are you using for the OS and how are you using the 2 Raid 0 arrays?"

The OS and other apps are on the 600g Raptor. I would love to RAID the system drive for redundancy, but that's not going to happen right now. The same goes for SSD's. They are awesome and fast, but I have already spent just about 5 grand on this system so far including software, so those upgrades will have to wait a bit.
Also, the tidbit about Matrix being able to creating a raid on one drive is very interesting. How could I take advantage of this feature?

I don't have any real media on any of the other internal or external drives yet, but following is how I planed to use them:
External drives: Long term storage for media files, saves and completed projects. I don't plan to actively pull or write data from these drives while I am editing.
Internal R0s: Used for scratch files, page files, storing temp media while I edit it. In all honesty, I don't really know the best way to use these two RAID 0's, so any advice would be great!

I also don't really know how to manage backup of files besides the auto save feature in PPro, so any advice on this would be useful as well.

Thank you again for the help. I look forward to your next answer/installment.

[Andrew Smith]

Needless to say, my head has just about spun off ...

Kinda makes you feel welcome to the community, eh? :-P

Andrew

[Blane Nelson]

Originally Posted by Andrew Smith

Kinda makes you feel welcome to the community, eh? :-P

Andrew

Haha! :) Actually, yes it does. Some folks that have replied to this thread have left some lengthy answers full of great content, and I appreciate that very much. Thank you all for the help thus far!

Update:

Yesterday afternoon I got back into the Intel RAID utility and managed to configure the 1tb hdds in the tower to two R0's. The confusing part for me last time was that when I tried to configure that second RAID volume, the program wasn't letting me choose what disks to include in the raid. Let me explain this a little better: when I created the first volume, I selected drives 0, and 1. Then I selected the other parameters followed by the creation of the volume. When I began creating the next volume, the option to choose what drives to include in the second volume were unavailable. I suppose it was at this point that I gave up on the Intel raid for the time being.
After looking at it again yesterday, I came to the understanding that the reason the Intel Matrix program wasn't allowing me to choose what drives to use to create the second volume was because there were only two drives left to use, and the software is smart, so it already knew that.

So now the 4 1tb drives in the tower are in a proper RAID0 x 2, and the external 4 x 1tb Glyph box is in a R10.

I could still use some information on how best to use these two R0s with Windows 7 and the Adobe Production Premium software.