Best storage solution for wedding videographers

[Luc Spencer]

Hey guys! First off, I apologize if this matter has been discussed, I briefly browsed the forum to look for it but did not find anything relevant.

Last summer I upgraded my PC to an i7 4790k and 16GB of RAM (for those who don't know already, 16 gigs is NOT enough to work on a lengthy wedding in Premiere, I sometimes get errors that I'm running out of memory) and I also paid about $300 for 3 Western Digital 2TB *Red* HDDs out of which I made a RAID5. It seemed like the best idea to have 4 out of 6TB of space and not losing anything if either HDD failed.

As for Premiere, it's on a SSD along with Windows. Here's my problem.

I'm using the mobo's RAID controller which apparently is producing very slow speeds for my RAID5. Even when I'm copying footage off a memory card, sometimes the transfer speed goes down to 30-40 MB/s even though my card is capable of 60. I noticed that if I turn off buffer flushing in Windows, speed goes up to around 80-90 MB/s, but in the event of a power surge, I'm screwed (data is corrupted/lost).

I would just like to know what you guys use to store a lot of wedding footage (I have filled my 4TB already), if you work directly from the storage HDDs (which I do), and if you think that getting a ~240GB SSD just for my currently active project would be a good idea.

For some reason, as I go past an hour or so in my timeline in Premiere, everything feels sluggish. For instance, if I apply a fast color corrector to a clip, it takes maybe 5-6 seconds to see any modification appear in the monitor. I'm hoping that it's a HDD issue that will be resolved if all my footage is on a SSD.

Any input is greatly appreciated. Thank you!

[Rob Cantwell]

you dont say what video card your using
I have a similar system except my processor is i7-4770 CPU @ 3.40GHz i do daily backups to external drives and one internal (slow) green WD drive. Yeah my system could do with more RAM, I dont apply a lot of effects so it chuggs along good enough.
I wouldn't edit from storage at all they are just for storage and nothing else, i eventually archive projects to BD and free up HD space that way. I was looking at getting an SSD which should improve performance.

[Luc Spencer]

Sorry, my video card is a GTX 760 2GB. During those "freezes", that period that I have to wait to see the applied effect to my video - I looked at the GPU usage and it's nothing. I assumed it doesn't play a key role. CPU is barely used, too. That's why I'm looking at my HDDs.

By the way, I recently got a GH4 and started shooting a bit of 4k, especially where low light was an issue (such as some receptions). I'm guessing it would be wiser to get a SSD just for the wedding I'm working on, render the final cut, move it to a storage HDD and then clear the SSD. Rinse and repeat. I know these SSDs can handle 70-80 TB of data traffic so I don't have a problem with replacing it every few years.

[Jeff Harper]

You don't want to work on slow storage drives for editing, you want a drive ONLY for editing. Try another SSD.

As for storage, delete anything you no longer need.

You need a drive for your current video
You need storage for holding pending projects
You need separate storage for archived projects.

For archived I use WD Passports with USB 3 and I detach them when not in use.

Total I have six drives, everything has a backup.

Running raid off of a MOBO controller is not good idea, it's a waste. Just get fast drives and backup everything manually, it's simpler, cheaper and faster.

Also, I suspect the RED drives are no good for raid anyway, but I could be mistaken.

[Luc Spencer]

Thanks for your input Jeff. I'm very nearly about to order a new SSD. Excited about it, too. I shall be sticking with Samsung since the one I have for my OS has been treating me very nicely.

The Red series from WD is specifically made to be used in RAID. That is the only reason why I chose Red instead of Black. If you look on WDs website it even says that the Blacks are not made for RAID purposes, but I did hear of people who run them that way. Then again, I guess you can run any 2 identical HDDs in RAID. It's just that the reds are made to run 24/7 and *should* last longer than a Black. They are also made for use in a NAS.

[Jeff Harper]

My mistake on the Reds, I used to be current on RAID stuff, but I haven't run RAID for two years. I ran it for ten years and am so glad to be done with the controllers and ridiculous boot times.

I ran Seagate Cheetah 15K SAS drives and they were really nice, I do kind of miss the speed, I like them better than my SSDs, my system has never been as stable since I went to SSDs for some reason.

[Peter Rush]

I have 16GB Ram in my i7-2600K and have no problems with Premiere - and that's a long Catholic wedding filme complete with mass - well over 2 hours on the timeline and 3 or 4 video tracks and the same number of audio tracks

Premiere requires you edit from a drive capable of 7200rpm - I'm not sure your RED drives are up to this, pretty sure they are 5400rpm - I use 2 X Seagate Barracuda drives for directly editing from have had no problems

Pete

[Noa Put]

For reliable and fast raid performance you really need to get a dedicated raid card, also red drives are 7200rpm, wd black drives are not designed for raid as they can result in dataloss, you need to get the WD re or WD red (pro) discs who are designed for nas systems.

I don't have a raid system, I use several separate internal and external discs instead, my videodata can be on disc A and when I export out from my NLE this is done to disc B, if I then want to continue working and build a dvd menu from another project I make sure those files are on disc C. As long as you dedicate discs for specific tasks you get the best performance, if you write to disc A and simultaneously try to read another file from that same disc performance drops considerably.

[Noa Put]

Originally Posted by Luc Spencer

For some reason, as I go past an hour or so in my timeline in Premiere, everything feels sluggish. For instance, if I apply a fast color corrector to a clip, it takes maybe 5-6 seconds to see any modification appear in the monitor. I'm hoping that it's a HDD issue that will be resolved if all my footage is on a SSD.

I have used premier up to cs3 and back then noticed exactly the same problem, the longer my projects where the slower and unstable my pc became, not sure if a SSD will solve your problems as a 7200rpm disc is plenty fast to handle today's codecs, unless you have a very high bitrate one where raid discs are needed. I"m sure if you just get a WD black disc instead of the discs connected to your mobo's RAID controller you wil see a performance gain, your problem is most likely caused by the RED drives you have in raid.

[Luc Spencer]

like you said Noa, the Red is a 7200rpm drive. it's not that slow either, just about 25% slower than a Black. and the Black is not that much cheaper than a 240gb SSD. i'm still leaning towards the SSD, especially considering the write speed when i export a 20gb wedding :)

[Noa Put]

Yes, but could it not be that the raid setup you have is causing the problems? Also when comparing price of SSD vs HD, isn't it a fact that you can get much more storing space with HD for the same price? I have also read articles that a SSD is not that suitable to constant write and read data to like you have when dealing with editing data and that it is more likely to fail in a much shorter time period then with a HD.

It also is not sure your export speeds will increase as it will be the processor that is the bottleneck depending which codecs you are dealing with.

[Craig McKenna]

Currently, I use a RAID 0 configured editing based solution, which is a 4TB from G-Tech. I am most likely going to upgrade in the coming year, so that I have a redundancy with a drive running with a Thunderbolt 2 connection to either a Mac Pro or iMac 5K.

Then, the drives are backed up to a Synology 20TB NAS, which runs in RAID 5 and allows for one HDD to fail. I have the industry standard drives inside those (Red), and that NAS is protected via a battery that plugs into the wall, and gives the server around 40 minutes to power down in the event of a power cut - which has happened once thus far.

This was a recommendation by Danny, who used to post regularly here.

Ultimately, we're at a very difficult time with technology vs storage at the moment.

If I can get a 12TB RAID to work alongside my NAS, I will likely rid most of my weddings RAW footage following 2 weeks of delivery.

[Luc Spencer]

Alright, so I received a few ideas from you guys, for which I thank you. Here's what I currently have and what I intend to do based on your feedback:

SSD for OS and Adobe CC
3 x WD Red 2TB in RAID5 (containing both active and archived projects)

What I intend to do:

- SSD for OS and Adobe CC
- 2 x WD Red 2TB in RAID0 or RAID1 (still debating this - I might go for 0 for a few years then switch to 1 for safety. either way, the performance will be greater than that of a single WD Black)
- 1 x WD Red 2TB for EXTERNALLY backing up active / unfinished projects (will buy a USB3 HDD rack for it) - this will be kept in a drawer most of the time
- BUY a 3 or 4TB HDD (thinking about WD Green) to be used as an archive

Thoughts?

[Daniel James]

Originally Posted by Luc Spencer

What I intend to do:

- SSD for OS and Adobe CC
- 2 x WD Red 2TB in RAID0 or RAID1 (still debating this - I might go for 0 for a few years then switch to 1 for safety. either way, the performance will be greater than that of a single WD Black)
- 1 x WD Red 2TB for EXTERNALLY backing up active / unfinished projects (will buy a USB3 HDD rack for it) - this will be kept in a drawer most of the time
- BUY a 3 or 4TB HDD (thinking about WD Green) to be used as an archive

Thoughts?

Is the raid on the motherboard?

With regard to your data, work on the basis that any hard disc WILL fail at some point, so make sure you have more than one copy of any work. I personally wouldn't just rely on Raid 1 backups.

You would need to do some testing to see if Raid 0 is of any benefit on your setup, if you are using SATA3 / 6Gbs you many not see enough benefit to offset the worry.

Raid 1 could mean a reduction in performance on your system as everything has to be written twice.

[Luc Spencer]

Yes, the RAID will be managed by the mobo. I had a RAID0 on my previous system before and it was very fast. I've been told that RAID5 is a whole different matter, though, and that reverting to RAID0 or 1 would give me significant performance increases.

I would be relying on the RAID1 as backup for my active projects AS WELL AS the external HDD. So 2 copies of the work that is not done.

I know RAID1 is slower, but I still think it would be much faster than what I have right now. I attached a benchmark that I just ran, I think it's safe to say those are not normal values for Red drives. My 4TB RAID5 is almost full, I am aware that does slow things down, but the scores are still unacceptable.