High-Def (HDV) drive solutions? at DVinfo.net

Robert Jackson · August 29th, 2003, 07:50 PM

I have a dual 1 GHz Quicksilver machine with a pair of internal 80 Gb 7200 rpm drives. That's done fine for me with Mini-DV editing projects. Next year I plan to work with the new HDV standard, providing some new cameras pop up at the next CES. I'm trying to decide what kind of storage solution will be best suited to that level of video editing. My intent is to capture the compressed video stream and convert it to an uncompressed format for editing, so I imagine the demands on the drive system will be fairly heavy.

Does anyone have any thoughts on this? I have some time to think about it and no projects to do this semester, but I'd like to be ready when the time comes around a few months from now.

-Rob

Jeff Donald · August 31st, 2003, 07:15 AM

I would wait to see how feasible that is before jumping into HD uncompressed. The new standard is using MPEG2 and the need for going to uncompressed may not be justified or desirable. However, you may just want faster drives. In that case I would suggest exploring some of the RAID 0 options if you crave speed. I would lean toward FireWire 800 (either with or without RAID) as that seems to be Apple's emphasis for the next year or two. If you want greater speed look at SCSI RAID 0 or the Apple Xserve. Just out of curiosity, once the project is finished, how are you going to get it off your drives? If you recompress it to FireWire to go back to tape, I'm not sure you'll have gained much.

Robert Jackson · August 31st, 2003, 07:28 AM

<<<-- Originally posted by Jeff Donald : I would wait to see how feasible that is before jumping into HD uncompressed. The new standard is using MPEG2 and the need for going to uncompressed may not be justified or desirable. However, you may just want faster drives. In that case I would suggest exploring some of the RAID 0 options if you crave speed. I would lean toward FireWire 800 (either with or without RAID) as that seems to be Apple's emphasis for the next year or two. If you want greater speed look at SCSI RAID 0 or the Apple Xserve. Just out of curiosity, once the project is finished, how are you going to get it off your drives? If you recompress it to FireWire to go back to tape, I'm not sure you'll have gained much. -->>>

Well, my assumption, and these are just assumptions since I have no experience with the format, is that working with compressed video may be problematic during editing. I wasn't thinking of uncompressed video as a way to gain any kind of quality that wasn't there when the footage was shot, it was so there wouldn't be a compression codec grinding away while I was editing. I'll probably screen directly from DVD when the project is done and back up the files once I'm done with editing by burning them to DVD, as well. Chances are I'd just burn the compressed MPEG-2 files to DVD as my backup, though, and only use the uncompressed footage while in the process of editing.

I've read people's opinions on using RAID for audio editing where they claimed it wasn't a great idea. Glyph actually has a little blurb about it right on their site. I keep wondering if there's some way to spread video out on a RAID array (for the sake of speed) and audio on an IDE drive or if that's even really necessary. It seems to get complicated at this level of performance. It's kind of a step past where I'm at with my little Mini-DV projects. Still easier than doing student films on 16mm, I imagine, but I figured I'd start asking around and getting opinions ASAP before I have to find it all out through trial and error.

-Rob

Jeff Donald · August 31st, 2003, 09:16 AM

The problem is the Codecs for doing this aren't even written yet. Apple (or someone) will have to write the Codecs to deal with this new format. So, you are way ahead of the game, so far ahead that an answer really isn't possible. FCP can't really edit the new JVC HD format, although several people are working on a solution. By the time new cameras for the new format are introduced, FireWire 1600 could be available and your investment in drives would be for naught. My advice would be to wait and see what solutions the manufactures offer.

Dean Sensui · August 31st, 2003, 10:55 PM

Robert...

For an empirical review of Firewire, Ultra-SCSI and ATA RAIDS, take a look at barefeats.com.

http://www.barefeats.com/hard29.html

Besides giving tests results, Rob also posts his opinions in terms of pure performance and performance vs. price.

Dean Sensui
Base Two Productions

Robert Jackson · September 1st, 2003, 07:36 AM

<<<-- Originally posted by Dean Sensui : Robert...

For an empirical review of Firewire, Ultra-SCSI and ATA RAIDS, take a look at barefeats.com.

http://www.barefeats.com/hard29.html

-->>>

That's actually kind of a depressing picture of the situation, as it stands right now. The Ultra320 SCSI array was strangled by the slow G4 PCI bus and the excellent performance of the Firewire800 array came at the cost of installing three Firewire PCI cards in addition to using one of the existing Firewire ports, essentially making the machine useless as a ProTools chassis. The four-channel ATA RAID setup wasn't too shabby, but requires a machine with bays to accomodate five internal drives and my Quicksilver only has room for three.

This probably means that the X-Serve RAID solution will be just as hampered by bus speed on the G4. It's starting to look like the G5 machines will be a necessary upgrade for people working in HDV editing.

Thanks for the link!

-Rob

Jeff Donald · September 1st, 2003, 07:49 AM

That's why I said it's too early to tell. The HDV standard won't even be finalized to the end of this month. Editing in uncompressed formats is very expensive and may offer very little quality improvement over HDV for most applications.

Robert Jackson · September 1st, 2003, 07:56 AM

<<<-- Originally posted by Jeff Donald : That's why I said it's too early to tell. The HDV standard won't even be finalized to the end of this month. Editing in uncompressed formats is very expensive and may offer very little quality improvement over HDV for most applications. -->>>

Well, it wasn't really an issue of quality. I don't expect any kind of improvement in quality by editing uncompressed. My limited experience has been that working with compressed video lends itself to trouble. Corrupted files popping up and glitching, etc. Who knows, though, the failure rate of this new video standard may not be as bad as previous MPEG-2 solutions. It does worry me, though. The compression is necessary at the capture stage to allow a Mini-DV tape to contain the data, but it seems like the sooner that footage is moved to a workstation and uncompressed, the more reliable the editing experience will be.

-Rob

Dean Sensui · September 1st, 2003, 06:29 PM

Robert...

Just FYI, I've been working with a Media 100 system for the last several years and haven't had a problem with compessed files at all.

Of course M100 compression works through hardware, so that might be part of the reason for its success. Perhaps that might provide a helpful clue in how to avoid the kind of problems you're facing?

Good luck
Dean Sensui
Base Two Productions

Glenn Chan · September 1st, 2003, 07:21 PM

<--- This probably means that the X-Serve RAID solution will be just as hampered by bus speed on the G4. It's starting to look like the G5 machines will be a necessary upgrade for people working in HDV editing. --->

People are using the XSERV RAID right now to edit uncompressed, so it isn't a problem. The G5 has even worse storage than the G4 (ignoring the PCI-X instead of PCI expansion slots) as it has less PCI slots and only 2 drive bays.

Robert Jackson · September 1st, 2003, 07:37 PM

<<<-- Originally posted by Glenn Chan :

People are using the XSERV RAID right now to edit uncompressed, so it isn't a problem. The G5 has even worse storage than the G4 (ignoring the PCI-X instead of PCI expansion slots) as it has less PCI slots and only 2 drive bays. -->>>

I had no idea that people were editing uncompressed HD video with the X-Serve RAID already. I'd love to read some feedback from some of them if you can point me in the right direction.

As for the G5 storage issue, the speed of the PCI bus is really the issue. If you're using an external RAID array then the number of internal drive bays is kind of a non-issue, IMO. The G5 DP has five PCI slots of varying configurations, BTW. Not too shabby, really, especially with the speed boost.

Dean's M100 experience may actually hold the final answer, though. I wonder if we can look forward to hardware-accellerated editing for HDV MPEG-2 files in the coming months? That might solve both the drive-bandwidth problems and the problems regarding file corruption/glitching when editing compressed video.

Waiting for someone else to do the trailblazing on this issue is the biggest problem since patience is my least abundant resource. ;-)

-Rob

August 29th, 2003, 07:50 PM	#1
Robert Jackson Inner Circle Join Date: Jun 2003 Location: Santa Rosa, CA Posts: 390	High-Def (HDV) drive solutions? I have a dual 1 GHz Quicksilver machine with a pair of internal 80 Gb 7200 rpm drives. That's done fine for me with Mini-DV editing projects. Next year I plan to work with the new HDV standard, providing some new cameras pop up at the next CES. I'm trying to decide what kind of storage solution will be best suited to that level of video editing. My intent is to capture the compressed video stream and convert it to an uncompressed format for editing, so I imagine the demands on the drive system will be fairly heavy. Does anyone have any thoughts on this? I have some time to think about it and no projects to do this semester, but I'd like to be ready when the time comes around a few months from now. -Rob

August 31st, 2003, 07:15 AM	#2
Jeff Donald Warden Join Date: Mar 2002 Location: Clearwater, FL Posts: 8,287	I would wait to see how feasible that is before jumping into HD uncompressed. The new standard is using MPEG2 and the need for going to uncompressed may not be justified or desirable. However, you may just want faster drives. In that case I would suggest exploring some of the RAID 0 options if you crave speed. I would lean toward FireWire 800 (either with or without RAID) as that seems to be Apple's emphasis for the next year or two. If you want greater speed look at SCSI RAID 0 or the Apple Xserve. Just out of curiosity, once the project is finished, how are you going to get it off your drives? If you recompress it to FireWire to go back to tape, I'm not sure you'll have gained much. __________________ Jeff Donald Carpe Diem Search DVinfo.net for quick answers \| Where to Buy? From the best in the business: DVinfo.net sponsors

August 31st, 2003, 09:16 AM	#4
Jeff Donald Warden Join Date: Mar 2002 Location: Clearwater, FL Posts: 8,287	The problem is the Codecs for doing this aren't even written yet. Apple (or someone) will have to write the Codecs to deal with this new format. So, you are way ahead of the game, so far ahead that an answer really isn't possible. FCP can't really edit the new JVC HD format, although several people are working on a solution. By the time new cameras for the new format are introduced, FireWire 1600 could be available and your investment in drives would be for naught. My advice would be to wait and see what solutions the manufactures offer. __________________ Jeff Donald Carpe Diem Search DVinfo.net for quick answers \| Where to Buy? From the best in the business: DVinfo.net sponsors

September 1st, 2003, 07:49 AM	#7
Jeff Donald Warden Join Date: Mar 2002 Location: Clearwater, FL Posts: 8,287	That's why I said it's too early to tell. The HDV standard won't even be finalized to the end of this month. Editing in uncompressed formats is very expensive and may offer very little quality improvement over HDV for most applications. __________________ Jeff Donald Carpe Diem Search DVinfo.net for quick answers \| Where to Buy? From the best in the business: DVinfo.net sponsors

August 31st, 2003, 07:28 AM	#3
Robert Jackson Inner Circle Join Date: Jun 2003 Location: Santa Rosa, CA Posts: 390	<<<-- Originally posted by Jeff Donald : I would wait to see how feasible that is before jumping into HD uncompressed. The new standard is using MPEG2 and the need for going to uncompressed may not be justified or desirable. However, you may just want faster drives. In that case I would suggest exploring some of the RAID 0 options if you crave speed. I would lean toward FireWire 800 (either with or without RAID) as that seems to be Apple's emphasis for the next year or two. If you want greater speed look at SCSI RAID 0 or the Apple Xserve. Just out of curiosity, once the project is finished, how are you going to get it off your drives? If you recompress it to FireWire to go back to tape, I'm not sure you'll have gained much. -->>> Well, my assumption, and these are just assumptions since I have no experience with the format, is that working with compressed video may be problematic during editing. I wasn't thinking of uncompressed video as a way to gain any kind of quality that wasn't there when the footage was shot, it was so there wouldn't be a compression codec grinding away while I was editing. I'll probably screen directly from DVD when the project is done and back up the files once I'm done with editing by burning them to DVD, as well. Chances are I'd just burn the compressed MPEG-2 files to DVD as my backup, though, and only use the uncompressed footage while in the process of editing. I've read people's opinions on using RAID for audio editing where they claimed it wasn't a great idea. Glyph actually has a little blurb about it right on their site. I keep wondering if there's some way to spread video out on a RAID array (for the sake of speed) and audio on an IDE drive or if that's even really necessary. It seems to get complicated at this level of performance. It's kind of a step past where I'm at with my little Mini-DV projects. Still easier than doing student films on 16mm, I imagine, but I figured I'd start asking around and getting opinions ASAP before I have to find it all out through trial and error. -Rob

August 31st, 2003, 10:55 PM	#5
Dean Sensui Inner Circle Join Date: Oct 2001 Location: Honolulu, HI Posts: 2,054	Robert... For an empirical review of Firewire, Ultra-SCSI and ATA RAIDS, take a look at barefeats.com. http://www.barefeats.com/hard29.html Besides giving tests results, Rob also posts his opinions in terms of pure performance and performance vs. price. Dean Sensui Base Two Productions

September 1st, 2003, 07:36 AM	#6
Robert Jackson Inner Circle Join Date: Jun 2003 Location: Santa Rosa, CA Posts: 390	<<<-- Originally posted by Dean Sensui : Robert... For an empirical review of Firewire, Ultra-SCSI and ATA RAIDS, take a look at barefeats.com. http://www.barefeats.com/hard29.html -->>> That's actually kind of a depressing picture of the situation, as it stands right now. The Ultra320 SCSI array was strangled by the slow G4 PCI bus and the excellent performance of the Firewire800 array came at the cost of installing three Firewire PCI cards in addition to using one of the existing Firewire ports, essentially making the machine useless as a ProTools chassis. The four-channel ATA RAID setup wasn't too shabby, but requires a machine with bays to accomodate five internal drives and my Quicksilver only has room for three. This probably means that the X-Serve RAID solution will be just as hampered by bus speed on the G4. It's starting to look like the G5 machines will be a necessary upgrade for people working in HDV editing. Thanks for the link! -Rob

September 1st, 2003, 07:56 AM	#8
Robert Jackson Inner Circle Join Date: Jun 2003 Location: Santa Rosa, CA Posts: 390	<<<-- Originally posted by Jeff Donald : That's why I said it's too early to tell. The HDV standard won't even be finalized to the end of this month. Editing in uncompressed formats is very expensive and may offer very little quality improvement over HDV for most applications. -->>> Well, it wasn't really an issue of quality. I don't expect any kind of improvement in quality by editing uncompressed. My limited experience has been that working with compressed video lends itself to trouble. Corrupted files popping up and glitching, etc. Who knows, though, the failure rate of this new video standard may not be as bad as previous MPEG-2 solutions. It does worry me, though. The compression is necessary at the capture stage to allow a Mini-DV tape to contain the data, but it seems like the sooner that footage is moved to a workstation and uncompressed, the more reliable the editing experience will be. -Rob

September 1st, 2003, 06:29 PM	#9
Dean Sensui Inner Circle Join Date: Oct 2001 Location: Honolulu, HI Posts: 2,054	Robert... Just FYI, I've been working with a Media 100 system for the last several years and haven't had a problem with compessed files at all. Of course M100 compression works through hardware, so that might be part of the reason for its success. Perhaps that might provide a helpful clue in how to avoid the kind of problems you're facing? Good luck Dean Sensui Base Two Productions

September 1st, 2003, 07:21 PM	#10
Glenn Chan Inner Circle Join Date: Jun 2003 Location: Toronto, Canada Posts: 4,750	<--- This probably means that the X-Serve RAID solution will be just as hampered by bus speed on the G4. It's starting to look like the G5 machines will be a necessary upgrade for people working in HDV editing. ---> People are using the XSERV RAID right now to edit uncompressed, so it isn't a problem. The G5 has even worse storage than the G4 (ignoring the PCI-X instead of PCI expansion slots) as it has less PCI slots and only 2 drive bays.

September 1st, 2003, 07:37 PM	#11
Robert Jackson Inner Circle Join Date: Jun 2003 Location: Santa Rosa, CA Posts: 390	<<<-- Originally posted by Glenn Chan : People are using the XSERV RAID right now to edit uncompressed, so it isn't a problem. The G5 has even worse storage than the G4 (ignoring the PCI-X instead of PCI expansion slots) as it has less PCI slots and only 2 drive bays. -->>> I had no idea that people were editing uncompressed HD video with the X-Serve RAID already. I'd love to read some feedback from some of them if you can point me in the right direction. As for the G5 storage issue, the speed of the PCI bus is really the issue. If you're using an external RAID array then the number of internal drive bays is kind of a non-issue, IMO. The G5 DP has five PCI slots of varying configurations, BTW. Not too shabby, really, especially with the speed boost. Dean's M100 experience may actually hold the final answer, though. I wonder if we can look forward to hardware-accellerated editing for HDV MPEG-2 files in the coming months? That might solve both the drive-bandwidth problems and the problems regarding file corruption/glitching when editing compressed video. Waiting for someone else to do the trailblazing on this issue is the biggest problem since patience is my least abundant resource. ;-) -Rob