News posts from 2002

[Don Berube]

Hi Ken,

What's your take on the actual real world gain of using a firewire RAID 0 drive config versus one that connects via SCSI?

Do you use any RAID's at all?

Thanks for your input Ken!

- don

[Ken Tanaka]

Hi Don,
I do not currently use any RAID configuration although I have used, and directed the use of, RAID levels 0 and 1 configurations for very large system implementations.

For clarity's sake it's important to note that "RAID Level 0" is not really RAID (Redundant Array of Inexpensive Devices) at all because none of the drives are redundant in such a configuration. RAID 0 is essentially a pact with the devil; you're betting on the reliability of each drive in the array in exchange for a theoretical i/o performance increase. Data is allocated across all drives in the arrray ("striped") and the loss of any drive in such an array effectively destroys the entire array's contents.

My opinion is that RAID 0 is an unnecessary bet for DV and, in fact, probably produces little if any gain in exchange for the additional inherent risk. This is especially true if the array is connected to the system via a moderate bandwidth medium such as FireWire. The dirty little secret is that "RAID 0" doesn't perform well with small arrays (ex: 2 spindles) and large file transfers (such as those found with video footage). The overhead imposed by the controller and RAID software actually erodes performance as compared to single drive performance. RAID 0 is best implemented on systems where (a) the performance of the single system is more important that its reliability, (b) the array can be implemented largely enough, and closely enough to the system's main bus, to truly boost performance, and (c) where the average i/o size is small to moderate.

Of course in the case of high-bandwidth video applications such as HD RAID 0 may be the only practical choice to meet the application's demands. I would bet, however, that looking at such a production's installation would reveal the use of 10,000 rpm SCSI drives in arrays of no less than 4 elements connected to systems via very hi-band buses.

[Jeff Donald]

Infoworld has put together a series of articles about Apple Computer, Macs and emerging technologies from Cupertino.

http://www.infoworld.com/features/feapple.html

Jeff

[Don Berube]

Thanks for your input Ken. Very informative.

I'm gonna go for it and ask you a *dumb question* hehe

Am I to assume that the Wiebetech FW Raid is not a Raid 0, but some other form of Raid? They claim up transfer rates in excess of 60MB/ sec write and around 52MB/ sec read, using their included PCI FW card. That's a lot faster than the Super Desktop GB, isn't it?

I'm torn between the two. I need another external soon, mainly for digital video and imaging and audio. I use OS X Jaguar with FCP 3.

Do you ever browse into xlr8yourmac.com?

- don

[Jacques Mersereau]

Here's some interesting news from Charles F. McConathy of Promax.com
____
We just got in a couple of the brand new IBM 180 GB ATA drives and have
tested them in the new G4/1000 and G4/1.25 GB DP DDR computers.

In these two computers we see the full unfomatted capacity of 172.5 GB.
Also we see the full capacity with our TurboMAX ATA-66, 100, and 133 PCI
ATA card. However in some of the older G3 and G4's you will see only
about 128 GB.

In the new G4's you can install 4 ATA drives, move the boot drive to the
lower optical bay, which means you will have 690 GB of useable storage
space. Stripe the four drives with OSX and you will get about 95 MB/sec
sustained which is capable of doing uncompressed SD/RT using the AJA Kona
card and will store 7.91 hours of 8-bit SD uncompressed.

Four 180 GB drives will store 48.36 hours of DV Video - or about 12 hours
on each drive - in this case no need to stripe as RAID-0.

Like always IBM gets premium at first for new technology so the new 180
GB drives are a bit pricey right now - $349 each. Once they start to
deliver in quantity I think they will drop in price quickly. These new
drives employ 60 GB per platter technology...the 120 GB drives employed
40 GB per platter. The new drives will come in 60, 120, and 180 GB's
being 1, 2, 3 platters respectively.

I like IBM drives since they have the best zone technology - meaning the
inter zones do not fall off radically in performance - have one of the
lowest fail rates - and their electronics/firmware offers the lowest
error rates, etc.

[Jacques Mersereau]

You noticed that too eh? I don't know if DV mag is "fishy,"
but they don't review anything that doesn't place a nice
big ad. Hey, that's business. Two hands wash each other.

Most products don't get thrashed in their reviews, but
let's face it, most of the stuff out there is pretty good, and
nothing is perfect. DV mag. is one of my favorites, maybe my
most favorite video magazine. They seem to cover the topics
I'm thinking about.

[Bill Ravens]

I beg to differ with this thread. In one of my machines, I am using a Promise TX2000 RAID controller card in the RAID 0 configuration, with two older 5400 RPM Maxtor hard drives. The measured thruput on this config is about 39 Mb/sec. I agree that there is an inherent risk of data loss due to HD failure, however, in 8 years of using RAID arrays, I have yet to experience a failure. I suppose the odds are now running against me...;-). That's why I use the RAID 0 array as a working HD, used only for captures and edits. The final editted AVI always gets stored elsewhere. My point is, I would NOT give up my RAID 0 array for anything except cheaper SCSI HD's.

[Ken Tanaka]

Bill,
You actually aren't "differing" from my remarks, at least not much. You're using an -internal- 2-channel RAID controller on your IDE bus which is a different proposition than connecting an array to a Firewire port.

Nor do I doubt that your array seems zippy. But I do doubt that it features markedly better sustained performance than, say, today's single 7200rpm 120-180Gb drive. Just as with financial investments, pc configuration is often a balance of risk and reward. If you were configuring a from-scratch personal computer computer today it would be impossible to rationally justify a small (2-drive) striped array for the added expense and ongoing (2x) risk of failure.

So I still stand by my opinion that "RAID" level 0 is a completely unnecessary deal with the devil for dv work.

[Bill Ravens]

can't argue with that, Ken. However, I have read test reports of internal RAID 0 arrays featuring WD1000JB HD's that test at a thruput of 45-50 Mb/sec.....at least at the outer edge of the HD platter.

[Don Berube]

Hello Ken, Jacques, Bill,

So what would your opinion be of the new Wiebetch FW Raid with the included PCI FW card? Isn't the 60MB. sec wirite and 52MB/ sec read times a significant advantage over a single drive's throughput? Especially for a program like FCP, which is always creating thumbnails, caches, etc.?

Is it worth the cost over the Wiebetech SuperDescktop GB?

- don

<<<-- Originally posted by Don Berube : Thanks for your input Ken. Very informative.

I'm gonna go for it and ask you a *dumb question* hehe

Am I to assume that the Wiebetech FW Raid is not a Raid 0, but some other form of Raid? They claim up transfer rates in excess of 60MB/ sec write and around 52MB/ sec read, using their included PCI FW card. That's a lot faster than the Super Desktop GB, isn't it?

I'm torn between the two. I need another external soon, mainly for digital video and imaging and audio. I use OS X Jaguar with FCP 3.

Do you ever browse into xlr8yourmac.com?

- don -->>>

[Ken Tanaka]

Don,
In response to your WiebeTech question, yes I believe that their Firewire RAID system is configured as a striped ("RAID 0") array rather than a true RAID level 1. They are boasting of its performance rather than its reliability.

Humorously, though, I just noticed that WiebeTech indicates that the system has -dual power supplies-, presumably one for each drive. What a hoot! Dual power supplies are actually a hallmark of large RAID 1 arrays where half of the array continuously mirrors the other half. If one half of the array fails, mirroring stops and the other half immediately takes over. Power supply failures are the most likely cause of trouble and such systems split their supplies between two physically separate circuits to help ensure that one side of the array remains in operation.

But splitting a "RAID 0" array between two power supplies would actually have the effect of reducing the engineering reliability of the whole array by at least half if the end user actually connected each side to a separate house circuit (which is highly unlikely anyway).

I really like WiebeTech's drives; I own two. But I'd stick with their single Firewire drives. In fact they just announced that their Super Desktop GB drives are now available in capacities up to 200Gb! That's where I would spend (and have spent) my money on Firewire storage.

[Bill Ravens]

Firewire has a bandwidth of 400 Mb/sec. That gets shared with all the devices on the 1394 bus. My own simplistic tests with firewire RAID have shown that this sharing with other devices on the bus can impact the performance...dropped frames, and such.

May I suggest that, if time is not of the essence, ie, you can wait until January, there is a new IDE standard already approved and released called Serial ATA. Motherboards are already being sold with the SATA controller implemented. SATA is backwards compatible with ATA100 IDE bus hard drives. What happens in January is the new SATA hard drives are scheduled to hit the streets. The SATA HD's are advertised to have thruputs on the order of 100 MB/sec!! Sounds too good to be true, but, the initial independent tests I've read have verified the numbers as real and achievable.

Oh, and BTW, there already are serial ATA PCI controller cards available from several sources, including PROMISE. Can you imagine a SATA RAID 0? hehehehe

[Ken Tanaka]

Yes, Bill makes a very good point. During dv capture via Firewire the incoming footage is jostling down the same cable as the outgoing footage if you're using a Firewir drive. I've not had a problem doing this but, admittedly, I don't do this very often.

An onlooker to this thread just asked me offline (via email) how I would implement a RAID 0 array if I had to do so. My answer is that I would use 10,000 rpm ("10K") SCSI drives with the best caching SCSI RAID controller I could afford. Yes, SCSI is a bit of a hassle with its byzantine variety of connector interfaces and hard address assignments. But, once set-up, this is tried-and-true technology that keeps the drive traffic off of the Firewire bus as well as off the main system bus to the greatest degree possiblle.

[Gary Bettan]

GLEN COVE, New York – Specialty Video Supply, the educational division of The Electronic Mailbox and Avid Authorized Reseller, has joined forces with Avid to offer the lowest price ever on the Xpress DV 3.5 EDU and Student Pack bundles. For a limited time students, teachers and educational facilities will save $100 when they purchase the Xpress DV 3.5 Student Pack or EDU Pack from Specialty Video Supply. The Student Pack is available to students and teachers with current and valid school ID at only $399.95 and the EDU Pack is available to any educational facility at only $899.95. EDU Packs may also be purchased in 10-seat license bundles for use in larger multimedia labs for $6,995 ($1,000 off the regular 10-seat price).

Avid Xpress DV 3.5 educational bundles include everything needed to teach & learn the professional Avid editing interface on any Windows XP or Mac OSX system. Xpress DV 3.5 EDU Pack and Student Packs include the full version of Avid’s professional non-linear editing software that is based on the same Avid architecture as their broadcast solutions and offers hundreds of real-time transitions, filters and special effects plus the complete Avid color correction tool set with realtime color correction. The educational bundles also include Avid Xpress DV Filmmaker's Toolkit with extra capabilities for students and independent filmmakers shooting on film or on video; Avid Image Stabilization for correcting shaky camera work, one license key (USB Dongle) and one CD set containing all software and documentation necessary.

Specialty Video Supply, a division of reseller The Electronic Mailbox, is dedicated to providing solutions for schools and educational institutions. For more information on the Avid Educational Discount Program call Dan at 1-800-323-2325 ext. 123 or go to http://www.specialtyvideosupply.com/prodpage/avid.html.

[Jeff Donald]

I wouldn't use SCSI unless I was doing uncompressed video. It is totally uncalled for in mini DV applications. SCSI is too costly, too high maintenance costs, and too complicated for most users. Many of the SCSI controllers are problematic with the newer operating systems.

Instead I would use an ATA RAID 0. It could be put together for less than 1/2 the price, with nearly equal performance.

First you need an IDE RAID controller. There are 4 I know of for the Mac.

http://www.sonnettech.com/product/tempo_raid133.html

It supports up to 4 drives and is bootable and also allows you to break the 137 gig drive barrier.

I would choose Western Digital Special Edition drives (8MB cache buffer), probably 120MB offer the most bang for the buck.

http://www.wdc.com/products/Products.asp?DriveID=27

Total cost is around $200 for the controller and $280 (after rebates) for 2 120MB drives. So, $500 bucks for a blazingly fast RAID 0 set up. That's 18 hours of mini DV footage.

Jeff