Capturing HD from *Real* HD Cams

[Peter Moore]

So one huge advantage this camera has I suppose is capturing via Firewire.

How is caputre to a computer handled with, say, the Panasonic Varicam or Sony Cinealta? I gather these things do not output firewire MPEG_TS so how do they work?

[Scott Anderson]

The typical route is to go from a pro deck with Serial Digital Interface, or SDI into a PCI card specifically designed for HD, like the Pinnacle Cinewave HD or AJA Kona HD. Of course, most high-end systems use uncompressed or very gently compressed codecs, requiring fast and huge amounts of storage. For instance the Kona suggests a SCSI3 Ultra 160 or 2Gb Fibre Channel system with a whopping 200 MB of sustained transfer speed to the drives - that's capital B: Bytes not bits/second! That's over 4 times the theoretical limit of a firewire to even carry! (200MB/s=1600Mb/s. 1394a, or Firewire maxes out at 400Mb/s. 1394b, or Firewire800 maxes out at 800Mb/s)

So as you see, we're not going to be capturing uncompressed HD from CineAltas or Varicams with off the shelf consumer equipment anytime soon. That doesn't mean that we won't be editing HD content, even 1080i content, on very reasonably priced stuff in the very near future. Most HD transport and broadcast is based on MPEG-2 compression, which brings transfer bitrates and storage requirements into a much more reasonable size.

I have no doubts that in a year or three, you'll be able to take a variety of consumer-grade HD camcorders through a Firewire400 or 800 connection, into a fast IDE or serial ATA drive, and edit your HD content just fine in native MPEG-2. This camera is truly just the tip of the iceberg.

The future's so bright I gotta wear shades...

[Paul Mogg]

...and it probably should be mentioned that you're looking at about $30,000 for the HD deck and $12,000 for HD capture card. Then god knows how much for the SCSCI Raid arrays.

[Peter Moore]

Thanks for the info. But isn't MPEG-2 compression bad for editing because of inter-frame compression? There's like a 2 second lag whenever you jump to a new frame to edit.

What we need is a good HD compressor with no inter-frame compression. I wonder: could DV be adapted for this purpose? 1080i is exactly six times the data rate of DV, which is still far less than firewire's theoretical max, right? I'd be willing to buy a couple 250 GB hard drives just to edit HD video in this format.

[Scott Anderson]

I'm no compression expert, but I'll take a shot at it.

Inter-frame compression isn't necessarily a bad thing, as long as you're able to reconstruct the original frames with some degree of accuracy. But the the longer the GOP (Group Of Pictures), the more you're dependent on the accuracy of the codec to reconstruct those IB&P frames. I believe that's one reason why JVC decided on a relatively short GOP of 6.

The problems with editing MPEG-2 were many a few years back, but no current solution would ever offer anything less than frame accurate editing, and the ability to composite or re-render a few generations without noticable loss.

If I remember correctly, the Cineform basically ups the original 19Mb/s MPEG-2 to a proprietary codec at up to 80Mb/s, restoring all the original frames, and that's what you edit with. You then transcode back to MPEG-2TS to write back to the camera or DVHS deck. It wouldn't surprise me if Apple decided on a similar arrangement with the Pixlet codec for FCP. So far they have only mentioned it in terms of transport/internet broadcast, but if it's just avery efficient intra-frame wavelet compression, it might be useful as a (virtually) lossless editing codec.

Until the processing power and storage of your typical home computer gets to where $12,000 HD cards and fast SCSI arrays are now, I think we're going to be stuck with inter-frame compression. I wouldn't be surprised, however, to see that kind of advance in the next 5-6 years. FCP on a fast Mac or Vegas Video or Premiere on a fast PC can do a lot more in software and with off-the-shelf parts than 1st or 2nd generation Avid or Media100 editors could ever do on the $40,000 machines from just 7 or 8 years ago.

Until broadcasters, cable systems and satellite providers completely revamp thier infrastructures, and have more bandwidth that they know what to do with, the final delivery systems will undoubtably use this kind of compression. The only market interested in going far beyond HD's current limitations is the film industry, who's bread and butter are superior sound and picture quality.

About 1080i being six times the data rate of DV: Even 150 Mb/s (18.75MB/s) is really small. An Avid DS/HD system that works with uncompressed 1080i requires a stunning 994Mb/s! With 8 channels of audio added, that's over 450 gigs for just a hour of footage. No computer could process that (right now) without hardware assistance. Maybe that's a figure taken from broadcasters, who have settled on about a 20:1 MPEG-2TS compression for delivery.

[Peter Moore]

Right but what I'm saying is why couldn't you use the DV compression on HD material to give you footage that is about 60 GB per hour - is that really that unreasonable, both for hard drive space, and for firewire bandwidth?

[Scott Anderson]

I think that it might be unreasonable for HD material, especially 1080i. DV compression is only intra-frame, after all, so just ganging 4 DV compressors together would produce higher bitrates than that. It might also be beyond the level of DV tape to hold. I remember hearing at some point that the CineAlta used a similar scenario as this. They list the compression as 4.4:1 and the bit rate as 185 Mb/s at 60i. But then you get a camcorder the size of the CineAlta that costs $100,000, and tapes the size of DigiBeta that run faster, only give 40/tape, and cost $80 each.

I would suspect that it's also beyond the capabilities of low cost, low-power DSP's. One of the reasons that JVC was able to develop this camera is by using a third-party MPEG-2 signal processing chip. I forget who made that subsystem, but it was small enough to fit in the camera, and had a low enough power consumption to work for this form of camera.

[Chaim Bianco]

question to scott anderson,

scott, your posts are lucid and informative.
what do you do for a living?

.

[Peter Moore]

I see your point. So I guess we'll be stuck with intra-frame compression at the consumer level for the time being. I'm less worried about the frame accruacy than I am about the lag time. It's just so frustrating to try to edit MPEG2 video.

[Scott Anderson]

Chaim -

I'm a frustrated old film guy. I went to film school in the mid-eighties, went on a variety of life adventures, and have found myself doing government/corporate video work for the last eight years.

Not that I'm complaining, mind you. I was lucky enough to jump onto the very first versions of both Media100 and AfterEffects, back when it was from a little company called CoSa. I am still riding the wave of excitement I first felt when I discovered what amazing things you could do with video on a computer.

But I'm so darn tired of waiting for technology to catch up with the needs of the indie filmmaker. The idea of being able to shoot a viable feature film on what is basically a consumer camera has always been a question of "when" rather than "if". I don't consider DV to be viable for that kind of work, but this camera is certainly a big step in the right direction.

I'm just getting tired of waiting for the "when".

[Maury McEvoy]

Apple is currently developing an HD compression codec called "pixlet" for their new Panther version of OS-X. See the thread:

FCP4 + G5 + Pixlet QuickTime codec = HD playback

At:

http://www.2-pop.com/ubbthreads/show...5&o=14&fpart=1

Maury

[Scott Anderson]

Thanks much for the post, Maury. I was just talking with friends about some of the same things, without having seen the article. Great timing!

Thing might get very interesting indeed if Pixlet turns out to be a production as well as a delivery codec.

[Heath McKnight]

<<<-- Originally posted by Scott Anderson : Chaim -

I'm a frustrated old film guy. I went to film school in the mid-eighties, went on a variety of life adventures, and have found myself doing government/corporate video work for the last eight years.

Not that I'm complaining, mind you. I was lucky enough to jump onto the very first versions of both Media100 and AfterEffects, back when it was from a little company called CoSa. I am still riding the wave of excitement I first felt when I discovered what amazing things you could do with video on a computer.

But I'm so darn tired of waiting for technology to catch up with the needs of the indie filmmaker. The idea of being able to shoot a viable feature film on what is basically a consumer camera has always been a question of "when" rather than "if". I don't consider DV to be viable for that kind of work, but this camera is certainly a big step in the right direction. I'm just getting tired of waiting for the "when". -->>>

Scott,

It's not the technology, it's the art. The tools are just there to tell a
story. If you can't afford film, go digital. I had to learn that lesson,
after buying every new (at the time) DV device I could get my hands on. it
never made me a better filmmaker, other than picture quality with the hd10.
That's why I know so little of the tech terms. If I did, I'd probably lose
focus of the films.

But, I agree, I am frustrated, too. Why am I not Spielberg or Sayles? :-)

heath

[Scott Anderson]

Way to cut through the bull, Heath.

Of course they're just tools. Anyone who's worth anything in this craft knows that. Chaim had asked what I do for a living, and I was just trying to explain how exciting it is to have all these great tools at our disposal, but be just tantalizingly short of the final goal.

In my mind that goal has been the ability to shoot a project and see it on the big screen, with a theater size audience. That's what got me hooked in the first place when I was making 16mm films in school. There's no other rush like seeing an audience respond to your work in that way. It's terrifying, surprising and hugely satisfying, sometimes all at the same time. That visceral rush has been replaced for me, in a way, by the ability to have thousands of people see my DV originated work over the net, like this:

http://members.cox.net/scottandersonpictures2/film_club.htm

(BTW this film has been basically abandoned and needs a permanent home - any suggestions?)

The thing is, it's been a long time since I've been able to afford a project originated on film, and I don't think I'll ever shoot film again. DV is a fine format if all you want is a web film or even a video release, but I want to have that thrill of the theater audience again.

http://members.cox.net/azscott/otphq.htm

This was plenty cool, too, as I cracked the top 250 in Project Greenlight. DV was perfect for this project. It was shot in just 2 evenings, and then cut in 3-4 days.

They're all just tools, but using GOOD tools is a whole lot different than using GOOD ENOUGH.

I think a lot of young filmmakers have been led down the rosy path of believing that shooting a feature on DV is GOOD ENOUGH, and you have just as much of a chance as any other project shot on 35 or 16mm, as long as your film is dynamic and artistic. To this I call a hearty "Bullshite!". If you don't have the clout of a Soderberg or Boyle, you won't get far with larger festivals much less distributors unless there's a name actor in the film, or if your project originates on film.

The thing is, I came to this forum because I have been very interested in this camera since late last year, and you folks have seemed to have the most hard facts and first user feedback.

I truly believe that when low-cost HD tools become as available as DV equipment is today, with as many options and choices, you’ll see a huge number of new filmmakers emerge. The cream will always rise to the top, creatively speaking. I know this. I agree with you on the point that fancier tools will never, ever make you a better storyteller.

However, if you have the ability as a storyteller to pursue a similar production pipeline to DV: shoot on cheap, self-owned camera equipment and tape stock, post on your home computer, etc. And if you then have the ability to take that final project to festivals, HD cable, HD DVD AND theatrical releases without ever going to the costly options involved with “filming out”…

I believe you’ll see a lot more Soderbergs, Sayles, LaButes, Aronofskys and Tarrantinos. You might even get a new Spielberg or two.

[Heath McKnight]

I tried the Project Greenlight in 2000, but their website kept screwing up, even when they gave us second chances. I gave up on web stuff after pop.com died before it launched....

But I have had a movie open on DV in a movie theatre. Looked pretty good and sounded GREAT!

We'll get there...

heath

ps-Sorry if I caused any offense.