Need Advice on Upgrading from HDV

[Jim Andrada]

Hi Stuart

I believe the V1U is CMOS based and CMOS technology is famous (or infamous) for so called rolling shutter aka jellocam, etc

The sensor cells in a CCD camera all store their values in an associated holding cell at the same instant when the shutter "closes" and then the camera scans down the holding cells in some amount of time. But since all the holding cells were filled at the same time, the image is just the same as if the cells had all been scanned instantaneously.

CMOS on the other hand doesn't need/have holding cells so the camera scans the actual sensor cells sequentially at the end of the frame's exposure. This means that the cells at the bottom of the frame are scanned later than the cells at the top of the frame and upright objects can look like they're "leaning" a bit as a result.Slow pans look OK, faster pans can look tear-y or jello-y. Effect is worse in some cams than others

Just guessing and this may not be what you're referring to, but hope this is relevant

If by tearing you mean the effect you see in really old still pictures of a racing car where the wheels look like the top is ahead of the bottom then this is probably it.

[Stuart Graham]

Andrew: Thanks for getting back to me.

Jim: Hi. Luckily my XH A1 is a 3CCD sensor so shouldn't have the jello effect.

I had a quick look at the problematic footage in Avid last night.

Bizarrely the video looks fine paused but as soon as you play it you see the effects regardless of if it's the original HDV footage or if it's transcoded to DNxHD. Hmm, I wonder what it means?

I wonder if there's a way of fixing it?

Will post some video of the tearing problem later today...

[David W. Jones]

Sounds like a field reversal or something of the sort.

[Stuart Graham]

I captured progressive footage and edited the footage in a progressive Avid project. So it shouldnt be a field reversal problem should it?

It's only seen when there's camera movement that's too fast. When it happens the screen almost looks as if it's splitting into 3 or 4 sections along horizontal lines.

[Ron Little]

Has anyone used the V1 + Nano combination? I am really interested in how well this works.

[Stuart Graham]

I've attached a clip that shows the tearing artefacts worst of all. This clip was recorded in HDV and exported from Avid as a Quicktime movie. It also displayed the tearing artefact when viewed in Avid, but not when paused.

I was going to use a different clip, but when I exported it as a Quicktime Reference file then reformatted it to an MP4 (AVC) file it looked fine. Boy, did I feel like a plonker!

But the attached clip is definitely looking awful on my system.

Has anyone else experienced problems like this?

Is it an Avid or Quicktime format problem?

Or is it a graphics card or graphics driver problem?

Maybe even the bitrate is too low?

Thanks!

[Andrew Smith]

I'm wondering if is something where it's easier to notice that your computer isn't keeping up with displaying the content to the screen.

Best thing would be to either put it on to a blu-ray disc or play back to a monitor from a HDV deck as these playback options are pretty much guaranteed to not have this sort of issue. This way you can tell for sure if it is your footage at fault or not.

Andrew

[David Heath]

Originally Posted by Stuart Graham

I've attached a clip that shows the tearing artefacts worst of all. .......... It also displayed the tearing artefact when viewed in Avid, but not when paused.

I think the latter comment gives it away - if you are seeing a "tearing artifact" on playback (which you earlier described as "the screen almost looks as if it's splitting into 3 or 4 sections along horizontal lines") but not when paused - I'd be fairly certain it's a function of the computer you're playing it on. And absolutely nothing at all to do with HDV, rolling shutter or anything else to do with the camera. If it was, you'd see it on the still frame.

The only way to be absolutely certain is (as Andrew says above) to replay the original HDV tape (either in the camera or a separate deck) and look at the output on an ordinary TV. (Look at the SD downconvert if you haven't got access to an HD set.) I'd be prepared to put a sizeable bet that the problem will have gone away. And conclusively prove that the problem is nothing to do with the camera or the codec it's recording to.

I've looked at the clip you posted and can't see the effect you describe ("....splitting into 3 or 4 sections along horizontal lines.....") at all, which leads me even further to thinking the issue lies with your computer system. I'm assuming you've further compressed the material down from HDV? (11MB for an 11 second clip seems far too small for raw HDV.) I can see static compression artifacting, but that's more visible on stills than the moving image and I'd assume that's down to this further compression.

Originally Posted by Stuart Graham

Just having more colour to work with in colour correction would be nice. And I'm going to be doing some chroma key and compositing work in the future so 4:2:2 colour sampling might help give better results.

Hmm, sounds so simple, doesn't it? I'll agree that 4:2:2 is preferable to 4:2:0....... IF (and only if) all else is equal. Unfortunately that's rarely the case.....

Firstly those numbers are ratios - not absolute numbers. The former is fundamentely saying there are as many colour samples as luminance in the signal, the latter is saying only half as many colour as luminance. So the former must be better, mustn't it? Well - if I offered you all the money in my left pocket, or half the money in my right pocket, which would you choose? The left pocket sounds the best deal? But what if you found out I had £1 in the left pocket, £4 in the right? Now, it's better to have half of £4 than all of £1, isn't it!?!

And you can get similar situations with video. For full raster 1080, we have 1920x1080 luminance samples, and with 4:2:0 chroma 2x960x540 chrominance samples. (1,036,800) For subsampled 1080 (such as DVCProHD) you have 1280x1080 luminance samples, so with 4:2:2 chroma sampling 2x640x1080 chrominance samples. (1,382,400) It's more - but nowhere near the twice as many you may have expected. And since the results of the chromakey will depend on the luminance signal as well as the chrominance, the far greater number of luminance samples of the first example MAY prove more significant than the extra chrominance samples of the latter!!

In practice, it gets vastly more complicated, and heavier compression can adversely affect chroma key and compositing work far more than the 4:2:2/4:2:0 issue, for example. And all sorts of things to do with the front end of the camera - is that inherently noisy? It doesn't matter what the recording spec is if the image sensor etc is giving poor quality to it.

[Stuart Graham]

Andrew: Good thinking. I'll dig out the tape and try playing through the camera and into the the telly at home.

David: I feel a bit embarressed I didn't work this out before... but I think your're right. Maybe my graphics card isn't up to spec or there's a driver issue or something like that. Your computer is obviously superior to mine! That clip looks terrible on my computer, I did compress it and it came out smaller than I thought.

Will dig the tape out this evening and check it on the old telly...

Firstly those numbers are ratios - not absolute numbers. The former is fundamentely saying there are as many colour samples as luminance in the signal, the latter is saying only half as many colour as luminance. So the former must be better, mustn't it? Well - if I offered you all the money in my left pocket, or half the money in my right pocket, which would you choose? The left pocket sounds the best deal? But what if you found out I had £1 in the left pocket, £4 in the right? Now, it's better to have half of £4 than all of £1, isn't it!?!

And you can get similar situations with video. For full raster 1080, we have 1920x1080 luminance samples, and with 4:2:0 chroma 2x960x540 chrominance samples. (1,036,800) For subsampled 1080 (such as DVCProHD) you have 1280x1080 luminance samples, so with 4:2:2 chroma sampling 2x640x1080 chrominance samples. (1,382,400) It's more - but nowhere near the twice as many you may have expected. And since the results of the chromakey will depend on the luminance signal as well as the chrominance, the far greater number of luminance samples of the first example MAY prove more significant than the extra chrominance samples of the latter!!

In practice, it gets vastly more complicated, and heavier compression can adversely affect chroma key and compositing work far more than the 4:2:2/4:2:0 issue, for example. And all sorts of things to do with the front end of the camera - is that inherently noisy? It doesn't matter what the recording spec is if the image sensor etc is giving poor quality to it.

I see what you mean, it is very comlicated!

I didn't realise DVCProHD was subsampled David.

Is HDV subsampled at 1440x1080 luminance samples?

So HDV has 1440x1080 luminance samples and 720x540 chroma samples... but it is compressed more heavily than DVCProHD so although the resolution may be higher for HDV the end image is worse? Does that make sense?

If HDV is 4:2:0, how bad is that zero at the end compared to the 2 in 4:2:2?

I was under the impression it mutes the colours quite a lot?

Would you say CMOS sensors are generally worth avoiding in a video camera because of the artefacts they can give?

[David Heath]

Originally Posted by Stuart Graham

Is HDV subsampled at 1440x1080 luminance samples?

Yes, but it would be wrong to think of it (and the same with the subsampling of DVCProHD) as necessarily a terrible thing. It's just one factor of a very complicated picture - same with 4:2:2/4:2:0.

Originally Posted by Stuart Graham

So HDV has 1440x1080 luminance samples and 720x540 chroma samples... but it is compressed more heavily than DVCProHD so although the resolution may be higher for HDV the end image is worse? Does that make sense?

The more you go into it, the more complicated it gets! Yes, HDV is compressed more heavily than DVCProHD if you look at it purely in terms of number of bits in, versus numbers of bits out. But it also takes advantage of more "tricks" than DVCProHD does - looks for similarities frame by frame, not just within each frame. Hence the end result isn't as cut and dried as simple bitrate numbers might suggest.

But the differences are much deeper than simple "quality". The more "tricks" a codec uses, the lower the bitrate it can use for a given quality. The drawback is that the "tricks" add complexity, and can mean much higher computing power necessary. The positive side is that a high quality can be achieved with a fairly small bitrate compared to a less complicated codec - that can make a big difference if (for example) it means cheap memory such as SDHC can be used, P2 or SxS being necessary for the higher bitrate codec.

Originally Posted by Stuart Graham

If HDV is 4:2:0, how bad is that zero at the end compared to the 2 in 4:2:2?

I was under the impression it mutes the colours quite a lot?

It doesn't "mute" the colours, just means that the vertical chroma resolution is half that of the vertical luminance resolution. In each case, the horiz chroma res is half that of the luminance, in the case of 4:2:0, that means there is now symmetry vertically and horizontally. (There is a difference between an interlace system and a progressive one, but I'm not going to start on that.)

As for how bad is it, what do you think when you watch any Blu-Ray disc, or BBC-HD, say? Do the colours look "muted"? Because nearly all delivery systems are 4:2:0.

Viewing is one thing, when we think about chromakey etc otherwise invisible differences can start to make a difference. Yes, colour space is one such factor, but arguably factors such as general compression, luminance subsampling etc can be even more important - and that's before we even get on to the front end of the camera, and things like noise levels. It's wrong to obsess over any one factor, and 4:2:0 certainly shouldn't be seen as "muting" colours - that just sounds like the sort of terminology a salesman with a vested interest in 4:2:2 products may come out with...... ;-).

If you want to read more, start with Chroma subsampling - Wikipedia, the free encyclopedia . To put things in perspective, analogue PAL savages the chrominance information far, far more than 4:2:0 does!

[Stuart Graham]

I'm officially a right nana. I hooked up the camcorder to the telly last night, played the original tape with the clip of a car wheel turning, and there were no tearing artefacts at all. It's definitely my computer that's at fault. I must apologise profusely to the gods of HDV for thinking it was a no good format.

I'm going to reshoot my panning tests in DV and HDV at the weekend to work out what an acceptable panning speed is in terms of image quality. I'll try and post them on here to see if anyone has any opinions on it.

Thanks for all the info David!

Here's a tricky one, would you say DV or HDV is a better format in terms of image quality?

I know you have more resolution in HDV, but are motion artefacts less severe in DV because it's trying to squeeze in less information into the same bitrate on the little miniDV tapes?

I'll have to try and get my head round subsampling a bit more, I'll study it in more depth next week.

[David Heath]

Originally Posted by Stuart Graham

Here's a tricky one, would you say DV or HDV is a better format in terms of image quality?

I know you have more resolution in HDV, but are motion artefacts less severe in DV because it's trying to squeeze in less information into the same bitrate on the little miniDV tapes?

If you want a simple answer, it would have to be HDV.

If you want a more complicated answer ( :-) ) then "HDV, but it comes at a price!"

The price is greater complexity (because it's long-GOP MPEG2, so more powerful computers to edit), though nowadays that's less of a problem. Going long-GOP does mean far better quality assuming all else equal, including bitrate. It does also mean that it may get tricked if there is a lot of motion in a scene - though often the blurring due to the motion covers the artifacting unless you pour over still frames. HDV is really defined as a tape format, and it is possible that a dropout can be more disturbing on a long-GOP system than on one such as DV - it can last over about a second in extreme cases instead of a frame.

Going to solid state eliminates the second concern, going to a higher bitrate such as 35Mbs more or less gets rid of the first one. Nowadays, if you're serious about your video, I wouldn't look at DV OR HDV, I'd go straight to XDCAM 35Mbsor 50Mbs, as used by the new Canon.

But don't get hung up on the codec. Overall quality is far more likely to be determined primarily by camera front end.

[Stuart Graham]

Hey David, thanks for the info again, I'm definitely becoming more enlightened!

Originally Posted by Stuart Graham View Post
Here's a tricky one, would you say DV or HDV is a better format in terms of image quality?

I know you have more resolution in HDV, but are motion artefacts less severe in DV because it's trying to squeeze in less information into the same bitrate on the little miniDV tapes?
If you want a simple answer, it would have to be HDV.

If you want a more complicated answer ( :-) ) then "HDV, but it comes at a price!"

Phew, I'm glad I've been shooting in HDV rather than DV then!

So DV is intraframe and HDV is long GOP. I didn't realise that, I did wonder how they managed to fit HDV into the same bitrate as DV.

One worry I have is that if I make a film I think could go on the telly the broadcasters will turn their nose up because it's shot in HDV. If I were onto something I thought would definitely be good enough to go on the telly - I might have to hire or invest in a new camera.

...maybe I'm being too ambitious, but you've got to aim high.

I watched The Stone Tape last night, an old BBC ghost story made in 1972. It was all shot on video and the footage and audio quality were a bit poor in places, but it didn't matter because the story, direction and performances were brilliant. What I'm trying to say is that if a film were shot in HDV, and it were captivating and novel enough, would it need to be in a 4:2:2 50 Mbs format or could it have been shot in HDV and still get on television?

One problem I thought about if upgrading cameras is storage. Sometimes I go out and record HDV onto a bunch of tapes at an an event or to get stock footage. And when I make a short film I usually end up with about 15 hours of footage. With tapes that's fine, the tapes are cheap and I can store them in a box on the shelf. The only worry comes, like it has now, when the box is too full and I need another box to put them in. But if I go tapeless they'll have to be stored on something other than tape, but where do you put it all? I know external disk drives are cheaper now, but they're not that cheap, not to fill up with random footage from an airshow or suchlike and have shelves full of external hard drives. Unless you could write it to tape somehow for long term storage? Any ideas anyone?

[Jim Andrada]

Besides the space and money, storing disk drives on the shelf is not a really great idea - they're designed to spin, not to sit idle, and their "shelf life" is an unknown. I know a lot of people do it, but it is really not ideal from a technology viewpoint.

Lots of people are in love with the idea of using DVD's as long term storage, but this technology is fundamentally not as permanent as a lot of people seem to think even though there's a lot of advertising about "archival quality" DVD's.

The really best medium for long term storage is tape, Not the kind of tape you use in the camera but something known a LTO (Linear Tape - Open) which was developed by a consortium of IBM, HP, and Seagate, although the Seagate tape business was sold to Quantum a few years back. Membership in the consortium brings a substantial benefit as the consortium members receive a license fee from the companies that make the media.

The problem with LTO is that the most recent version (Generation 5) drives and media are still a bit pricey, although as new generations are released approximately every two years, the media and drives get a lot cheaper. You've also needed some application to perform the actual copy to tape, although with Generation 5 IBM has released a version that supports a file system on the tape so files can simply be dragged and dropped onto a tape cartridge. As of today I'm not sure about HP's plans to implement this, but it was formally approved by the LTO consortium as part of the official LTO architecture. Unlike disk and flash memory and DVD etc, LTO IS designed to sit on the shelf for long periods of time. It has features like coatings on the back of the tape that are designed to repel the coatings on the front side of the tape to prevent layers from sticking together when tightly wound on the tape reels for years.

LTO is also very durable - the tape is officially capable of surviving 5000 (five thousand!) load-unload cycles, but I've actually seen some cartridges survive over 20,000 cycles in our LTO test cells.

Current (Generation 5) capacty is 1.6 Terabytes uncompressed with an average lossless compression algorithm implemented in the drive which can sometimes increase the storage per cartridge to something like 2.5 to 3 X the base number. Put another way, a single cartridge could hold anywhere from 100 to 300 hours of HDV. Last time I looked a G5 cartridge was about $100, but Gen 3 and 4 cartridges were in the $25 to $30 ballpark for half and one-quarter the capacity respectively. Haven't checked prices in a while but that was the ballpark. Dell external LTO 4 tape drives can be had for around $3k and LTO 3 for around $2k, but they would not have the drag and drop capability - G5 will be initially more expensive but should drop in price in a year or so and if it used the IBM drives would have the drag and drop capability.

Probably more than you want to know about tape!!!

[Andrew Smith]

Jim,

How long does it take to transfer a terabyte on the new fangled models? Just curious.

Andrew