Artifacting normal on HD100
 

I am about to send the cam to the rep. because of quite hevy SSE. But I am wondering if the artifacting that you can see on the image is normal? Do you get those big blocks when filming the sea?

Its filmed in 25P and captured with Aspect HD

http://www.plonk.se/testpic1.jpg
(176kb)

It seems that you've found the limitation of the HDV codec.. Good enough for a lot, but not for things like heavily moving water or waving trees..

Uhm, no that doesn't seem quite normal. I've had the GY-HD101 for about half a year now and filmed a lot of similar things. That kind of bad image never occured with me. If I saw it, I'd think it would be abnormal...

I've shot material with similar fine detail, as have other in this forum, and not seen macroblocking like that.

I'd also, from the good words I hear about Aspect HD, would guess it's not strictly from that either.

Is it possible there's yet another layer of recompression happening somewhere in your flow?

It is not from the jpg-recompression of the image, you see the same macroblocking on the clip in the program-monitor in premiere.

Anybody with samples of the sea?

Here is another picture (a more butiful one) where you can see what happens when the sea is moving compered with the rock.
http://www.plonk.se/macroblockinghd100ecineform.jpg

Note: The camera is not moving at all.



Is it somthing wrong with my cam or is it souposed to be like this?

There's nothing wrong with your camera; what you're seeing is what HDV does.

Here's another example, ducks on a lake, that shows the same type of artifact. It's a pixel-for-pixel extraction from the 1280x720 frame.

http://www.icexpo.com/HD100/TwoDucks-1.JPG

Rippling water is tough for HDV to cope with; there's too much changing all over the place for the motion prediction to handle. If you're shooting in 24P mode it'll be a little more resistant to artifacting than it would be in 30P mode. It's also dependent on how much of the frame is filled with moving water; if it's a small portion, then HDV copes much better, but if the moving water covers most of the frame, that's a lot more challenging for it to deal with.

Is this good or bad news, I dont now?

Huum, 70% of the Earth is coverd with water, didnt the JVC peapole now that?
Do you peapole agree on this one, Is HD100 sopoused to be like this?
Hmm, I ges big waves are better than the small ones.

Well, maybe I dont have to send it to the rep. then, and I will be saving some monye.

Well - I have to say that there are some artifacts on the edge of the arm of one person. I really have never seen an image thjis bad from a HD100, defenately not mine... although I've shot similar things with lots of moving fine details. I'd say it is somewhat worrysome and perhaps you should ask the rep (or the JVC techs) if that's normal.
Is it just a few frames with such bed artefacting or is it constantly like this (in this shot obviously)?

Well only you can answer that question. On the plus side, the chances that viewers will notice the MPEG artifacts while watching your video in motion (emphasis on MOTION) are extemely low. While an editor sees every frame and srutinizes every pixel, the viewer just won't look at it like that.

By the way, I think the worst thing I've seen was some footage on CNN playing in one of the smaller "boxed" videos next to the news caster. It was night time footage of George Bush getting off Air Force One. There were photographers all over taking flash photos. I'm pretty sure the videographer was using something like a Z1, because every time a flash went off the MPEG2 broke the whole image down into giant blocks. The image changed so much from the dark frame to the lit frame that the real-time encoder just died. Too much change within a GOP is just not ideal for HDV, its just the trade off to shooting generally great quality HD video on $5 tapes.

Anyway, back to your water footage, perhaps you should screen it for some of your friends or family on an HD set and just ask them how they liked it. I bet you won't hear a single complaint about artifacts in the water.

www.philipwilliams.com

To me it al just seems to be normal HDV compression. People who claim to have better examples in the same conditions, can you post some stills here? I'm very interested in the differences.

I will test
 

Mikael,

If the weather/wind cooperates, I will go to the water today and make some test footage and then post some shots on my site later this evening
I will make the same shots in both 24 and 30p for a comparison

I shoot at minimum detail almost exclusively regardless of subject, yet today I will make some shots at higher detail to see if I can create shots similar to what you have posted - I will record all camera settings and equipment used
as well as time of day and available natural light description

cheers
Pete

Glad to hear that! It will be very interesting to see the result. I will do some testfilming as well.

Vincent, do you get the same result with your new Canon?

Hmmmm...... Mikael quick question. Philip mentioned seeing footage on CNN that "broke" when trying to adapt to light changes caused by flash bulbs. I noticed a large area of white in your jpeg... perhaps the shimmer off the water had the same effect as the flash bulbs. Could someone shoot the same scenario with an ND grad to prevent the water from clipping and see if you get the same macro-blocking?

Thanks.

Tim

Most anything seen "broadcast" these days has MPEG2 encoding much more drastic than what the HD100 does to tape. Digital cable is MPEG2, as is digital satellite.

I see MPEG2 artifacts at home on every channel I receive!

More or less, can't tell if it's exactly as bad/good, but these results do not surprise met at all. I have some difficult footage as well, a swan in water, if I have some time I'll upload it. In the meantime you can look at some edgy footage I've uploaded for an other topic, captured with FCP: http://www.vincentrozenberg.com/files/tree.zip

Nate speaks the truth. I've seen broadcast HD break down worse than I've ever seen HDV fail, and I've pushed HDV harder than just about anyone.

Tim, did you see this picture (same link as above), there is no clipping light here. but the blocks are there anyway.
http://www.plonk.se/macroblockinghd100ecineform.jpg

I must say its a litle bit surprising, its not an extrem sceen, and if you have moving clouds you will see the artifacts there as well. And the artifacts in the clouds will be musch easyer to see.

Broadcast and what you get at your tape from your own cam is not the same thing.

It's not the brightness level that causes the problem with MPEG-2 and flash bulbs.

What happens is that MPEG-2 gets its efficiency from repeating duplicate information between frames. It only encodes the changes between frames. It doesn't have nearly enough bandwidth to re-encode all the frames, so it relies on the relatively-unchanged nature from frame to frame in order to get its efficiency and fit all the data within its limited bandwidth.

Usually this works quite well; think of an interview setting: the background doesn't change at all, so it gets carried over frame to frame unmodified, and usually only the person's mouth and maybe their hands are changing, so the available bandwidth is allocated to compress those changes.

But when a flashbulb goes off? EVERY PIXEL changes! HDV hates that. Broadcast HD hates it even more. When every pixel changes, there's no way that the MPEG-2 bandwidth can cope and keep up with all the changes, so you get macroblocking and (in worst-case scenarios) lego-blocking. And it doesn't just affect the one frame where the flashbulb went off, either -- it affects every frame in the group. HDV allocates its bandwidth across a group of six (JVC) or 15 (Canon/Sony) frames, so all the available bandwidth gets spread across those six or 15 frames. If one frame contains a flash where all the pixels change, that one frame will require a lot of the available bandwidth from the group, which means all the other frames in the group will be robbed of the bandwidth that they would otherwise need. So all 15 (or 6) frames get degraded.

So a worst-case scenario for HDV would be a strobelight. Especially a strobelight that flashes more than once per group of frames, but that doesn't complete its on/off cycle within one frame. Say it ramps up to peak brightness on one frame, and dies down to dark over the course of two frames -- it would mean complete pixel changes on every pixel over the course of three frames. That will lead to massive artifacting.

Rippling water is kind of the same thing -- every pixel is changing in the body of water. So HDV and MPEG-2 have a very tough time coping with it. But usually it's not so bad as the strobelight comparison, because in a rippling-water shot there'll often be skyline and shoreline that don't change, so only the portion of the frame with the rippling water will be overwhelming the codec, and the rest of the frame will be relatively static (which compresses very efficiently). So the codec has a better chance of dealing with it. But the higher the percentage of the frame that consists of rippling water, the worse off the codec will be.

Also, MPEG-2 employs motion prediction -- if something is relatively unchanged, but in a different part of the frame, MPEG-2 can usually "find" that and copy it over (to grossly oversimplify the explanation!) So for a panning shot, things might be mainly unchanged but just moved; MPEG-2 is designed to cope with that. But there's no way MPEG-2 can predict rippling water, nor can it predict the effect a strobelight has, or smoke or fire or other random/unpredictable things. So those elements will seriously challenge MPEG-2/HDV.

It's affected by the same principles though. HDV has more resiliency to artifacting than broadcast television does because it has more bandwidth per pixel allocated to it. Broadcast HD sends 19 megabits of 1080i; HDV uses 25 megabits for 1080i, so more bits = more resiliency.

In 720 mode, broadcast uses the same 19 megabits, but it has to broadcast 60 frames per second, whereas HDV only encodes 30 or 24 frames per second, so again, there's much more bandwidth allocated per frame in HDV tape recording than there is in an HD broadcast.

So even though the same factors are at work (MPEG-2's limitations), the camera originals will usually be more resilient to artifacting than the final HDTV broadcast will be.

You mean, why didn't Sony, JVC, and Canon think of this. It is low-bit rate MPEG-2 that can cause this. But, you should consider what folks are saying about your camcorder.

Depends on what question you're asking. If you're asking "is my camera defective and does it need service?" then the answer is good news: no your camera is not defective, it doesn't need service, it's performing like all the other HD100s would.

If you're asking "does this mean I can't shoot artifact-free water footage with this camera?", then you may consider the answer bad news. There are limits to what HDV can handle. You've run into one of them.

But -- do you see the artifacting during playback? Or is it only when you examine the still frames? If you don't really notice it during full-speed playback then is it really a problem?

Steve is correct. Furthermore, I have found the JVC to be more resilient to artifacting than the Sony or Canon implementations. You should try that same shot on a Sony or 1080/60i Canon, you'll see that the JVC is controlling the artifacting better.

Don't know about 1080/24F Canon; I suspect that it will perform more robustly than the 1080/60i version would.

I can understand the technical limitations of HDV and how it works, but I just tought my HD100 would do better in a situation like this.

As I said before, its not that extrem, a litle bit more than 1/3 of screen is coverd with water and the rest is static. The cam is not moving. How much blocks would it be if I had the cam on my shoulder and did some panning as well?

I would say you can see it, at least it looks diferent than without the blocks. The blocks is simply creating another picture, and it is definitly not better. If you have moving clouds in the picture as well the artifact gets wery obvius, becouse they are esyer to display in the clouds.

today's artifacts testing...
 

As promised

go to www.vidprostudios.com .... on the home page pic of JVC camera, click on the white rectangular "record indicator" ... this will take you to the page where the test photos are located

note: footage was ingested via AspectHD to Premiere 2.0 - cineform intermediate codec ... exported as .tga files into photoshop where I then created the pip (at approximately 160%)
exported as .png files

I will put up the 30p comparison tomorrow

cheers
Pete

Unless I'm missing something the shots with detail at minimum are dramatically better... so leave the sharpening to post?

Joel

exactly that ... I think it's apparent that in a shot like that, anything above minimum detail is going to magnify blocking artifacts
I found too that keeping the exposure up near 100 helps to soften artifacts without losing actual focus ... none of those shots posted were CC ... I could have easily bumped the colors to make a richer scene without bringing in additional blocking

cheers
Pete

Peter, when did you first notice this? Thats a very big difference in blocking, compared to almost none! Both of these are Aspect HD ingested on what capture setting?

Also, off topic, I am using the same workflow (PPro 2 / Aspect HD) but have had a lot of people asking me if PPro 2 is as buggy as is the rumor, it's been very stable for me on our 64 machine. Have you run into anything odd? I only ask because I'm about to commit a rather large project to it and I fear you can't turn back mid project. Sorry for off topic.

I'm not sure I understand this... are you saying Zebras are at 100 and you're putting the highlights very near 100?

Daniel

these tests were done today in response to Mikael's posted concerns
I have been aware of the effective differences in the detail settings for some time now but I was not fully appreciative of the differences until I completed this test earlier today - those shots speak for themselves

all shots were ingested at "medium" cineform settings

no probems yet with 2.0, although I haven't completed anything beyond 30 minutes thus far
cheers
Pete

quote:
I'm not sure I understand this... are you saying Zebras are at 100 and you're putting the highlights very near 100?

yes, zebras set at 100 ... highlights as close as possible without clipping
I could have gone to F8 and still got a reasonable shot, but I found that keeping it open and towards the lighter side of exposure helped to soften the visible blocking. If I had an ND8 I would likely have opened it up to F4

Very cool findings Peter, even more so just in time for some pro motocross I'm shooting this weekend! All the fast moving action had me a little concerned with HDV. Thanks for sharing. I may post my results from the race as well

As for the PPro 2, are you on an AMD x64 system or an Intel P4.

Thanks Peter!
Very useful information, I will test that.

Post it!

If you can play around a little consider getting some closeups of bikes landing so you can get the dirt/mud flying. Try out the the 50p or 60p modes in SD so you've got some slow motion too. You might try zooming in with a wide aperture and then pulling focus. I bet with some practice you could get it working. Make sure to have some background in there like trees so you can appreciate the shallow DOF.

That should be a great testing ground. Test your black level in advance. In our tests we seem to like -1 or so to avoid crushing detail you may want to see.

Going way OT here, sorry...

Joel, those are all on my list of things to do... and then some, you are a mind reader no doubt. After shooting all of the GNC/GNCC motos last year with the Canon XL1s and a new 20X lens, i'm looking forward to the JVC / Fugi and overcranking some of the action. I use Paolos DSC settings but with the blacks at 0, even -1 is too crushed for me.

Cool - yeah... I'm a big fan of getting a wide range too. Ok - back on topic... I'm interested to see if you have HDV issues. Minimum may be the key to artifact free HDV.

Yep. That's pretty much exactly it. I shot a music video about a week ago and so far that's been the only time I've had the opportunity to use my HD100, but I've been reviewing the footage while I'm out on the road doing research for the doc I'm working on. It looks great, but when I capture a frame there will invariably be some area of the frame with blocky compression artifacts. I can't see anything out of the ordinary while the image is in motion, though. And this footage was taken in a little 1000-seat theater in Oklahoma, not on the water or while falling towards the Earth from a plane or any of the other cliche examples of things that stress the HDV codec. It seems to just be a side-effect of crunching the stream down in size, but it seems to be nearly impossible to detect without stopping the stream and examining a single frame carefully.

30p tests
 

My 30p test shots are now posted
all of the same same settings as used in the 24p shots with exception of going to F4 ...
regardless, Barry's suggestion that 24p displays less blocking artifacts seems to hold true ... see for yourself www.vidprostudios.com
click on the small text in the middle of the page "settings" when it first opens
cheers