Ghosting / Double Image

[Matt Faw]

Dear all,

I'm doing my best to understand creating stereo images, both via camera and via CGI. But one core issue has me perplexed, one which really hurts the otherwise beautiful effect of 3D: ghosting. In particular, when there is a great deal of depth between the fore- and background, the left and right images cannot line up on all depth planes, and some mis-alignment is introduced.

Unfortunately, my 3D cameras are all fixed lens, so I have only one IA, but in 3DS Max (using the 3D Hippie Stereocam rig), it's easy to experiment with various IAs and ratios. I have found that the ghosting can be reduced by lowering the IA, since, of course, the two frames are more similar to each other. But this means sacrificing depth, so it's a trade-off. What I end up doing is rendering foreground and background (and sometimes middle-ground) layers separately, so they can be separately aligned, and then composited in After Effects. But this doesn't work for elements that stretch from foreground to the background, and of course, this means that I'm converging on each element separately, which sometimes introduces strange depth effects.

Of course, I have even less control over my live footage, and the only blessing is that the fixed-lens cameras have such small IAs that the depth needs to be rather extreme, in order to create excessive ghosting. This is part of the reason why I was determined to shoot all my documentary interviews in front of green screen, so I can create and control my own layers. I am determined, however, to start shooting on a side-by-side rig, to find deeper IAs, but I don't know what I'll do about the inevitable double-images.

So, my question: what do you all do about ghosting? It's dirty nasty stuff, that quickly shatters the immersive quality of 3D. It seems to force the HIT, so my choice of convergence has almost nothing to do with deciding depth, and everything to do with reducing ghosting. Are there shooting tricks to reduce it? Post techniques?

I don't see ghosting in the natively stereo and CGI Hollywood films, but I don't yet know how they're able to get around it. I imagine that Pixar and Dreamworks must have some specially written code for creating stereo images that is less problematic than just two virtual cameras. But live action films like "Drive Angry" and "Step Up 3D" manage to project live action without ghosting. Does anyone know how they did it?

Thanks a lot!

-matt faw

[Roger Dass]

A lot of films out there are only 'converted' to S3D meaning they aren't actually shot using stereo cameras. In this case ghosting is easy to control since depth is generated completely from scratch (and is fake).

For true s3D live action, there are tools that can actually re-converge the scene through the use of image processing. 'Ocula' from The Foundry is a tool that can help with this. Ocula actually generates interpolated, 'in-between' views from your left and right source images. It then can recompose a new stereo pair using image data from these interpolated views as well as the source views to give you control over the entire depth range of the scene.

Aside from this, you may want to explore getting finer grained control of your stereo rig's IA using mirrors, etc.

[Matt Faw]

Originally Posted by Roger Dass

For true s3D live action, there are tools that can actually re-converge the scene through the use of image processing. 'Ocula' from The Foundry is a tool that can help with this. Ocula actually generates interpolated, 'in-between' views from your left and right source images. It then can recompose a new stereo pair using image data from these interpolated views as well as the source views to give you control over the entire depth range of the scene.

Aside from this, you may want to explore getting finer grained control of your stereo rig's IA using mirrors, etc.

Hi Roger, thanks for your response.

Yeah, this was a sub-topic that I was hoping would come up. I've read through some of the Ocula material, and the scene reconstruction seems very useful. I haven't yet invested in it, because of the cost, and because I want to make sure that I choose the software solution that works best for me. I've been looking at the following stereo reconstruction software, as well, although I doubt any of these are as robust as Ocula:
SourceForge.net: 3D Reconstruction using Stereo Vision - Project Web Hosting - Open Source Software
EStereo | Free Graphics software downloads at SourceForge.net
emotion3D - Unleash reality: Overview

If you (or anyone) has had personal experience with Ocula, I'd love to hear about it, how user-friendly it is, etc.

Also, I've invested in an AE plug-in that allows me to do some interesting things with depth maps. It's supposed to allow me to do "depth wipes", i.e. clip out content from a shot based upon z-space. If I can generate a good depth map of a live shot, then I could use the "depth wipe" to separate the various depth layers into separate AE layers, and align each layer separately. Good-bye ghosting.
EFX Render Elements Z Depth Tools

However, creating an accurate depth map also seems to be pretty tricky. I've done some testing on various solutions, including PFTrack, which (along with its match-moving abilities) generates depth maps from stereo pairs. Unfortunately, I'm only using the trial version of PFTrack (the full version of which costs over $3G), and thus, I can't export the depth maps it generates to test them. If the depth map isn't accurate, then the rest of that solution won't help.

So far, some sort of stereo reconstruction / depth map solution seems like the best possible finishing workflow, but all of the software I've looked into is either hugely expensive (for an independent), or is experimental and involves a great deal of manual work. That's why I'm reaching out to this community, and see if anyone has found or developed your own cost-effective and elegant workflows.

Thanks again,

matt faw

[Pavel Houda]

Ghosting is an issue of display technology and filtering. Ocula allows you to build an artificial video, but it really dowsn't solve the ghosting issue.

[Bruce Schultz]

Originally Posted by Matt Faw

. But live action films like "Drive Angry" and "Step Up 3D" manage to project live action without ghosting. Does anyone know how they did it?
-matt faw

Matt, I can remember a conversation I had on set with the convergence puller on the last Transformers movie about what IA sizes they were using. He said he was not using any IA over 20mm on any shot - that's less than 1 inch. When you have a camera like a 3DA1 with it's 60mm fixed IA or the Sony TD300 with it's 42mm IA you can begin to see why background divergence and ghosting can be such a problem with those large IA sizes. Ghosting is many times an artifact of a 3D monitor and specifically the angle of view of any given monitor can create ghosting artifacts if you are not positioned at the exact right spot in front of it.

Tim Dashwood's latest Stereo Toolbox has some pretty good ghost-busting filters built into it, but basically the wider the IA the more susceptible it is to ghosting.

[Pavel Houda]

It's like Dolby in audio, the closer two busy images overlap, masking the ghost image, the less likely the inadequate glass filtering will be noticeable(S/N). The main reason not to diverge has to do with screen size. Tim Dashwood has a nice calculator useable with any camera. It will constrain the useable 3D space within which one should capture. With respect to convergence/HIT, we don't want for the eyes to have to diverge with large screens in distance, so the closest object need to be sufficiently far, specially constraining in case of for large stereo base rig. IMAX @ 2.5" max divergence doesn't need very wide stereobase, in general case. One has to be very careful at capture time, ot there will be lot of thrown away footage and floating windows. Of course Ocula can help increating completely manufactured footage. As far as ghosting, there are many techniques, such as pre-bias, lowering contrast and saturation, avoiding high contrast edges, etc. If you use some viewing technologies, such as Vuzix, the problem doesn't exist. Otherwise ghosting will depend on specific display technology and quality of filter (display and glasses).

[Matt Faw]

Originally Posted by Bruce Schultz

Matt, I can remember a conversation I had on set with the convergence puller on the last Transformers movie about what IA sizes they were using. He said he was not using any IA over 20mm on any shot - that's less than 1 inch. When you have a camera like a 3DA1 with it's 60mm fixed IA or the Sony TD300 with it's 42mm IA you can begin to see why background divergence and ghosting can be such a problem with those large IA sizes. Ghosting is many times an artifact of a 3D monitor and specifically the angle of view of any given monitor can create ghosting artifacts if you are not positioned at the exact right spot in front of it.

Tim Dashwood's latest Stereo Toolbox has some pretty good ghost-busting filters built into it, but basically the wider the IA the more susceptible it is to ghosting.

Hey Bruce! What you said about Transformers makes sense; it explains why the 3D was so shallow. I have wondered if double-images was part of why bigger budget films often choose smaller IAs, rather than just concern about alienating the audience with "too much depth".

Certainly, with my passive monitor, I have a tiny window of being able to see 3D, without ghosting. Which would make it terrible for watching movies, etc. But I find that the off-axis view is useful for seeing how much the two views need to be adjusted. It was only after getting the monitor that I found I was able to align the images with confidence. Prior to that, I was relying on anaglyph previews which were too forgiving, and didn't reveal problems with alignment. Once I align the images, according to the off-axis view, then the on-axis view looks a lot cleaner.

I am very curious about Dashwood's ghost-busting tools. Hopefully, he will weigh in on this thread!

[Matt Faw]

Originally Posted by Pavel Houda

It's like Dolby in audio, the closer two busy images overlap, masking the ghost image, the less likely the inadequate glass filtering will be noticeable(S/N). The main reason not to diverge has to do with screen size. Tim Dashwood has a nice calculator useable with any camera. It will constrain the useable 3D space within which one should capture. With respect to convergence/HIT, we don't want for the eyes to have to diverge with large screens in distance, so the closest object need to be sufficiently far, specially constraining in case of for large stereo base rig. IMAX @ 2.5" max divergence doesn't need very wide stereobase, in general case. One has to be very careful at capture time, ot there will be lot of thrown away footage and floating windows. Of course Ocula can help increating completely manufactured footage. As far as ghosting, there are many techniques, such as pre-bias, lowering contrast and saturation, avoiding high contrast edges, etc. If you use some viewing technologies, such as Vuzix, the problem doesn't exist. Otherwise ghosting will depend on specific display technology and quality of filter (display and glasses).

Thanks for your response, Pavel. You mention the "many techniques, such as pre-bias, lowering contrast and saturation, avoiding high contrast edges, etc." Are these techniques applied to the entire image, or just to portions of the video? I'm wondering how much I can, in After Effects, effect or replace sections of the image, to spot-correct ghosting? How much could masks and corner-pinning fix them?

[Pavel Houda]

It all really depends on the source of video and the filters. For example, Tim's ghost buster is basically Real-D technology, where one basically pre-biases the image for the left eye with the opposite of the anticipated leak of the glasses from the right eye, and vice versa. Bernard talks about it in his book. That method subtracts the ghost from the image for each eye. Again, for that they must have a control over the projection system and the filtering characteristics of the glasses. It works similarily as noise reducing headphones, that injects the ambient noise out of phase, which subtracts some/most of the ambient noise from what gets to your ear drum. Some projection systems do the same thing in the projectors. Same should be doable for example with LG passive panels (maybe they are doing it, I don't know).

Sometimes people adjust the output from the monitors, e.g. lower brightness and contrast. I am not sure how you'd modify the it with AE, but it should be doable, however it would generally help for only a specific display system, and perhaps make other ones worse. I don't know if there are any AE filter plugins for that. Some people on this forum do know more on this than I. Obviously you'd have to be careful to preserve the image balance if you modify just a part of the frame. I normally try to control huge contrast when capturing video and lower contrast and saturation of the video, specially if it will be displayed by anaglyph output system - generally for the whole group of pictures.

I always play with HIT to minimize discomfort for movie screens, it will help to camoflage the ghost, but if the parallax is too small, the video looks 2D on computer screens, so one must be careful. Tims Stereo3d Toolbox has aiding vertical lines to help with that. This small parallax helps ghosting, but you'll lose depth budget or be forced to use floating windows if you overdo it.

Perfect filter, like headmounted display works the best. Active glasses and LCDs have sometimes temporal crosstalk/ghosting problem due to slow LCD switching speed, even though this is getting better. I don't know how to fix that. That is one reason I like DLP or plasma with active glasses better.

There is no simple answer to your question, in general. The more specific output system you ask about, the better answer you'll get.

[Wolfgang Schmid]

I think it is very true that ghosting derives from the actual displays. But it is also true that high contrast and a larger disparity brings up the danger of ghosting too. So I think the best solution is to adress those two aspects, since we have to live with our existing display technologys (they are not perfect but it does not help really to know that).

So what you can do is to use gamma curves that tend to result in lower contrast, if your camera alows such adjustments. And to avoid hard contrasts wherever it is possible. You could also try to bring down the contrast in the postproduction, but I think it makes more sense to controll that during shooting - if possible.

The other aspect is the IO. But the IO is not the solution in itself. I did not like the Transformer, at least not the 3D-BD version, since the 3D-depth was very low - and a low IA explains that quite well. I think you have to use the formulas available to end up with a low disparity - that brings down the danger to ghosting too. But for sure you need some minimum IO to end up with a sound depth but also with a sound roundness.

[Bruce Schultz]

Wolfgang makes a good point about high contrast creating ghosting. Many times I've had to tone down a hot spot in the frame to eliminate ghosting issues.

On Transformers, only the "A" camera (Sony F35) was a mirrored 2-camera 3D rig, the "B", "C", and "D" cameras were all Panavision film cameras shooting 2D, so the conversion of those angles to 3D was also probably a factor in the shallow depth that you saw. Screen size is important, and the larger the screen size for viewing, the more the IA can be pinched to reduce and eliminate BG divergence.

[Tim Dashwood]

Originally Posted by Matt Faw

I am very curious about Dashwood's ghost-busting tools. Hopefully, he will weigh in on this thread!

Ghosting is most prevalent with high-contrast images and circular polarization. There is always going to be some bleed-through/cross-talk with circular polarization systems like RealD, but if you preprocess and compensate for it on the source material it can be perceptually eliminated. Prior to 2010/11 a ghostbusting process was required for all Real D releases, but then they started implementing the process directly into the server software so that it now occurs realtime when projected. Now when we make DCPs we just have to indicate whether it requires ghostbusting or is already ghostbusted.

Most of the content I have produced and sent to DCP for RealD projection did not require any ghostbusting because it was normal contrast and total parallax was limited to under 1% positive and 1.5% negative. Larger parallax values will make the ghosting more obvious.

However, a good example of a film that required lots of ghostbusting is TRON Legacy. The high-contrast grid lines and light strips on black created huge ghosting problems in the screenings I attended, even the IMAX linear polarized projection. Linear polarization has the lowest amount of crosstalk but you must hold your head exactly level for the polarizers to work their best. Any tilting of the head will introduce crosstalk.

Our method of ghostbusting in Stereo3D Toolbox is probably similar or the same as RealD's. I designed the "Ghost Toaster" plugin out of frustration when Technicolor was charging me $1000/5 minutes to ghostbust my DCPs in 2009. The process basically involves finding high-contrast elements in one eye and reducing the black levels for that exact pattern in the other eye based on the amount of crosstalk in the display or glasses. Since the black levels in projection are never completely black the natural ghosting fills the "hole" in the blacks on the opposite eye and the apparent black levels are evened out on the opposite eye.

[Matt Faw]

Dear Tim,

Thank you very much for weighing in! Your "Ghost Toaster" sounds like a very valuable tool.

Interesting to hear that about Tron; it helps to illustrate what causes the ghosting.

Thanks again!

-matt faw

[Pavel Houda]

The general equation for ghostbusting is on page 177 here: 3d Movie Making: Stereoscopic Digital Cinema from Script to Screen - Bernard Mendiburu - Google Books.