Best way to shoot a smart phone screen without moire?

[Shawn Whiting]

Hey everyone,

I’m shooting a promo for a smart phone app this weekend which will require close ups of someone navigating the app. I'll be shooting on my 5D MKII. Previously I've shot stuff like this with the sharpness at 0 and the mosaic engineering filter in, which helps a bit, but doesn't eliminate moire entirely.

Are there any digital cameras that could film the phone screen without producing moire? Are there any that are known for being better than others? I know film would be best, but that’s not possible.

I don’t have much motion graphics or compositing experience. Is there a super easy way to screen capture a smart phone? Even if there was it would only be the resolution of the phone and nothing close to 1080p like the rest of my footage right? Also screen capping the phone won’t include the fingers interacting with the app, which would be a downside.

What do you all think would be the best solution to this problem?


Thanks!

[Jon Fairhurst]

I've done stuff like this by showing a full-screen green image with some targets to help motion tracking. Composite with After Effects. The key is to ensure that the actor doesn't obscure too many targets at once. When three targets are clear, tracking can be fast and easy. When obscured, you can spend hours trying to get it right.

The other challenge is to get the actor to interact with the app well. They need to touch the correct spots and move naturally. One guy I worked with had a serious case of "Vanna White Hands". It was like he was directing attention to "for our grand prize... a brand new 1986 Honda Accord!"

If they can practice with the real app and if you place dots where they need to touch, than can help greatly.

BTW, to really sell it, add some glare and glint on the composited screen. You might also need to apply a lens filter if the phone sometimes appears out of focus due to shallow DOF. Personally, I really like applying these effects - the tracking and keying is technical and either looks right or wrong. The glare and filtering effects are subjective and are a bit more artistic to apply.

Best of luck!

[Shawn Whiting]

Great info Jon thanks for the reply!

I'm guessing this works best when the phone is perfectly still? Can it still be done with a moving phone where the angle of view is changing and the screen is tilting / skewing?

In my case I have very little compositing experience, just keying people out of green screens for interviews. Would taking on something like this take a long time for someone like me? I've never used markers on any of my green screen stuff.

Is the stuff you're replacing the green screen with all animations? Or are you replacing it with a screen capture from the smart phone that was from another take where someone was actually interacting with the app? Seems like in that case it would be very hard for someone to perfectly sync up to how the app was navigated in the screen capped take.

[Jon Fairhurst]

As long as you can see three or more targets on the phone (could be a button, a graphical mark, or the corner of the green screen), you can apply After Effects' motion tracking as the phone moves. Regarding the content on the screen, you can put whatever you want there. We mocked up an app that doesn't exist and animated it to show a use-case scenario.

[Shawn Whiting]

Do you have a link to your project? Would be interested to see how it looks.

I dont have much experience with compositing and animating. Is this something a computer / editing savvy person could pull off given a good amount of time? I know the basics of keyframing / moving graphics around screen right now. But to construct that and have it dynamically shift size and perspective relative to the phones screen seems like it would be difficult? I guess if the phone is steady it would be way easier, but still a bit confused on how to construct that entire screen simulation then have it be motion tracked onto the phone.

[John Carroll]

I think Jon is giving you sound advice, but this is an advanced technique... if you've never done it before and this is for a paying client, proceed with caution! There are a lot of great tutorials on videocopilot.net that would help you out with this kind of thing and would give you a good idea of where you are in terms of compositing skills and how long it might take you to complete a project like this.

Not trying to scare you away from it at all because it is totally doable, and probably pretty fun... just hate to see you dig yourself into a hole if this isn't something you've done before...

Not saying you cant do it, but do yourself a favor and do some tests before you promise your client you can do it.

Good luck!

[Jon Fairhurst]

John is right about this being somewhat advanced. If this is for a paying client, make sure you've done tests and can pull it off before you commit.

I can't show my project, but you can see how it's done with numerous tutorials from Adobe, Lynda, etc. According to a presentation at the Avid booth at NAB a few years ago, they do this all the time on TV shows like 30 Rock. They showed many examples of photos and screens that were added in post, since they don't necessarily have all those creative resources ready before filming starts. Most any commercial advertisement that includes the text "simulated screen image" was done this way. When done well, it's completely realistic. (And often better looking than reality!)

Note that you can lease CS6 from Adobe for a monthly fee, so you don't need to pony up a lot of cash to try out After Effects. The video tutorials on the web make it very achievable. It's worth doing some test shots this way.

As for the screen, you might film or photo it up close and straight-on in a dark room with a macro lens to get your resources. Then, when you composite it into the scene, the screen will be bright and clear and without unwanted glare.

Anyway, check out the tutorials and try the trial and lease versions of After Effects to test your skills. If you want the best quality and highest level of control, this is the way to go.

[Shawn Whiting]

Thanks for the info!

How do most people capture the screen / compositing resources in these cases?

The moire problem arises once you try to capture a screen made of pixels (iphone, computer, TV) with a sensor that is detecting in a pixel pattern that is not equal to the screen you're filming. So it seems like any sort of digital camera used to gather the resources is out of the question.

So screen capturing the device, or creating custom animations seems to me the only way you could start to go about something like this?

[John Carroll]

Exactly.
Moire is only one of the problems... even without moire problems, there are still exposure and color balance problems to deal with too.

If you can pull off a believable composite in post, you can ignore most of these other problems when you shoot...

Another tidbit would be to shoot with a healthy frame rate and shutter speed (maybe 60fps at 1/120) to reduce motion blur when the fingers go in to tap the phone... this will help to get a cleaner key, or if you are having to roto each frame, it will make it easier to get nice clean frames. You may want to add motion blur back in to the final composite.

Plenty of little tricks like this will reveal themselves once you do a test or 2.

Good luck!

[John Carroll]

Take a look at this tutorial for the reverse tracking technique... this could be really helpful to you if the phone needs to be moving. Once you have it tracked, you can composite your screen in and it will "stick" where it needs to, you'll just need to mask out fingers etc.

Demon Face Tutorial

It's a pretty fun little tutorial, and the technique can be useful for a lot of different applications!

[Jon Fairhurst]

You could take digital photos of the screen in a dark room with a macro and tripod. From there, you can clean up the images in Photoshop, or make new ones with vectors in Illustrator. The big advantage is that you can mock up anything. In fact, producers typically want to avoid using 3rd party apps or even a known PC operating system in order to avoid copyright issues.

Remember Jurassic Park? "It's a UNIX system! I know this!" ...yet the image looked like this... http://cdn.chud.com/6/6f/6fb069b3_jurassic-park.jpeg

[Brian Brown]

I think a HD video camera (and NOT a DSLR) will give you the best image and less moire. That said, I shot a phone app promo for a local developer with my Canon 7D a few years ago. It turned out OK, and moire wasn't horrible. For the close-ups, I had her hold her hand in a jig, so it didn't dance in and out of focus. I also used a 20" monitor to insure that I liked the focus and moire... sometimes just a touch of camera movement (or focus) made a big difference on image quality.

ChoiceView - 4 minute spot - YouTube

videohive.net has some phone and tablet comps for After Effects with human hands and swiping that you could use to mock-up composites with screen shots. By, I vote for live interaction when possible. It takes a lot of work to fake it.

[Jon Fairhurst]

The decision really depends on the context, doesn't it? If the app doesn't exist on the phone, then you need to fake it. If you need to avoid copyright issues, you need to fake it. If you're doing a feature film, national broadcast production, or national ad spot, you need to fake it.

On the other hand, if the app exists, there's no copyright issue, and you're doing a training video, a small market ad spot, or something casual for the web, shooting the phone directly is the fastest way to get it done. The video above looks reasonably good, but not as good as a good composite. (Then again, a bad composite will look much worse!)

Come to think of it, I've taken both approaches (composite & direct). I've composited for a few projects where the app didn't exist and we needed a truly professional result. And I've shot a phone directly for a training video using a 5D Mark II and Mosaic Engineering filter.

I've also done a hybrid when shooting a PC screen for a training video. I shot the screen directly and replaced and animated some text in order to make it technically correct for the story. That was as tough as anything as the font had to be exact and positioned just right and I had to dirty it up to keep it from drawing attention to itself as an obvious patch.

For the full composite job, there are three variables:

1) Can you get and animate the resources? This could be as simple as grabbing one static screen, or it can be complex and very, very time consuming.

2) Are three targets clearly visible at all times? If so, tracking is automatic. Setting up the project and rendering might take longer than the tracking, when done well!

3) Is the lighting good on the green screen, the hands and anything that obscures the screen? If so, it can be easy to pull a good key. If not, it can take a lot of layers and time to do really well.

Anyway, the right decision depends on the project. And ease of implementation depends on how well the images are planned and shot.

[John Carroll]

It really does need to be composited if a broadcast quality end product is required. If your client has high expectations for image quality, shooting the phone directly is going to cause you some embarrassment.
For a simple training video or demo, certainly save yourself the time and shoot the screen directly...

Basically, like Jon said...

I think Brian's example of shooting the screen directly is probably about as good as you could hope for if you plan to shoot the actual screen (unless you go into extensive color grading, which could be an option for you...)

[Oren Arieli]

I can't confirm this will work, but perhaps adding a screen protector with a matte finish might do the trick.