Removing a moving object from moving video

[Gene Cornelius]

Hi,

Having been hurting myself (and my relationship) for the last two weeks over 4 seconds of video, I thought I'd look for help. I'm using Vegas Pro 9.0b.

The concept is that I need this vehicle to disappear completely from one point in a shot to another, so I can make it drive/fall off the cliff and fly away. There is no chance to reshoot the road without the vehicle, which would have simplified things greatly.

The problem is that there are so many things in motion that the Photoshopped stills behind the mask that have no vehicle are nearly impossible to blend together and to make them move in a semi-convincing manner (using pan and crop with LOTS of keyframing).

Is there a better way to do this? I tried 3D track motion but ended up getting more confused.

Here's a basic outline (I'm trying to keep things brief)

Track 1: Additional noise generated for end of clip which will fade into cloud
Track 2: Replacement vehicle, masked positive from original, made to fly away
Track 3: Original clip, with negative mask, removing vehicle
Track 4: Stills taken from original unmasked clip, vehicle removed in PS, avg. still length=6 frames

I've tried various manner of feathering on the masks, and lots of various ways of lining up the replacement stills to fill the hole made by the mask on Track 3, but there is still too much unnatural motion.

I'll see if I can't generate a clip to post that is not too embarrassing....

Thanks!

Gene

[Marty Welk]

well you wont be to alone :-)
think of all the movies you have seen 70s-90s where the car is weaving back and forth, cuts to the driver pannic stricken, someone squeals, the tires squeal, dust kicked up to try and cover the scene, then a Matchbox car goes flying over grassy lump trying to look like a hill, then it cuts to a Huge explosion that would never happen in real life , from stock footage of some different car , then dissolve out quickly to the hospital or funeral scene.

Remember Hitchcock , in film school, nobody ever dies on camera, they did it in shadows and cut in the sounds , you could always fly a kite over a hill and let them use thier imagination :-)

[Gene Cornelius]

Thanks Marty, I do need to be reminded of these things. There is most likely a non-technical way of delivering the concept to the mind of the viewer, but I find this particular challenge intriguing. A BIG part of the effect, as it were, is to lead my hapless viewers into the scene with nothing to tip them off, and have them suddenly feel the effect of seeing a vintage vehicle "accidentally" drive off the cliff (these are killer cliffs!). That few milliseconds of time is crucial: when the car goes straight instead of turning, and drops down....just before its rocket motors kick in and I can use the flame and exhaust as a distraction from the wobbly mass of garbage that is wavering away as if through an ever-shifting distorted lens.

This footage is from an annual event, and part of the challenge for me has always been to participate in this event, and also to get as much good footage from it as possible at the same time. With very very few exceptions, all the shots are real-life and real-time ... no control, really.

The video I am working on now has no footage to support the ending, and as it is third in a series of three all expressing the linear series of events throughout the event, I NEED to make an ending that is really cool (or at least interesting) and makes some sense.

I thought of having rocks rolling down from above and that is why the vehicle turns off the road.... that may be my best bet, as the rocks and dust can hide the vagueness where the vehicle used to be, and then I can use pan/crop to "follow" the vehicle towards the void and up through the valley.

But I have to make the rest of it look not-too-distracting. Perhaps I should contact the driver and beg him to find someone with a camera who can shoot him reacting, thus giving me even more leeway.

Thanks again!

Gene

[Nicholas de Kock]

Yikes, I don't know if something like that is even possible without 3D reconstruction of the entire scene. The simplest solution is always best.

[Ian Stark]

Originally Posted by Gene Cornelius

Perhaps I should contact the driver and beg him to find someone with a camera who can shoot him reacting, thus giving me even more leeway.

How about a lookalike driver (ie same hair and clothing) shot from behind, head and shoulders, so all you see looking out through the windshield is the back of his head and the approaching blue sky? You would only need to 'suggest' the windshield and could probably get away without any other interior aspects of the car being seen. You'd need to get his reaction (from behind, obviously) to whatever it is that forces him off the cliff. Some clouds in the sky that suddenly shoot vertically up out of view might also suggest the car is plummeting down. I'm guessing wildly here but I would have thought all you need is a very brief shot to segue between your real world and the fantasy world without it looking cheap.

[Gene Cornelius]

all fairness, I need to state that the main reason I posted was to make sure there wasn't alternate technical means of achieving my goal. I totally appreciate exploring the "workaround" options with you guys though, and learning other stuff at the same time. Keeping you all guessing about possible solutions when you haven't seen the clip is not at all fair!

One of the ways I was undertaking the process of replacing the content in the hole created by the mask was to place keyframes for size and position of the replacement still at the beginning and ending of each still clip, with the first keyframe being the default setting, as the still was taken from that exact spot, and aligned perfectly, naturally. The other end, and in-between is much more challenging. With the opacity of the original track set to approx. 50%, I could see the outlines of the road, the trees, the cliffs, and other markers. I thought that perhaps if I could align the outlines of the two images, then the motion in between would approximate that of the real video, and in some cases, it worked "well enough". For some reason, though, the replacement shot "swells" too much, and as the video transitions from one to the next still (whether I use fades, overlaps, or straight cuts) it "jumps". The question here is how do I make the shape of the replacement still change with regard to perspective....remember, I am a rank newby, even though I have been editing for years, I am no learned pro!

I would like to make as much of this shot work, as it would really further the effect. However, I am OK with using other methods to help get the point across. I think it will be a happy marriage of effect and displaced attention that does the trick.

There is no sky, btw, in this shot....just a glow in the far distance...it is in a steep valley, the roadway is a single lane with a vertical rock face on the left, and a vertical drop to the river hundreds of feet below. I do think I could probably get someone to shoot the driver from behind...but it would have to be him! I don't know anyone around here with hair like that!

I think I will use the landslide/rocks/dust (or at least attempt it). The problem is I will have to work in miniature (fun!) under the house, as it has been raining a lot, and I NEED that dust! This will be a nice experiment.

I'll keep thinking, and later tonight, I'll put up a couple clips on my website so the curious can see what I've been doing. So you get an idea of the level I am at, as well as the "type" of project this is, my name (Gene Cornelius) on Vimeo will get you to my two latest.

Thanks again - I really appreciate all thoughts, suggestions, and energy you all have put forth!

Gene

[Vito DeFilippo]

Hey Gene,

I haven't tried this, but I saw a demo at a Adobe presentation, and it was pretty cool. The newer Photoshops extended (CS3 extended and up) will open video files and let you paint frame by frame using all the photoshop tools. You can use the same effect on subsequent frames if needed.

4 seconds isn't that long, so even frame by frame you might be able to paint in the missing bits pretty quickly using the clone or healing brush.

[Gene Cornelius]

Thanks for the suggestion, Vito! I'm downloading the CS4 Extended demo now, and after reading the description of it, I'll probably end up buying it. I hope it works!

Even if it works beautifully, I think I will probably still do the landslide shoot and try for the interior shot, as those are fun-sounding challenges. I've never shot a model or in miniature, so I'm really looking forward to it.

I'll post back here if it works!

Thanks,

Gene