So we all do motion tracking in post using AE, Boujou, Mocha, etc.

Of course it is a pain.

I'm always thinking, why not simply record a X Y Z camera position data in the video file?

Kind of like the LOOKS are recorded and referenced?

This would require a (gyroscopic?) sensor attached to the cam at the time of video acquisition, I suppose.

Anyone has any experience with this approach in general?

How about doing this with SI-2K? The cam seems like a perfect candidate since it already requires a PC to capture/process its proprietary signal. Therefore modding it to add 3D camera positioning data to the signal seems more likely than other cams that output, say, HD-SDI signal.

Doable?

Hi

It depends do you want to get the direction from tripod, dolly, handheld?
1st is the simplest, just add 2 encoders to the tripod head and you get fairly accurate reading of angles.

2nd could be also done with different encoding or laser measure instruments, it could also be resolved by using tracking cameras, but then you need to add some markers for directions as well / not really recording to the cam head.

3rd requires a real positioning system to be used in camera recording something that has accuracy of grater than 1mm, you need to have at least 3 of these sensor, preferable 6 or tracking cam systems again.

But in the end of the day none of the systems would get you prefect solution for every situation because you have other problems than cam position, you have lens distortion, rolling shutter and motion blur to fight off.
Sometimes it is possible to "calibrate" the lens, but make sure you mount the lens same way up all the time because single pixel is 5 microns so distortion may be different already when you unmount a lens and mount it back (just how tightly you well get it back into the mount)

sometimes the financially most sane thing to do is add another synced camera to the scene that covers the entire shooting area, so you could do "fixed" solving.

if you need 3d geometry add slightly wider view stereo camera set on top of the original system, they do not have to be 2K but synced CCD HD would be my recommendation (as CCD do not have rolling shutter, best of my knowledge(maybe there are some perverse exceptions))

All magnetic, gps etc systems are useless because of large error.

-Kaspar

>> if you need 3d geometry add slightly wider view stereo camera set on top of the original system

Say, I'd like to add a 3D model into the live action video.

Camera is handheld and moves freely around the object.

The 3D model will be replacing part of the object (say, 3D modeled robotic arm instead of the human one.)

Can you please point me to the explanation of the workflow that involves a stereo cam footage as 3D camera tracking data source?

(Sorry if this is an obvious question, I'm a n00b at this right now.)