Blender Experiment: Camera Tracking

After watching Oliver Villar Diz's excellent and detailed tutorial on camera tracking using Blender, I was motivated to try it out myself. So on a recent trip to Florida, I shot a short piece of video that I planned to use (containing some lousy, voice-over acting.) As with any learning experience, I ran into some difficulties and made many mistakes. But this still ended up being a fun project and, despite being cheesy, I'm happy how it turned out.

Image Sequences

The tutorial strongly recommends that the movie clip should be converted to a sequence of images. At first, I planned on skipping this step because I didn't want the hassle of converting to and from different formats. But I went ahead and followed the advice, which worked out well since image sequences allow one to easily tweak individual frames of the result. Fortunately for me, iMovie HD supports exporting a video clip to a sequence of .PNG files, so it was easy to do.

The downside was that it took some effort to find a Mac utility that converts the images back into a movie file. Although iMovie was able to create the sequences of images, for some strange reason it doesn't support importing them! I eventually found ffmpeg for the Mac, which works great.

Camera Tracking

The point of this project was applying what I learned from the camera tracking tutorial, so my first effort was to try to get the camera to track my video clip. I found adding tracking points to the clip is a very time consuming process. The fitter needs a lot of data to accurately place the camera, so you spend a lot of time picking features that are easily traceable and following their path through the clip. The Blender automates much of the process so each point doesn't take too long to track. But the sheer number of points needed to produce the camera's path adds up so it still takes a while.

So after spending an hour, or so, I was ready to let Blender calculate the camera path. And that's when I ran into a problem.

Blender supports two modes of camera tracking: motion-based and tripod-based. With motion tracking, the camera has to move far enough in the scene that the traces provide enough information to compute the perspective. In tripod mode, the camera stays in one place and the trace points provide rotational information for the camera.

In Oliver's tutorial, he pointed the camera in one direction while it moved the length of a basketball court. That motion provided a lot of trace points so Blender was able to figure out the proper perspective. The motion in my clip, however, wasn't conducive to either mode: I had minimal motion while pointing towards the ocean and then, when I turned 90 degrees to the right, the camera also moved approximately the depth of the balcony. With this motion, neither mode was able to find a solution with a low error. To get it to work, I threw away all the track points I set in the ocean frames and just used the track points on the house. This worked much better and Blender was able to give a decent solution that I could use.

3D Model

We vacationed in Port St. Joe, Florida, which happens to be near Tyndall Air Force Base. My brother had heard the area has a reputation for UFO sightings, so it didn't take long to decide what the video clip was going to show.

The 3D modeling in this video was simple. I started with a simple saucer shape, and gave it a glossy silver finish. This was all I was going to do for the model but, when I animated the scene, it didn't seem "alive" enough. So I added detail to the underside, using a dark curved surface with glowing cyan spheres - you know, typical alien equipment. The bottom carriage rotates 45 degrees every 9 frames. Adding that little extra motion made it much more interesting to watch.

Next, I placed it in the scene. I used the more "stable" points (i.e. the points when it looked like the cameraman was focusing on something) of the camera motion to set the keyframes. The UFO then smoothly travelled between the points.

The last detail that needed to be done with the renderer was the lighting. To do this, I added a "sun" light source and tried to set its angle to match the shadows in the video. Although I didn't set up the saucer to cast a shadow, the light source created an accurate bright reflection.

Composition Editor

Using the composition editor was the most enjoyable part of the project because it's easy to use, fun to work with, and it provides much of the "magic" that makes the final clip believable. With the editor, I was able to:

• Adjust the contrast and colors of the rendered object to match the color, brightness, and saturation of the video clip. It would be very difficult selecting colors for the spaceship that would match the video's lighting, so I didn't even try. I chose some rough coloring and reflectivity for it and then used the composition editor to adjust it.

• Using the "motion blur" node, I could re-render individual frames where the camera motion blurred the background more than the motion blur from the renderer. This really added to the realism of the scene!

• Following Oliver's "green screen" tutorial, I set the sky as my key color (thank goodness it was a cloudless day!) I set up nodes to compute the mask and then merge the video with the rendered object.

Other Details

Even after creating the final scene, I found myself making tweaks here and there until I had to make myself stop.

• I made the flying saucer start moving before the camera began panning across the house next door. This made it appear that the cameraman was reacting to the saucer's motion. I also made sure the saucer reappeared from behind the house a little after the camera stopped.

• I added some "sci fi" sound effects from iMovie's library. When the saucer disappeared behind the house, I lowered the volume of the sound. When the saucer speeds off, I delayed the sound effect by a couple of frames to give the subtle hint of distance between it and the camera.

That's All!

Blender was intimidating when I first used it. But doing this little, throw-away project exposed me to more of Blender than I would have thought. I'm looking forward to learning more in future projects.

Thanks to BlendTuts for creating these tutorials and making Blender understandable!