top of page

Real-Time Projections - An Expedition into the Future of Filmmaking

Welcome to the set of Love & 50 Megatons. A short film about the explosive love story between Paul the mechanic, and Mary the atomic bomb. For our Filmakademie graduate short film we had to orchestrate a firework of different techniques: miniatures, matte paintings, greenscreens, on-set live tracking and real-time game engines.

In order to finish over 200 shots in under three months, with a core team as small as 4 artists, we had to use cutting edge in-camera effects, we called real-time projections. Using this technique allowed us to capture hundreds of believable in-camera effects directly on-set and saved us plenty of time in post-production. While some VFX studios are starting to use this technique for big budget productions, this is still far from being common use, and I thought I’d share my experiences, in case any of you are interested in using these techniques in the future.

We began by building a highly detailed miniature set in close collaboration between the Director, DOP and Production Designers. For the live action shoot we then photo-scanned the miniatures and converted them into 3D Models within the Unity game engine. On-set we combined live camera tracking, using the Ncam system, with large projections of the scanned sets using two Panasonic PT-RQ32k projectors sponsored by LANG AG. Taking advantage of the parallax effect also allowed us to get maximum depth and movement to the background.

On-set we used a custom made App programmed by our TD Paolo to control the light, its color and focus. This meant that we could adjust the digital background as needed in real-time to get as many shots done in-camera as possible.

The in-camera effects made the shooting process much smoother, because instead of giant greenscreens, the on-set crew could actually see the world around them and were immediately immersed in the world of Love & 50 Megatons. Everyone was able to focus on the actual task at hand, instead of having to imagine the backgrounds. The Director and DOP were able to make creative decisions based on the actual world where the story was taking place. The actors saw and therefore felt their environment and occasionally approached me to see the 3d scene to better understand their position and walking routes. Using a real-time game engine made this process really easy and everybody was on the same page.

This approach, however, requires a very complex pre-production process, an intensive cooperation of all relevant departments, especially the Director, DOP and Production Designer. All scenes must be planned ahead in detail, all backgrounds, miniatures and effects must be done before shooting begins.

Time lapse of the 16 day shoot

In order to realize this enormous project, our goal was to use the most efficient ways in the history of movie effects and combine old and modern techniques in the best possible way. By combining modern real-time projections with classical miniatures it turned out that especially the combination of various techniques and intercutting between them made the sequences most believable.

In general most close-up and mid-shots were practical and had a projected background. Wide shots were either full miniature shots, matte paintings, or a combination of various techniques. If the background wasn’t exactly what we were looking for, we could easily switch to a solid background and had full control over its color and intensity. We always had a perfect greenscreen at hand simply a click away.

We also found the miniature models to work great with this technique and this movie in particular. Besides a love story, the film tells a sinister story about separation, nuclear destruction and propaganda. Using miniature models has its own unique charm and definitely makes it easier to enter and accept the bizarre world of Love & 50 Megatons.

When building models by hand, the materials also develop a life of their own and these certain imperfections contribute to an overall lively set, which would probably feel too perfect when working exclusively digital. We shot the miniatures with a motion control camera and later inserted greenscreen elements to bring them to life. To extend the sets we mostly used sky backdrops or added matte paintings later.

In my experience the biggest advantage of making a film this way is that you’re able to make important decisions on the spot, at the right time and you see right away what you are getting. While shooting, the Director and DOP always saw a finished composite, which made framing and lighting a lot easier. During the cutting process the Editor and Director could focus on timings and telling the story without being distracted by many greenscreen shots.

Considering the intensive pre-production, the time spent creating these shots overall is probably roughly the same compared to a greenscreen approach. However it forces all departments to work closely together from day one and moves up the schedule of post-production tremendously. It is a different style of shooting and a different approach to filmmaking altogether. However, having large projections of all backgrounds gave us massive creative freedom, flexibility and spontaneity. On-set we’ve been able to just try new ideas as soon as the background was set up and didn’t have to worry about creating additional vfx shots or complicated light interaction. Working this way, in post-production we could focus more on crafting beautiful images and creating a stunning world, instead of spending our time on tedious tasks like keying and tracking hundreds of shots.

Live tracking in combination with high resolution displays and real-time game engines are great additional tools for telling a story. In the future I imagine having a huge library with digital assets and environments to be able to shoot more with in-camera effects.

Creating this film truly was both: A stroll down the memory lane of visual effects as well as an expedition into the future of filmmaking.

Make love not war.

Film stills



bottom of page