At its best, video production is like a good Swiss watch. It starts with a complex set of separate parts with different functions, but the end goal is to produce a single cohesive product with no visible imperfections.
With the many different elements at play in a typical video project (development, filming, editing, sound, color, graphics, titles), it can be tricky to produce a cohesive final product. However, the more tightly you can integrate the moving parts in the creative process, the easier it is to keep everyone on the same page. This is why some of the most significant advancements in modern video production have been in service of building bridges between the different components of filmmaking.
With that in mind, we recently became aware of an exciting new breakthrough that manages to bridge two realms of video production that are notoriously difficult to integrate – filming, and visual effects (VFX).
On-Set Facilities, a VFX company based in the UK, has developed an impressive new system that combines several technologies into an incredibly useful new tool for filmmakers. The company calls it ‘Live VFX’ – a fitting name, given its abilities.
The video demonstrates it best, but it merits an explanation – Live VFX allows filmmakers to deposit real on-camera subjects into a virtual space, in real-time, during filming.
To be clear, we're not talking about layering existing footage into a stationary camera shot. That type of live on-set green screening is relatively easy (just ask any weather man). This is more like real-time three-dimensional green screening. A virtual room that stays anchored in place, letting the camera and subject move freely within the shot.
What makes this truly significant is the fact that it's happening so seamlessly in real-time. Up until now, the logistical challenges of shooting a real subject for a virtual environment have been numerous. This often leaves filmmakers with less creative freedom, not more. Rather than imagining the new possibilities afforded to them by using green screens, they're more concerned with how well the real stuff will integrate with the artificial, once the footage is brought to the VFX lab.
Think of it this way – before smartphones, if you wanted to combine two photos into one composite image, you had to snap multiple variations of both photos on a digital camera, import them onto a computer, and try different combinations in Photoshop to see if any of them worked. If not, you had to either go back and shoot more photos, or settle for a sub-standard final product. With today's mobile technology, we can see if the combination works immediately, because the photo editing/compositing software is now on the same smartphone we're using to take the pictures. Think of Live VFX as the video equivalent ... instant video compositing.
To achieve this technical wizardry, On-Set Facilities uses a combination of three key technologies: green screening (or chroma key compositing), motion tracking, and a virtual reality game engine.
The first of these, green screening, has been used in video production for decades, with varying degrees of success and believability. It’s quite challenging for actors and directors to know if the footage they're capturing will look good in the final shot, especially if they have no idea what the visual effects will ultimately look like while they're filming. This quirk of modern filmmaking is ripe for comedy, which is why it’s the subject of a reoccurring sketch game on Whose Line Is It Anyway.
The other two technologies involved – motion tracking and virtual reality – go hand in hand already. The realism of VR relies heavily on tracking the movements of your head, while the virtual world around you remains 'stationary' to your point of view. The head-tracking hardware in those VR headsets may seem bulky when it's strapped to someone's face, but it's works remarkably well as add-on hardware for a full-size video camera.
Breaking it down like this, we see that it was actually the VR engine that served as the final (and most significant) piece of the puzzle in making Live VFX possible. Traditional green screening has certainly given us an abundance of movie magic over the decades, but instant on-set video compositing was the next giant leap that simply couldn't have been cleared without the recent breakthroughs in virtual reality. That's right – we just tricked you into reading another VR article. *mic drop*
