Is Virtual Production Undergoing a 2024 Glow Up?
Virtual production has been influencing production in film, TV and advertising since the covid-19 pandemic. Its potential to save money via bundling and to cut down on carbon emissions is taken as read, and it has also been praised for its power to unshackle creatives from the confines of domestic settings.
However, years in, it’s also clear that virtual production is still a work in progress and presents certain confines of its own, resulting in a signature VP ‘look’ and, sometimes, stilted and non-dynamic action and camera moves.
With all that in mind, LBB’s Addison Capper wanted to speak to experts in virtual production to find out how it has evolved, how they’re finding solutions for the limitations, and where the future lies.
Alan Bibby
Director at Psyop
The reality is that the evolution of this technology is still unfolding before our eyes and pioneers are rarely cognisant of their pioneering, especially while the iron’s still hot.
For now, we as an industry primarily view virtual production as being the answer for production issues where tighter budgets impose challenges when deciding where to shoot, how to shoot and for how long on location. These perks are all valuable and great, but leaning on this singular tool as a catch-all problem solver puts the production on a one way track to landing a one-note, uniquely VP 'look'.
It’s the same effect as if you were to use a steadicam with a 50mm lens for every shot. You’d never do that. You’d use the tool where it works to tell a specific story.
Collectively, we need to shift the mindset from looking at it as just an answer to some common problems to seeing it as an integral addition to the toolbox which frees creatives up to dive into the content deeper. What amazing ideas couldn't we shoot because they were too dangerous to do so? Or, better yet, too expensive? Zooming out and thinking about it from another direction we see that if money is no object it isn’t just good news for profit margins, it’s great news for filmmakers that have a phobia of limitations and would love to do crazier and crazier shit for their clients.
This is an extract from a longer piece that Alan wrote about virtual production. Check it out here.
Raphael Ruz
Creative innovation lead at Hogarth
The idea that virtual production (VP) results in a signature 'look' or stilted action isn't accurate in our experience. At Hogarth, we have developed thoughtful, robust evaluation standards to ensure VP is fit for purpose on the jobs we use it for. Without good evaluation criteria, you might feel confined by a specific look or other limitations, but this is true for any technology or equipment, not just VP.
We've found ways to use VP across various workstreams, benefiting clients and freeing creatives. It's not just about tier one shoots. Our global network of shared resources allows us to continuously refine our methods and push the boundaries of what's possible with VP.
Some of our recent work has included: shooting fiery bushfire locations without risk to the environment; tabletop products all across the globe in one studio; and a virtual store that doesn’t need to be closed for a traditional shoot – reducing the impact on our client’s place of business, and their consumers.
Looking ahead, the fusion of ideation and execution will become even more seamless. As generative AI moves into the 3D space, the power of this technology will take another leap forward. Imagine manipulating a set in real-time with just a few words - that future is approaching fast.
In short, virtual production is not a static or limited tool but a dynamic and evolving technology. With the right expertise and continuous innovation, it offers unparalleled opportunities for creativity and efficiency in production.
Morgann Brun
Production manager at La Planète Rouge (part of Global Production Network)
As a burgeoning technology, virtual production has made significant waves in the realms of film, television, and advertising, and its impact continues to reverberate throughout the industry. While some implementations of virtual production have yielded a distinct ‘signature VP look’, forward-thinking companies like La Planète Rouge are actively challenging the boundaries of this technology. Recognising the importance of research and development (R&D) in keeping pace with innovation, these companies are committed to pushing the envelope and ensuring that virtual production does not become confined by its own conventions.
La Planète Rouge's recent partnership with the Parisian visual engineering company Spline exemplifies this commitment to innovation. By integrating Spline's robotic arm with virtual sets, they have unlocked new possibilities for dynamic shots, slow-motion sequences, and high-speed cinematography. While achieving this technical feat required extensive R&D efforts, it has empowered directors to experiment with novel camera movements within virtual environments, enhancing the creative potential of virtual production.
Moreover, the ongoing exploration of artificial intelligence in virtual production represents another frontier in R&D. By leveraging AI technologies, filmmakers are discovering new avenues for content creation, including the integration of motion capture directly within virtual sets. This not only streamlines production workflows but also opens up new creative possibilities for storytelling and character animation.
In addition to these advancements, the industry is actively exploring emerging technologies such as extended reality (XR) and frame remapping to further expand the capabilities of virtual production. These efforts underscore the dynamic nature of R&D within the virtual production landscape, where continual innovation is driving the evolution of both technical capabilities and artistic expression.
In essence, by embracing a culture of experimentation and investing in R&D initiatives, companies like La Planète Rouge are shaping the future of virtual production, ensuring that it remains a vibrant and transformative force in the entertainment industry.
Brian Collins
Senior producer at Havas New York
Since March 2020, production has taken some huge steps forward, with VP coming to the forefront as the latest ground-breaking tech. At Havas, we’ve been fortunate enough to experience a variety of VP offerings to commercial production, from executing realistic physical spaces, to more fantastical worlds, and even to hybrid VP-green screen approaches. In some cases, it’s been a great success: the economic and logistical hurdles of shooting in dangerous terrain like a volcano or dark cave can become reality for any production - safely! We’ve all been familiar with shooting in large studios even before the pandemic, so implementing VP in those spaces has helped us quickly adopt the new technology. The performers need no mental leaps to immerse themselves in their characters’ worlds, and the integration of Unreal Engine with live-action shoots is truly amazing.
On the flip side, the economic benefits aren’t as drastic as you might expect. In execution, it’s not so much different from green screen, only we have MORE on-set tech, and we are still beholden to the same limitations we have for any green screen shoot. (It will always be complicated to seamlessly blend the floor.) Normally our pre-pro phase would be front-loaded with practical needs, in VP, a closer partnership with post and VFX teams is instrumental as well. And while any producer would welcome a closer tie from pre-pro to post, that actually means more time, energy, and of course money in our productions. And you still have to refine VFX, colour grade, and composite, so there’s not much time saved in post.
Surely in a few years time, VP will come to be a standard in CGI much like green screen, and we’ll find ways to improve the aesthetic and capabilities. There are certainly worthwhile benefits that validate the hype, but it’s not a complete solution. Much like the rapid evolution of AI in the production sphere, VP should be considered a new tool rather than a revolution. Production will always come down to the quality of the creative and a skilled team. At the end of the day, what’s most important is a camera and an idea. And now VP brings a lot more ideas within reach.
Julie Reali and Lesly Lynch
Founders at Space Cowboys
Generally, I think we are in a transition period of a transformative journey. Technologies like LED volume stages combined with Unreal Engine and motion capture have personally opened doors to explore new ways to approach filmmaking.
For instance, you can use LED walls to project vast surreal 3D digital landscapes. The immediate benefits - dynamic lighting and on-the-fly background changes - are palpable. You can animate characters in real-time with motion capture technology. This allows live performances to be captured and instantly translated into digital character movements within the Unreal environment, ideal for complex animation scenes involving CGI characters interacting with live actors. These used to be luxuries reserved strictly for post production.
For sure, it comes with its set of challenges. There's a distinct ‘VP look’ that sometimes emerges, where visuals might not appear as photorealistic as those captured on traditional sets or locations.
We think it’s mostly about choosing the right project for that tech, with a creative direction that would fit this type of look and feel.
One innovative solution we're excited about involves integrating AI-driven tools to create new possibilities of interaction between actors and the virtual environments. You can use AI to adjust lighting and shadows dynamically based on actor movements to enhance realism. But what makes our brains sparkle is to go further and create real time interactivity, such as having the 3D elements of the backgrounds evolving based on actors' movements. Imagine a dancer moving through a field, causing waves to ripple through the wheat, reacting in real time to the performance. It's an extra tool in your toolbox, you can choose to use it or not, it can expand a vision and allow you to explore new artistic possibilities.
On an ethical note, embracing virtual production is also a commitment to sustainability. It significantly cuts down on travel and the logistics of location shoots, reducing our carbon footprint. It's not just about creating content efficiently; it's about being responsible creators in a world that needs us to be mindful of our environmental impact.
Nicole Gabrielle
VP, director of integrated production/content, North America at Momentum Worldwide
Virtual production has sparked a revolution in filmmaking, changing how we think about cinematic storytelling with its advanced XR (extended reality) LED stage technology. This innovative method allows for the creation of vivid, immersive environments that respond dynamically to camera movements. For instance, shows like The Mandalorian have showcased how effectively this technology can create expansive deserts and futuristic cities, making the digital seem almost indistinguishable from reality.
At its heart, virtual production uses LED screens to transport actors to virtually any location imaginable, all from the comfort of the studio. This technology dramatically reduces the need for on-location shooting, making it easier and faster to change environments, and also allows for higher productivity and the ability to capture more deliverables faster. However, despite its impressive capabilities, virtual production isn't flawless. The high cost of setup and operation can be prohibitive, particularly for smaller budgets. Additionally, the challenge of the ‘uncanny valley’ can disrupt the immersive experience. Lighting discrepancies between the LED screens and the physical set can also detract from the realism, necessitating precise adjustments to maintain the illusion.
Despite these challenges, filmmakers are adapting and overcoming these limitations with remarkable creativity. By blending practical effects with digital backgrounds, they enhance the authenticity of the virtual environments. Moreover, advancements in real-time rendering engines are smoothing the interactions between actors and their digital surroundings, making these elements integrate more seamlessly.
Virtual production is becoming more accessible as the technology becomes more cost-effective. We are likely to see a wider adoption across all levels of film production. Upcoming innovations in rendering technology and machine learning promise to refine these integrations further, making the digital and physical elements blend so smoothly that they become indistinguishable.
As virtual production continues to evolve, it not only stands as a testament to the resilience and creativity of filmmakers but also as a driving force in the future of storytelling. With these technological advances, creators are not just telling stories, they are expanding the horizons of what can be achieved in production, pushing the boundaries of our collective imagination.
Tamer Shaaban
Director at Durable Goods
When covid hit in 2020, anyone who was in physical production felt the pain of staying at home. It was also the year that Epic Games released Unreal Engine, a solution to filmmaking that really flipped the process on its head. Curious, I decided to dive in and learn more about it. Four years later and it has been the answer to a lot of production questions that caused me to compromise in the past. In the last six months alone, I've shot twice on virtual production projects and pitched on five more in the first quarter of this year alone. Needless to say that this isn't just the future, but a method that's here to stay; and we'll watch the pre/post production process evolve entirely with it. What's been most interesting about the process is not that it saves money (which in many instances it can) but its ability to compromise less on the original vision and increase your scale of worldbuilding more. To do this though, it requires a level of understanding of the technology's capabilities that can be applied from the creative process to the post production process. However, once understood, it allows a scale of work that enhances the level of storytelling that is only bound by the limits of the creator's imagination.
Tasha Jameson
Executive producer at Makers
These days, we see more offerings for LED volume walls, such as different wall sizes, wall type/resolution, package of accessories (sky panels, robot arms), real engine teams at all skill levels. With more options available, we’re able to map the right project to the correct wall set up and team, based on creative needs, budget and timeline.
Current limitations of the wall can be broken down into three categories.
Technical constraints:
- Limits on shooting slow-motion (maximum frame rate of 240 fps).
- Challenges with high dynamic range colour space rendering on LEDs.
- Difficulty fitting large amounts of lighting sometimes required in smaller volume wall environments.
- Unreal Engine digital look/ moiré effect, especially in wide shots or where the wall is in crisp focus.
- Limitations on stunt rigging.
- Issues on sound recording (due to large sound reflective surfaces where there should be open air).
Creative limitations:
- Framing that requires viewers to see the floor or ceiling can be tricky. Curved LED walls meeting square floor treatments need foreground elements to hide the seam.
- Difficulty in shooting scenes requiring interaction between foreground and background elements or props.
- No live pyrotechnics.
Operational challenges:
- The expense. The main cost drivers are the LED walls themselves, the crew to operate them and the computational power required for live renders.
- Longer pre-production timelines for heavy background builds.
- Without an experienced producer and production company at the helm, matched with a tight scope of work, it’s easy to run into budgetary and organisational hiccups.
- Limited understanding and experience in LED volume wall shooting within the production community, leading to potential overselling of capabilities.
- Longer set-up times in single-wall setups (no ability to leap frog set-ups).
- Greater expense if shoot pushes into over time due additional crew to operate the wall and increased OT prices for the studios themselves.
Where does the future lie? From a producer's point of view, this technology continues to offer exciting new ways for us to support creative and brand teams in realising their most ambitious visions, while mapping back to the realities of their timeline, budget and commitment to decarbonizing their productions.
Unreal engine (UE) designers are becoming pivotal new members to our production teams. Having a skilled UE artist with knowledge of lighting and the ability to adapt scenes quickly (even live on the shoot day) is a tremendous build upon our current workflow. Now, there is an expansive database of backdrop options for us to pull from in the moment with the click of a button. The key is ensuring collaboration between a skilled team of lighting, art department and UE artist matching the foreground and background to gel everything together as organically as possible.
As adoption grows, we hope costs will decrease, making this technology accessible to projects of all sizes.
Sam Barbrooke
Head of content at CHS
At CHS, our roots in traditional photography have taught us much about the evolving landscape of content creation. With over 50 years in the field, we've seen first-hand the shift towards a digital-first approach, driven by an ever-growing appetite for content amidst unchanged or shrinking marketing budgets.
Embracing virtual production has felt a natural progression for us, blending our love for realism and creativity with the efficiencies and possibilities of new and modern technology.
Our foray into 3D generation, AI, and VP panels is less about adopting new tools for the sake of it and more about finding ways to continue delivering high-quality content that goes beyond expectation, in a world that constantly craves more.
We've learned that our background in photography isn't just a skill set, but a lens through which we view the possibilities of virtual production. Challenges such as balancing lighting, crafting scenes in the mid and foregrounds, propping and seamlessly layering these elements are challenges we've always faced, but now presented in a new medium.
Not only that, the cost, time, and environmental efficiencies of virtual production have broadened our ability to serve a more diverse clientele, by being able to produce high-quality content, quickly and across a multitude of budgets.
However, our journey into VP has reinforced our belief that the heart of content creation remains unchanged: it's about telling stories that resonate.
As we navigate the possibilities and challenges of VP, we're reminded that technology is a tool, not an end. It's our creative vision, grounded in decades of experience, that often guides us through the evolving demands of content creation in advertising.
