Enhancing Visual Effects in Filmmaking
How ProtoPie is changing the game for filmmakers.
The film industry always seeks cutting-edge techniques to create and enhance imaginative worlds, incorporating digital props and post-production finesse. In this relentless pursuit, Kosmos has turned to ProtoPie, a leading no-code, high-fidelity prototyping tool, to revolutionize the landscape of visual effects and post-production in cinematic endeavors.
In this blog post, we explore the journey of innovation and creativity within the motion picture sector, spotlighted through an insightful livestream with Mario Baluci, a seasoned Visual Effects Supervisor at Kosmos.
For those who couldn't join the live event, we invite you to watch the recording below.
Overview
- From green screens to interactive prototyping
- Enter ProtoPie: From static images to dynamic realities
- ProtoPie demo for visual effects: a real-time showcase
- Benefits of using ProtoPie for better VFX prototyping
- Q&A recap: Perspectives in VFX prototyping
From green screens to interactive prototyping
Kosmos VFX is a pioneering visual effects company located in the heart of Italy. They are renowned for their extraordinary contributions to film and television, with a diverse and impressive portfolio featuring collaborations with industry giants such as Netflix, Prime Video, Paramount+, and Eagle Pictures, among others.
Keep reading to uncover their journey from traditional green screens to dynamic prototyping for their film projects.
What is screen replacement?
When one thinks of visual effects, the mind often jumps to the spectacular – explosions that shake the earth, starships that traverse galaxies, men taking flight, or dragons breathing fire. However, the core of Kosmos’ work lies in a seemingly modest yet crucial aspect of visual storytelling: screen replacements.
Screen replacement is a meticulous process that, as the name suggests, involves replacing the screen of a device by means of chroma keying – be it a smartphone, tablet, or computer – with a different image or video. This technique is pivotal in creating a seamless narrative, allowing for the integration of fictional content into the devices that populate our screens. Unlike other grandiose VFX work, screen replacements require sharp attention to detail and accuracy.
The role of green screens and markers
The magic behind screen replacements is often achieved through green screens complemented by markers. The green screen serves a fundamental purpose – it differentiates the parts of the screen that might be obscured, for instance, by a character's finger interacting with the device. Meanwhile, the markers, those little crosses that peek behind the scenes, are indispensable. They precisely track the movement and rotation of the screen, ensuring that the replacement content moves in perfect harmony with the device.
What is virtual production?
Green screen technology was once celebrated as a pioneering achievement in the 1940s. However, despite its futuristic appearance, it has aged significantly and now appears outdated. As its popularity grew in the 1980s, the limitations and drawbacks of green screens became more evident, leading to a shift in contemporary visual effects practices.
Kosmos has been at the forefront of this shift, moving away from traditional green screen technology for the past three years. Instead, they use a method known as virtual production, which involves projecting realistic backgrounds onto large screens within a sound studio. This enables actors to be placed convincingly within any desired setting, enhancing the realism of the visual output.
Enter ProtoPie: From static images to dynamic realities
Kosmos shifted from traditional green screens to embedding JPEGs and videos onto device screens. They experimented with PowerPoint and Apple Keynote for interactivity and used Adobe XD until it was discontinued. Figma was considered a potential alternative but was set aside due to its inability to function offline, which is crucial for film production in remote locations.
That was until Kosmos discovered ProtoPie.
With a career spanning over two decades in motion design and visual effects, Mario has witnessed first-hand the evolution of VFX tools and techniques. However, as he explored ProtoPie's capabilities, he quickly recognized its potential to revolutionize his VFX prototyping process. Here's a breakdown of the key factors that have made ProtoPie a preferred tool for Mario and his team.
Offline functionality
ProtoPie stands out for its ability to work offline. This feature is crucial for film production in remote locations where internet access is scarce, ensuring the team can proceed without interruption.
Intuitive and code-free
ProtoPie’s drag-and-drop interface is user-friendly, making it easy for anyone to create interactive and lifelike applications without coding expertise.
Realistic prototyping with device sensors
ProtoPie allows realistic app prototypes by accessing smartphone sensors. For instance, it is possible to simulate the behavior of a phone screen that dims when held up to an actor's face and brightens up when moved away, accurately replicating the behavior of a real smartphone during a call.
Fast learning curve and support
Thanks to comprehensive tutorials and support from the ProtoPie team, Kosmos was able to integrate ProtoPie into their workflow efficiently. Within months, they went from beginners to proficient users, creating complex prototypes that enhance their visual storytelling.
Continuous updates and innovation
ProtoPie's commitment to refining and introducing new features through regular updates has been a significant advantage.
These reasons clarify why Kosmos has embraced ProtoPie to enhance their visual effects prototyping, making their process more efficient, realistic, and innovative.
ProtoPie demo for visual effects: a real-time showcase
The highlight of the session was Mario's live demo. He skillfully demonstrated ProtoPie's application in a visual effects scenario, taking the audience through creating an interactive app prototype designed for a fictional movie scene. The demo showed that ProtoPie can simulate real-life app interactions and create cool visual effects on the actors' devices, allowing them to interact naturally with a physical prop.
Benefits of using ProtoPie for better VFX prototyping
In the live stream, Mario highlighted the numerous advantages ProtoPie brings to visual effects prototyping.
- While green screen technology has been the industry standard for years, ProtoPie presents a groundbreaking approach to replacing green screens with tangible screens.
- ProtoPie empowers designers and VFX artists to create dynamic, interactive visuals without coding.
- It enables the direct application of prototypes in live production environments—even without an internet connection. This shift from traditional post-production methods to real-time, on-set execution results in instant outcomes, allowing actors to interact naturally with physical props.
Curious about how ProtoPie can elevate your VFX prototyping workflow? Schedule a demo session to learn more.
Q&A recap: Perspectives in VFX prototyping
The Q&A session opened the floor to a wide range of questions, ranging from ProtoPie's technical capabilities to its impact on the future of VFX. Below are some of the highlights.
Q. What is the actors’ reaction when they see these prototypes instead of green screens?
Mario: They think it's magic. They love it because they can interact with something real, like Adam said, with living props. I have definitely seen actors improve their acting and even change the line because they had something real on their hands.
But not just actors. Also, directors love it. Producers love it.
The actor loves it because he interacts with something real. The director loves it because you can see exactly what's going to be filmed—something that is not something green that is becoming something else impossible. Producers love it because it's cheaper. But it's not cheap because it's cheap.
It costs less, but it's a way better solution than green screens. You have better quality, and it costs less. So it's a win-win deal for everybody.
Q. Can you share insights or tips for effectively communicating and collaborating with directors and actors during the prototyping process?
Mario: Yes, it all starts with the script. We receive the script and read through it to identify any interactions with a smartphone, tablet, or similar devices. However, that's often not sufficient because the script doesn't always capture exactly what will be shot.
For example, scripts frequently mention that we start from the screen of a phone, which isn't always the case. There might be a wider shot where the actor is walking and then sitting, and during this time, whatever device they're handling needs to be doing something.
Take another instance: if it's mentioned that Mike takes a phone call, we need to consider where Mike's phone is beforehand—is it in his pocket, in his hand, and what is he doing with it before taking the call? Is the caller already in his address book? These are the types of questions we need to ask ourselves and the director as the director decides on the final actions.
In my presentation, I mentioned the advantage of having prototypes with ProtoPie in the cloud. It's incredible how many times I had to change the prototype live on set because we found a better approach or the actor wanted to perform differently. I had to take my laptop, make some modifications, re-download the prototype to the phone, and we were ready to go in 5 minutes.
This flexibility is a part of the process, and it's essential to be prepared for it. It's crucial not to assume that your prototype is ready to go just from reading the script. Instead, try to envision the scene as realistically as possible, not just as it's written for the movie.
Q. Can you share how you started in the industry and your thoughts on the current state of the industry with the rise of AI in filmmaking?
Mario: Okay, this is not a simple question, but I'll try to shorten it as much as possible.
I started with a passion for computers. We always had computers at home because my father is an engineer. And so I started to mess around with Windows 3.1 stuff a long time ago. I discovered my passion for design, so I started to draw with a computer. And then, I moved to motion design.
And from motion design to visual effects and now to prototyping. When I was 18, there were no schools to go to, so I learned by myself. The world had no internet, there was no YouTube. So it was books and friends who knew stuff.
And concerning today's world of visual effects and the AI thing. I think we are looking at a revolution. That it's something that is going to change forever because of artificial intelligence. We are intensively using it already. It has been integrated into our software since Photoshop and After Effects. We use Midjourney a lot.
We have seen what Sora can do from Open AI, so we are trying to go as fast as the evolution is going. We are looking at it very closely, trying to understand how things will evolve so we can act accordingly.
Q. How do you avoid display reflections in all these different use cases where prototypes are used?
Mario: This is something we handle with the director of photography. If there is a reflection, we just have to ask the actor to rotate his hand differently or move the lights on the set so that there are no reflections.
Q. Do you have a coding background? Is it needed for ProtoPie?
Mario: A coding background is not needed to use ProtoPie. I don't have a coding background myself. I can probably read some JavaScript and HTML, but I cannot write any code.
What is very useful to have is an IFTT logic in mind. The IFTT is “if this then that”. Because this is how this works in ProtoPie: something happens as the trigger and then something else happens as a consequence. If you have this logic in mind, this is ProtoPie.
Q. How long did it take you and your team to become proficient in ProtoPie?
Mario: I will say a month. The first movie we worked on was shot in November, and at the very beginning, I was scared to use ProtoPie immediately, so I started to make my prototypes both in [Adobe] XD and in ProtoPie.
I tested them on the set. I immediately abandoned [Adobe] XD because ProtoPie worked so well immediately, even if I had just one month of experience.
Q. How much time do you estimate you save with ProtoPie?
Mario: I save a lot of time because, let's say, an average prototype for a scene can take one day to complete. Post-production of a green screen scene can take even twice the time. And especially if the scene is not supervised, which often happens, we don't always have the chance to be on set.
Very often we receive the footage that has been shot without almost any attention to what has been done in post-production. We have to struggle to really postproduce it in a way that is at least decent. Let's say it takes at least half the time for a way better result.
Transform your VFX prototyping workflow today
Mario Baluci has shown how ProtoPie is changing the game for filmmakers. It makes VFX prototyping easier and allows for more creativity, helping bring movie visions to life with more detail and interaction. Join them in transforming your VFX prototyping process now.
