Directing Virtual Actors: Making My First AI Short Film

"After The Cinema” is my first proper AI short film. Unlike my earlier experiments, this project was fully scripted with a much more structured workflow to really explore what AI video generation could do alongside my own filmmaking background.

A work-in-progress frame from the original Unreal Engine 5 version of After The Cinema, before the project evolved into an AI-assisted film.

The script, originally titled Almost in Love, was initially meant to be produced as an animated short using iClone and Unreal Engine, and was already in early development before I started experimenting with AI video generation. Eventually, I realized this might actually be the perfect script to test the process.

The film itself is heavily inspired by the works of Eric Rohmer, Hong Sang-soo, and Woody Allen. It’s a dialogue and performance driven piece focused almost entirely on character interactions, which became the main thing I wanted to test with AI: whether it could convincingly handle subtle performances and conversations.

Chris and Alex, the AI-generated leads of After The Cinema. Looking at certain frames, it’s honestly hard to believe they aren’t real actors.

For this project I used Veo 3.1 alongside the newly released Gemini Omni. While Gemini Omni produces visually polished results, I still find Seedance 2.0 better at understanding cinematic language and narrative filmmaking prompts. Gemini often feels more optimized for commercials and social media content, whereas Seedance feels more suited for longer-form storytelling.

The biggest challenge in making this film was performance continuity.

In traditional filmmaking, you shoot entire scenes and shape performances later in editing. But with AI generation currently limited to short durations, dialogue scenes had to be created piece by piece. I essentially had to plan the edit directly into the script itself, generating every line and reaction shot separately while hoping the emotional energy remained consistent between clips.

As I generated shots, I assembled rough scenes inside Flow before moving everything into DaVinci Resolve for final editing.

Inside Google Flow’s scene builder. The newer updates unify the tools and workflow into a single consistent environment, making scene assembly and iteration surprisingly seamless.

Music, which I think is one of the most important aspects of filmmaking, became another interesting experiment during the process. I attempted to generate the score using Gemini Lyria 3, Google’s music AI model, while using ChatGPT to help write and refine the prompts for the music itself.

The initial direction was inspired by the ending music from The 400 Blows by François Truffaut, something melancholic, reflective, and quietly romantic. The prompts went through several iterations before eventually evolving into the final track, “Midnight in the Stalls.”

I found it fascinating how close the process felt to working with a real composer. I realized prompting music was less about describing instruments and more about describing emotional memories, moods, and cinematic feelings.

.An unused shot set outside the cinema. One interesting limitation of the process was how generating scenes separately could slightly alter the characters’ behavior and emotional tone.

What genuinely surprised me was how well Gemini Omni handled performances. I gave minimal acting directions, usually emotional cues or small physical behaviors like frowns, pauses, squints, or head tilts, and most of the time the AI interpreted them correctly, even down to subtle micro-movements and vocal delivery.

The voices were also automatically generated, and surprisingly matched the characters fairly well. Newer features inside Flow now allow users to create proper character profiles with assigned voices and reference images, though I didn’t have enough time to fully use them on this project.

The biggest remaining issue is consistency. Small changes in prompts or reference images can alter characters, props, lighting, or even the mood of a scene, which means continuity has to be reinforced constantly through repeated instructions.

The tools are still far from perfect, but I genuinely had fun directing these virtual actors and experimenting with stories set in places and cultures that would have been impossible for me to produce traditionally at this scale.

For me, this project wasn’t really about replacing filmmaking. It was about discovering a completely new production language.