Resistance: Where the Filmmaker Still Matters

Every clip begins with the frame.

Resistance originally started as a short film concept I wanted to make with iClone 8 and Unreal Engine about two years ago. I even made a test or teaser during my early days of learning animation, so it's pretty crude and clunky. For those who are interested, you can watch it here.

https://youtu.be/bT-eIwA67Kw?si=znmawPUhEgEnbHCE

But after two years, my animation skills are still mediocre. Animation is tough and it requires a lot of practice. It was always in my plans that once I got better, I would go back to Resistance. And then AI video models arrived.

That's when I thought maybe I could actually do this with AI. After a little more than two months of creating shorts, music videos and films, I thought I could pull it off.

The original plan was to create the images or frames with Nano Banana and use Seedance for the video generation. I did a preliminary round of tests and even converted Jane, the original 3D protagonist of the film. But eventually I felt I wanted the film to be more grounded in realism, something closer to the feeling of I Am Legend, The Walking Dead or Arrival.

Here's Jane a direct conversion from the 3D character, and still she looks like a famous personalities.

So I reworked the whole tone of the film and landed on the present character, Elena Morales.

Just a note, I didn't intend to create any recognizable real person. This is also because of Dreamina's strict policy. It won't generate a real-life human that looks similar to an actual person. Nano Banana first gave me a character that was hard not to notice. I ran a few more generations, and eventually I saw her. She looked and felt like Elena.

So I had a few backups just in case Seedance refused to generate the character, and sure enough, it didn't work.

Then the open-weight version of Minimax H3 came along, which I had already used for two music videos and a few clips. I finally decided to go with it, keeping the entire video generation in Minimax H3 through ComfyUI. And the big plus point is that it's free.

My present workflow Minimax H3 in Comfy UI.

I'm not going to do a full review of the model, but Minimax H3 is quite capable. The entire film was generated with it except for the scenes where the killer alien drone enters and attacks the armed men. Those were generated with Seedance 2 Fast in Dreamina.

Personally, I've written many times by now that Seedance wins in the action department. The firefight between Elena and the men, however, was all generated in Minimax H3.

For very simple shots, I generated them locally on my PC with an RTX 5060 Ti 16GB VRAM, which takes quite some time depending on the resolution and duration. It averages around 15 to 20 minutes per clip, and if I increase the steps it can go up to 40 minutes.

For the action and other scenes towards the end, the shots needed a lot more trial and error, so I used RunningHub and rented GPUs, which sped up the process. And this is still way cheaper than using paid AI models. A 15-second clip costs an estimated $0.14.

Organization is key especially working alone and in multiple apps.

Here's how I made the film.

I used ChatGPT to help me create the characters, drone, mothership, locations and other elements, with all the images generated with Nano Banana. I then proceeded to generate the keyframes for the shots and scenes I had in mind.

Once I had locked down a frame, I used storyboard prompts to instruct Nano Banana to create storyboard panels based on the scene description. From there I extracted the best images, and usually you're lucky if you get three good frames from a storyboard.

One thing I would really recommend is to spend more time on the images and keyframes before going into video generation. I find that having a strong image as the starting frame gives you a much better result because the video model already has the composition, character, environment, lighting and overall look to work from. Compared to reference-to-video, where the AI still has to figure out the shot and generate much of it from scratch, starting with a strong frame gives you much more control over what you want the final shot to be.

One of the storyboard generated by Nano Banana with a storyboard prompt.

Just a note, during the image generation I already know that it needs to look like a movie, so the placement of the camera is key.

Even though I have been directing and shooting films for almost 20 years, I never really thought about this concept until I started learning animation. In one tutorial, the animator explained that the reason some animations don't feel real or cinematic is not just the design and lighting, but where you place the camera.

In a 3D world, you can put the camera anywhere you want. And I think this is one of the reasons things can sometimes break the cinematic feeling.

If you place the camera realistically, as if you're actually on a film set with limitations and physical constraints, you get more believable and cinematic shots.

But this really depends on the vision and genre. A camera that flies from the sky, lands on the ground and then flies into a building can be super cool, but sometimes it's also when you start to feel that you're not watching something real.

AI's greatest challenge, for me, is spatial awareness and continuity.

A normal drama between two characters talking is not much of a problem these days. But for an action scene that involves a lot of position changes, it's still very difficult to maintain spatial continuity with the current workflow.

So for the action scenes, I have to rely on the good old montage editing. Cutting up random footage and putting it together into something coherent, while establishing an idea of space, action and story. That's how I approached the firefight scenes on the large office floor.

Lastly, the prompts for the videos.

I'm actually surprised by how simple it has become for me to write them without following the structures that are always recommended by the community.

Seedance is smart, so there is no point giving it a very long descriptive prompt. Sometimes it can actually make the result worse.

As for Minimax H3, most of the videos I made didn't even use the structure, which is actually quite complex. I used a much simpler approach. I just write like a director or a screenwriter.

The only few times I used the more complicated structure was when I couldn't get something right, especially when it involved simple movements like walking from one point to another.

Just a note, I wrote 99% of the prompts myself. I only used ChatGPT or Gemini when I couldn't get something right.

There's a reason I chose to write them myself rather than relying on AI to help.

I used a Seedance director skill with Gemini for The Yokai Journal: The Mask of Tengu, and it was amazing. It was so effortless and easy. But I realised afterwards that I didn't really learn much from the process.

And I think this is probably the one last part that still needs the filmmaker.

If I automate this part too, then eventually, what's the point of the filmmaker?

In fact, the music video prompts are also written by me, purposely avoiding relying too much on AI. I mean, writing is probably one of the few skills left that is still important to the filmmaker or creative.

So I hope you enjoyed the film and this rather long post.

I'll still be making more films, so please feel free to subscribe to my channel. Although, at the point of writing this post, my channel has recently been demonetized, so I'm now really doing this for free and for the love of the craft.

And honestly, that's probably enough reason for me to keep making them.

There are still a lot of stories I want to tell.