DU/ART · AI LAB Making-of · EP.08 ← The Lab

◆ Making-of · EP.08 · a silent film

The film that almost didn't happen.

Ninety seconds, no words. Ninjas with COPY and PASTE on their headbands, and a samurai who says no. It took about 31 hours over nine days, and more than once I was ready to give up. This is what it really took.

01 First, where I stand

This lab is about automation. Never against creativity, and never against design.

I always say that first, because it's the part people get wrong. I automate the repetitive work so there is more time for the part only a person can do. Nobody should spend their day copy-pasting.

This film was different. There was very little automation in it. It was learning, creativity, and in the end, mostly persistence.

02 The numbers

Ninety seconds of film. Thirty one hours of work.

Counted from the working sessions themselves, not from memory. The real total is higher: it doesn't include the hours alone in the video editor, or the waiting.

~31hof active work, over 9 days
7attempts before a line of ninjas stood in the right order
10attempts at one small shake of the head
4refusals for a face you can't even see

03 What YouTube doesn't show you

Refused, again, and nobody says why.

The samurai is me. From behind, the video AI accepted him. From the front, with the face completely hidden in the shadow of the hat, it refused him every time, even with no photo of me sent at all, even with a request that had worked two days before. The message was one word.

On YouTube you see stunning AI films, and most of them are sponsored by the tool they show. That's fine, but it means you can't take any of it for granted. Until you test it yourself, you don't know.

We searched for skills, read the documentation, watched the tutorials. We still moved slowly. Some walls we never got through, so we changed the film instead: the hat that the video could never take off became a drawn page.

04 The storyboards kept changing

What we asked for, and what came back.

Every shot started as a drawing and a sentence. The AI had other ideas.

A house built right across the street
Asked for a streetIt built a house across the middle of it. My jump started from a roof that couldn't exist.
Giant figures next to tiny houses
Asked for a rooftopGiants beside toy houses. What I wrote that day: "a fine mess of giants and roofs. I don’t know whether to give up."
Crates appearing in a ninja's hands
Asked for a chainThe crates appeared in the first ninja's hands by magic. Nobody was handing them over.
The ninjas grouped by colour
Asked for white, black, white, blackSeven tries. It always grouped them by colour.
The samurai raising a fist
Asked for a quiet "no"A raised fist. The film is about saying no calmly.
The samurai turning his back
Asked for a shake of the headHe turned around and showed us his back.

05 What finally worked

Blender solved what words couldn't.

Describing a shot in a sentence wasn't enough. So every shot was first blocked out in Blender: grey boxes for the houses, coloured blocks for the people, the real camera move and the real distances. The AI then dressed that rough film. The colours are labels: white and black for the ninjas, orange for me, brown for the cart.

Grey blocking of the chain in Blender
Blender blockingWhere everyone stands, and how far the camera is.
The final shot of the chain
Final shotThe same composition, dressed.
Grey blocking of the jump in Blender
Blender blockingThe side roof, the drop, the landing spot.
The final shot of the jump
Final shotNo more houses across the street.

For the line of ninjas, what worked was drawing the first frame as a still image, in the right order, from the same camera, and letting the video start from it. For the hat, I gave up on video altogether: it became three drawn panels.

The first frame with the ninjas in the right order
The fix for the lineA still first frame, in order. The video kept it.
Three manga panels of the hat coming off
The fix for the hatWhere the video hit a wall, a drawing took over.

06 Putting it together

Images, videos and drawings, in one video editor.

Everything came together in a video editor: generated shots, still drawings brought to life with slow zooms and small shakes, the way Japanese animation studios used to when money was short. Then the colour, so shots from different AIs look like one film, and the sound, built in layers: many laughs at once, many blades, one explosion.

The whole film as a contact sheet

07 The film

Creativity and persistence.

It's not perfect. It's finished, and it's mine.

The samurai and his apps facing the ninjas in the rain

Is it just me? Which sentence in a meeting sends you off imagining? Tell me under the video.