ACT3 AI — filmmaking platform
Script to storyboard to cinematography to render in one web app. Two years as a top-three contributor on the client and the API.
Background
ACT3 AI is a professional tool for making animated shows with generative models. Writers edit scripts, the system proposes storyboards and shot compositions, and a render pipeline produces frames and video. It is multi-tenant with organisations, credits and Stripe billing.
I have worked across the whole product since 2024: the Next.js client, the NestJS API, a FastAPI service for script generation and RAG, and the Python Blender automation for shots and upscaling.
The problem
Generative video is non-deterministic and expensive. The product has to let a writer iterate on a shot dozens of times while keeping characters consistent, keeping cost visible, and keeping the UI responsive while a generation runs for minutes in the background.
Approach
- Client: built the new home surface with design tokens, frame drill-down and a rendered-frame stage, so a writer sees the episode as frames rather than as a job table.
- Seedance prompt tiers end to end: tier selection in the client, prompt families and generation queues in the API, so cheaper tiers are the default and expensive ones are a deliberate choice.
- Storyboard first-frame cast-reference resolver, so each shot's generation is seeded with the right character references and faces stay consistent across a scene.
- Script and scene generation in the FastAPI service on GPT-4o, o3 and Grok, with formal and informal screenplay import and a copyright filter.
- Procedural Blender shot and camera-path generation, Real-ESRGAN upscaling and episode assembly in the Python visual-automation pipeline.
- BullMQ generation queues with WebSocket progress, a content-safety layer and Sentry across client, API and services.
Architecture
Outcome
- Production platform with paying organisations, in continuous development since 2024.
- Third-largest contributor on the client and fourth on the API over two years, across a team of about ten engineers.
Lessons
- On a large team codebase, the highest-leverage work is the seam between systems: the tier pipeline touched client, API and runner, and that is where the product value was.
- Generated API clients from OpenAPI pay for themselves in a week on a team this size.