Skip to content
Keshav Raj
A robot editor syncing several camera feeds by sound waves with an HDR monitor and a reel plan.
AI DISTRICT381 TESTS
SHIPPED Multi-camera FFT sync 5× faster; HDR tone-mapping; reel planner.I OWNED Sole authorOUTCOME 381 TESTS

AI video director

Keshav RajSole author · 2026 · PrototypeContact meResume PDF

Context

This is a personal project I built as the sole author. It turns long multi-angle appointment footage into short Instagram Reels for a small business. It is a prototype that runs locally.

The problem

Turning hour-long footage from several cameras into Reels was manual work: line up the angles, fix the colour, pick the good moments, cut and export. I wanted a tool that does the heavy parts and lets me approve the result.

What I did

  1. I wrote the multi-camera sync. It uses FFT audio cross-correlation, reports a confidence value and detects drift between cameras. I then optimised it to run 5x faster with a third less memory.

  2. I handled HDR footage (HLG, Dolby Vision and PQ) by converting it to Rec.709, with fallbacks when the machine lacks a capability.

  3. I built a lease-based job queue so the FFmpeg worker can restart without losing or duplicating work.

  4. I added a reel planner and renderer, and a Gemini analysis step that only labels phases when there is visible evidence. Cloud AI is off by default.

Key decision

I kept cloud AI optional and off by default, since the footage shows real clients. The cost is that the local path has to work well on its own.

I also split the project into a pnpm monorepo with a web app and a separate worker, so long FFmpeg jobs never block the interface.

Result

The prototype has 381 tests and produces reels locally. It is not deployed. I wrote about 24.9k lines of TypeScript over 18 commits in about two days, with AI assistance.

The headline 381 tests cover the sync, the HDR conversion, the queue and the planner.

What I learned

I would again put timing and queue behaviour under tests first, because those are the parts that fail quietly. Next I would run it on a second shoot to see how well the sync and planner generalise.

Stack

  • TypeScript
  • Next.js
  • Node
  • FFmpeg
  • SQLite
  • Drizzle
  • Gemini
  • Vitest
Play the world →