Context
This is a personal project I built as the sole author. It turns long multi-angle appointment footage into short Instagram Reels for a small business. It is a prototype that runs locally.
The problem
Turning hour-long footage from several cameras into Reels was manual work: line up the angles, fix the colour, pick the good moments, cut and export. I wanted a tool that does the heavy parts and lets me approve the result.
What I did
I wrote the multi-camera sync. It uses FFT audio cross-correlation, reports a confidence value and detects drift between cameras. I then optimised it to run 5x faster with a third less memory.
I handled HDR footage (HLG, Dolby Vision and PQ) by converting it to Rec.709, with fallbacks when the machine lacks a capability.
I built a lease-based job queue so the FFmpeg worker can restart without losing or duplicating work.
I added a reel planner and renderer, and a Gemini analysis step that only labels phases when there is visible evidence. Cloud AI is off by default.
Key decision
I kept cloud AI optional and off by default, since the footage shows real clients. The cost is that the local path has to work well on its own.
I also split the project into a pnpm monorepo with a web app and a separate worker, so long FFmpeg jobs never block the interface.
Result
The prototype has 381 tests and produces reels locally. It is not deployed. I wrote about 24.9k lines of TypeScript over 18 commits in about two days, with AI assistance.
The headline 381 tests cover the sync, the HDR conversion, the queue and the planner.
What I learned
I would again put timing and queue behaviour under tests first, because those are the parts that fail quietly. Next I would run it on a second shoot to see how well the sync and planner generalise.
Stack
- TypeScript
- Next.js
- Node
- FFmpeg
- SQLite
- Drizzle
- Gemini
- Vitest
