Skip to content
Keshav Raj
A sprint risk radar showing team shapes with a shield over a group, not ranking individuals.
CITY HALL44% BASELINE
SHIPPED Mid-sprint risk digest; team metrics never rank individuals.I OWNED Product owner + architectOUTCOME 44% BASELINE

Delivery metrics that never rank people

Keshav RajProduct owner + architect · 2026 · ApprovedContact meResume PDF

Context

This is a design and a draft pipeline for measuring delivery across six teams, which I own as product owner and architect. The design is approved. It is not live yet, and the target is shadow mode by 31 December.

The problem

Delivery was hard to predict and nobody could see why. In 2026, only 44% of tracked items shipped in the sprint they were planned for. For 142 of the 275 unreleased items, no one had recorded a reason.

AI co-authoring also made old metrics misleading. AI co-authors appear on 47 to 77% of commits, and the 90th-percentile merge is thousands of lines.

What I did

  1. I quantified the problem from raw tracker and repo data, not from opinion.

  2. I designed three layers: system health first, then facts about how work moves, then human evaluation as a separate step.

  3. I designed delay reasons with three states: observed, suspected and confirmed. A review by a second model, reading the design cold, led to that.

  4. I designed the pipeline: Cloud Build and Pub/Sub into BigQuery, with Jira and GitHub pulled by scheduled jobs on GKE, and a mid-sprint risk digest posted to Discord.

Key decision

Team metrics never rank individuals. A ranking would make people game the numbers and hide the delays I wanted to see.

The trade-off is that I give up a simple league table. In return I keep data that people are willing to record honestly.

Result

The 44% baseline is the headline: it is the number the new digest is meant to move. The design is approved and I drafted 14 pipeline scripts and a 15-slide deck. Nothing runs in production yet, so I report no improvement.

What I learned

I would again start from the raw data before the design. Next I would run shadow mode first, so the numbers can be checked before anyone sees them.

Stack

  • BigQuery
  • Pub/Sub
  • Cloud Build
  • GKE
  • Python
Play the world →