Building AI-agentic code review into a Fortune 500 Europe data platform
How a Python framework running GPT-5-mini via Copilot CLI reviews production code at minimal per-review cost — citing the exact rule behind every comment, letting developers challenge its findings, and never repeating itself.
The situation
At the scale this Fortune 500 Europe data platform reached — well over 4,000 dbt models across Snowflake, Databricks, and Azure — manual code review had become the bottleneck. Reviewer bandwidth couldn’t keep up with the rate of change, and quality gates were the first casualty whenever timelines got tight.
What changed
- Built an AI-agentic PR review framework — running as one of two mandatory reviewers on every change alongside a human — powered by GPT-5-mini via Copilot CLI, chosen specifically because it’s the cheapest capable model available, keeping per-review cost minimal at high pull-request volume.
- Built a Python framework underneath the agent that does the actual work: enforcing rules, managing context, and handling review logic, not just a thin prompt wrapper around the model.
- Gave developers a way to challenge an AI review comment directly — the agent re-checks the pushback against the same rule it originally cited, and confirms whether the work is actually complete before standing down or holding its ground.
- Added a token-budget maximizer, using Python’s tokenizer package, so every review packs as much relevant rule and context into the model’s window as the token budget allows, without waste.
- Built comment-deduplication logic so the same finding doesn’t get repeated across review passes, spamming developers with feedback they’ve already seen.
- Every AI comment cites the specific rule it came from, out of a maintained knowledge base — keeping the system auditable instead of a black box.
- Architected a multi-agent framework backed by that same knowledge base, letting agents pick up, scope, and deliver backlog items autonomously.
Why it matters
Most ‘AI for data platforms’ conversations stop at a chatbot bolted onto a warehouse. The more durable pattern is agents that participate in the actual engineering workflow — citing the specific rule behind every comment, taking developer pushback seriously, and doing it cheaply enough to run on every single pull request — without lowering the bar a human reviewer would hold.
Client details anonymized under contractual confidentiality. Figures and scope are accurate to the engagement.