Our development team spends almost 1/3 of our time doing Code Reviews. When we became a Payment Facilitator (original blog, current info), one of the requirements of PCI was formalizing a code review process. For every release we do (about 50-80 per week), we must have at least two additional people review the code before release. This has been enormously beneficial and has resulted in:
- 6 minutes of downtime total since 2015
- Clean, common code and standards that all of our developers can read and follow and reuse
- Learning – both people learning from each other, as well as multiple people knowing every line of code
- Satisfies our OCD itch of being picky and precise
New Code Review Agent
We have been working on a new code review agent that will become part of our overall development process. Of course developers are using the individual code review offered by Cursor and CoPilot and ClaudeCode, but we wanted something much better and to become a formalized part of code review and try to reduce that 1/3 of our time spent on code review.
Our goal was to encapsulate our standards and knowledge in the Code Review agent. Part of that is using the sets of documentation we have and standard things like our UX design system, part is using our Skills and Rules we have created for our AI coding agents, and part of that was the learn from our past code reviews. We set up a set of agents to review the past year of code reviews and evaluate them and create a wider knowledge base of rules (we actually weighted people like Stephen, our CTO, reviews more than more junior developers). See Step 4 below to see how we use those 1,800 rules. The result is to create an agent that is more wide ranging than any human can match – resulting in better and more thorough code reviews, and giving our developers more time to build more for our customers.
We released our initial Beta version about 2 weeks ago. We are using a multi-stage flow that involves:
- Non-AI based checks first
- Create a Plan
- Orchestrator fires up multiple specialist agents to execute the plan
- Over 1,800 Rules are applied in addition to other knowledge like our design documentation, Skills and Rules we have been building for our AI coding as well as third party standards for security like OWASP Top 10 and over 250 CWE’s.
- MARS process (invented by Google) is used as we compare multiple agent outputs to determine the right answer
- Integrated with our Github Pull Request system that provides tiered feedback to the developer with suggestions highlighted









Continuous Improvement
Of course we built Continuous Improvement into our Code Review Agent. As agent code reviews are passed back to our developers, the developers add comments and provide feedback that is ingested into our set of rules.

Costs and Benefits
Reviews typically cost $5-$15 each. There is a big payback:
- Save .25 – 4 hours of developer time. Assume developer time is roughly $100 per hour, that saves $25-$400 per code review.
- Better Code Reviews. Code reviews become fuller and more consistent because they are taking into account the full knowledge we have accumulated over years by a group of very skilled developers.
Future Directions
The next couple of months is being spent on improving the agent as well as improving the flow. On first release, the agent was too verbose, and the feedback was too scattered. That has been improved but still needs refinement. But there is still more we can do to improve developer efficiency.
We also need to see tangible proof that this is a trustworthy review process that will allow us to use the Code Review Agent as one of the required 2 code reviewers.
Several months down the line, we will begin to use another feature that is built into this framework to do the coding step in an automated fashion. The hope is that we can have an agent evaluate an issue, determine if it can solve it or it needs more input and guidance, and then write the code and then review the code before passing it onto a human with an explanation, the plan it used to write the code, and the code itself to make it into our release process.
We anticipate this is the critical unlock we will have on speed of development. While we have increased the number of releases by about 20% this year, we expect that to double by the end of next year to a rate of 4,000+ per year.
This means more features done for our customers. We anticipate that our developers will evolve more into architects and product managers. They need to increasingly have the “taste” to determine the best approach and priorities for our customers, and make sure the whole architectural integrity of our system remains pristine.
We live in an exciting time, where AI is enabling us to do far more than we have ever done before for our customers.
