First-hand notes from the team building ARC: where business rules hide, what breaks when AI agents plan and build at scale, and how to tell a real result from a marketing number.
We extracted every business rule from a VB.NET and Oracle system and gave each one an owner. Only 29% belonged in the new code. Here is where the rest went.
On one autonomous rebuild, every unit metric was green while the app was unusable. What that taught us about testing AI-built code, and what to measure instead.
Putting a whole legacy system's rules into one prompt produced a 2-million-token request and 92 blocked outputs. What fixed it, and what it means for large codebases.
Pressing retry on an AI planning pipeline grew the plan instead of fixing it. The cause was unstable identities, and the fix was a one-time clean-up.
AI reviewers check the code our agents write. What the research says about AI judges, the false failures we saw, and why tests still decide.
SWE-bench grades AI coding agents with hidden tests on small fixes. What a score tells you, why Verified was retired, and why SWE-bench Pro has the same problem.