What changed
FACT: Vercel shipped three separate capabilities in its changelog. (1) `vercel deploy --dry-run` inspects framework detection and the full file manifest a deployment would contain without creating one [vercel.com/changelog/dry-run-deployments-with-vercel-cli]. (2) `konsistent` deterministically enforces cross-file and folder-level structural conventions that TypeScript and ESLint do not model, plus a skill to generate the config [vercel.com/changelog/enforce-consistent-code-for-agents-and-humans-with-konsistent]. (3) Agent Runs are retrievable through the Vercel MCP and CLI, including subagent data, reasoning, tool calls, and token usage, without a separate observability vendor [vercel.com/changelog/agent-runs-vercel-mcp-cli]. HYPOTHESIS (not in sources): that these three were designed to compose into a self-improvement loop, or that anyone is currently using them that way.
Why now
FACT: the trace-retrieval capability is described in the source as removing the need for a separate observability vendor, and the dry-run explicitly gives a machine-readable pre-deployment check an agent can loop on. HYPOTHESIS: the marginal cost of a deterministic pass/fail oracle for agent-written code just dropped to roughly zero on this one platform, which is the missing ingredient for regression testing a non-deterministic system. Counter-consideration that weakens 'why now': none of the three source items mention regression testing, replay, or prompt evaluation. The 'now' is inferred from adjacency, not from stated demand.
Converging signals
The composition is genuinely non-obvious and is the strongest part of this idea. Regression testing anything requires three things: a recorded input, a deterministic oracle, and a way to re-run. Agent Runs supplies the recorded input including subagent reasoning. Dry-run manifests and konsistent configs supply oracles that are deterministic in a domain (structure, file layout, framework detection) where LLM output is notoriously unstable and where a unit test suite says nothing. Re-running is the trivial part. So the convergence is real as an engineering observation. What it is not is evidence that anyone will pay.
Customer pain
HYPOTHESIS throughout — no source here documents any user complaint. The presumed pain: a team edits an agent prompt or a skill file to fix a failure, has no way to know whether the edit fixed that failure class or merely reshuffled the sampling, and cannot tell whether the edit regressed three things it used to get right. This is a real and widely discussed problem in agent engineering. But 'widely discussed' is not the same as 'urgently budgeted', and I have zero source-backed evidence of the latter in this input. Treat the entire demand side of this brief as unproven.
Who pays
HYPOTHESIS: teams of 3-30 engineers running coding agents in CI who have already been burned by a prompt edit. That buyer is a team, not an individual, which means a team-plan sale, a security review, and a procurement conversation about handing over execution traces that contain source code and reasoning. That is the enterprise-sales failure mode arriving through the side door. The individual developer who would buy this on a credit card in ten minutes does not have enough failing traces for the product to say anything statistically meaningful.
Solved today
HYPOTHESIS (from general knowledge, not from the provided sources): LangSmith, Braintrust, Langfuse, Weights & Biases Weave, Arize Phoenix, and Vercel's own evals tooling all occupy this space. Most teams do it informally — eyeball a few runs, ship, hope. The 'nobody is doing this' framing is false. What is arguably underserved is the specific replay-against-historical-failures-with-a-deterministic-oracle slice.
Why current solutions are bad
HYPOTHESIS: existing eval platforms make you author test cases and graders by hand, and many lean on LLM-as-judge, which is itself non-deterministic and so cannot cleanly answer 'did this edit fix the failure class'. The pitch would be that the failing traces already are the test cases and the manifest/konsistent diff already is the grader. That is a sharp wedge. It is also a feature Vercel can ship as a checkbox, since they already own all three inputs.
Proposed product
Hosted harness: connect it to your Vercel Agent Runs, it ingests failing runs, clusters them into failure classes, and on every prompt or skill commit replays the affected traces and reports fixed / still-broken / newly-broken per class.
MVP version
Two weeks of build for a competent AI-assisted solo dev: MCP client pulling Agent Runs, a clusterer over failure signatures, a replay driver, a diff view against dry-run manifests and konsistent output. Speed to MVP is genuinely fast and is this idea's second-best attribute.
30-day build
Build the harness against your own agent runs. Publish the failure-class taxonomy you discover as a technical writeup. This costs nothing and produces a real artifact.
60-day build
Try to hand it to 5-10 teams. HYPOTHESIS: most will not connect it, because handing an outside service their agent execution traces — which contain proprietary source and reasoning — is a security review, not a signup. Expect this step to be where the idea dies.
90-day revenue plan
HYPOTHESIS, and I do not believe it: $500-2000 MRR from a handful of design partners. The realistic 90-day outcome is zero revenue and a good blog post. Dev-tools infrastructure sold to engineering teams has a long evaluation cycle, and the buyer has to trust you with their code before the product does anything at all.
Distribution path
Open-source the harness, write the taxonomy post, live in the agent-engineering discourse. This is a credibility-accretion play measured in quarters. That is precisely the multi-year trust-building motion the founder profile rules out.
Pricing hypothesis
$50-200/mo per team. Note the arithmetic: at $100/mo you need 20 paying teams for $2k MRR, and each one costs you a security conversation. There is no path here to fast cash.
Technical difficulty
Moderate and pleasant. Clustering failure traces into stable classes is the only genuinely hard part, and it is the part that determines whether the product is useful or is a random-diff generator. Do not underestimate it.
Legal / regulatory risk
Low in the abstract, real in practice — you are ingesting customers' source code and agent reasoning. That means a DPA, a security posture, and probably SOC 2 pressure the moment a second serious customer appears.
Platform dependency
Severe, and this is the decisive objection. All three enabling capabilities are Vercel's, shipped in Vercel's changelog, reachable through Vercel's MCP. The product is a thin composition of three primitives owned by one vendor who has every incentive to compose them himself, and whose changelog cadence suggests he will. Building a company in the seam between three features of one platform means racing that platform's roadmap with none of its distribution.
Founder fit
Poor, and the mismatch is not marginal. Charles's edge is industrial operations, recycling and scrap, public records, complaint mining, fire-service credibility, and compliance monitoring — domains where his operational history is the moat and where he sells by demonstrating value to an operator who feels the pain daily. Agent-infrastructure tooling for AI engineering teams gives him none of that. He would be a first-time entrant, with no domain credibility, no distribution, and no unfair information, competing with funded observability companies and the platform vendor. Sells-through-demonstrated-value works when the buyer can see the value in one demo; here the value is only visible after weeks of accumulated traces.
Breakout potential
Real if it works — a durable regression-testing layer for agents is a category. But breakout requires exactly the things the founder profile excludes: team sales, a trust cycle, and probably capital to outrun the platform.
Final recommendation
KILL as a business, KEEP as a technique. The convergence observation is legitimate and the composition is clever — a deterministic oracle plus a trace archive really does close a self-improvement loop that neither piece closes alone. But every commercial precondition fails at once: no source-backed demand, a team buyer requiring a code-access security review, total dependency on a single platform that owns all three inputs and can ship the composition natively, no founder edge, and no plausible 90-day dollar. The clever-composition feeling here is doing the work that evidence should be doing, and that is exactly the trap this screen exists to catch. Charles should build the loop for himself, because it will make his own agent work measurably better, and then go sell something in scrap, public records, or compliance monitoring where his twenty years of operational credibility is the moat rather than a non-sequitur.
Next action
Spend one weekend wiring dry-run manifests and konsistent as a deterministic oracle into your own agent loop. Keep the failure taxonomy. Do not build the hosted product. Revisit only if a stranger asks you to run it on their traces — that unsolicited pull is the single piece of evidence this idea currently lacks, and until it exists there is nothing here worth 90 days of runway.