The State of API Testing

About this paper

Three markets in this series have been asked whether they run the thing they sell. API management did not. API documentation did not. SDK generation did not. API testing is the first one that does.

Four vendors in this cohort ship a working, publicly verifiable MCP server — Tricentis, SmartBear, Microcks and Playwright — against zero across the three markets scored before it. And Tricentis posts a composite of 62.8 against an agent-readiness score of 61.7, a gap of 1.1 points, where every previous market leader posted a chasm: Apigee 43.4, SwaggerHub 35.7, Redocly 49.1. It is the first provider measured in this series that is as ready for a machine caller as for a human one. Three more providers score higher on agent readiness than on the composite, which has not happened in any prior area cohort.

The most defensible reading of why: this market’s customer already drives APIs programmatically. A testing tool’s users are machines by construction — CI runners, schedulers, scripts — so an agent interface is a product extension rather than a positioning claim.

That is the good news. The rest is harder.

This is also the weakest-scoring area cohort measured. It averages 34.5, contains no Exemplar at all, and puts only 21.6% in the upper three bands against 19.5% catalog-wide — barely outperforming the catalog it serves, where API documentation put 55.2% in those bands and API management 85.7%. Governance is the weakest facet at 20.9, with 24 of 37 providers at zero — including both security scanners.

And four dimensions still return nothing across all 37. One of them is pointed: dry-run mode is 0 of 37, in the market that sells trying things without consequences. Alongside idempotency 0 of 37, consent and delegated identity 0 of 37, and scopes at 2 of 37 against tools that routinely hold production traffic captures.

There is one more structural finding, and it explains why testing tools never compose: this market has no shared model of its own product. Across the 17 providers publishing a contract, no resource is named by more than two of them. Every tool models what a test is differently.

The full report maps the six-tier structure, the facet and dimension data behind every number above, the standards layer where the honest answer is that no standard exists, and the investable thesis on why the interface is no longer the scarce thing in this market. It closes on a provenance audit of a score with two authors — and on the four record-level problems resolved before any figure was computed, including a slug collision that would otherwise have placed a defunct semiconductor manufacturer in an API testing report.

What's inside

1. Executive summary — the first market building for agents
2. The state of the market — six tiers that share a word
3. The resource taxonomy — a market with no common model
4. The composite Kin Score — no ceiling, and a governance problem
5. Agent readiness — where the pattern finally breaks
6. The blueprint — what the leaders satisfy, and what nobody does
7. Provenance — a score with two authors
8. The standards layer — a vocabulary without a specification
9. The regulatory layer — no mandate, and one regime moving toward it
10. Artifacts for the agentic turn — testing is where the agent arrives first
11. Security, authentication, scopes
12. Provider-by-provider intelligence
13. The investable thesis
14. Where this is going

What you get for $500.00