Convergence Radar Convergence Engine

← Feed

D

CrawlToll β€” hosted pay-per-crawl broker for Cloudflare publishers

40/100

A Cloudflare OAuth app that shows site owners which AI crawlers are hitting them, blocks agent/training bots by default, and sells them metered access via x402 stablecoin settlement β€” for a take-rate plus a flat monthly analytics fee.

Archive. Β· created 2026-07-10 00:08 UTC

saasapiagentaifast cashtoo complexrevisit later

Scorecard

newness 8/10
convergence 5/10
demand evidence 3/10
existing spend 4/10
solo feasibility 8/10
speed to mvp 8/10
speed to revenue 5/10
distribution 7/10
competitive gap 3/10
expansion 3/10
founder fit 7/10

Penalty flags
enterprise sales marketplace approval risk no urgent pain platform policy risk (βˆ’23 from raw 54)

Opportunity brief

What changed
FACT (per provided sources): Cloudflare shipped four things in close succession β€” (1) OAuth app registration open to all developers, so a third party can act on a user's Cloudflare resources with delegated auth [oauth-for-all]; (2) Attribution Business Insights, which exposes which AI crawlers hit a site, how much, and an estimated value of that traffic [attribution-business-insights]; (3) per-bot-purpose rules (search vs agent vs training) plus ad-page protection on ALL tiers, not just enterprise [content-independence-day-ai-options]; (4) a Monetization Gateway that prices any resource behind Cloudflare per-request, payable by autonomous agents via x402 stablecoin settlement [monetization-gateway]. HYPOTHESIS: taken together these are the ingredients of a paid-crawl market; no source states that such a market has clearing volume today.
Why now
FACT: the control plane (per-purpose bot rules), the measurement plane (Attribution Business Insights), the settlement plane (x402 Monetization Gateway) and the distribution plane (OAuth apps for all) all became available to non-enterprise customers at the same time β€” that is the actual novelty, and the sources explicitly frame the first three as previously restricted or previously not exposed. HYPOTHESIS: there is a short window (call it 6–18 months) before Cloudflare itself, or the first well-funded entrant, ships the packaged version of this. This window being real is unproven and is the single biggest risk in this brief.
Converging signals
Four platform signals from one vendor. That is a weakness, not a strength: this is not four independent markets converging, it is one company's product roadmap converging on itself. The genuine convergence is functional β€” measurement + control + machine-native payment + third-party install β€” but the correlation of the signals means the 'convergence' score should be honest about single-vendor origin.
Customer pain
HYPOTHESIS (not established by the provided sources): publishers feel that AI crawlers extract content value without sending traffic back. The sources imply Cloudflare believes this pain exists β€” they built Attribution Business Insights specifically to give publishers 'concrete per-crawler data to negotiate crawl compensation' and framed the bot-rules launch as 'content independence' β€” which is strong corroborating evidence of demand, but it is Cloudflare's assessment of the market, not measured buyer behaviour. The pain is real for large publishers with a licensing team. For the long tail of sites, the honest read is that the pain is felt but the dollar amounts are small and the pain is therefore not urgent.
Who pays
Buyer candidate A: mid-size content sites (recipe blogs, niche news, docs sites, forums, review sites) with 100k–5M monthly pageviews, already paying for SEO/analytics tooling, already on Cloudflare. Buyer candidate B: the AI crawler operators, who would pay per-request. Note carefully: the take-rate revenue comes from B, but B is a small set of well-capitalized firms (OpenAI, Anthropic, Google, Perplexity, Meta, ByteDance, plus scrapers). Selling to B is enterprise sales in disguise. The only escapable, non-enterprise revenue in this design is the flat monthly analytics fee from A. That is the business. The take-rate is a lottery ticket layered on top.
Solved today
FACT (from sources): today a Cloudflare customer can already see per-crawler attribution [attribution-business-insights], already set per-purpose bot rules on any tier [content-independence-day-ai-options], and already price a resource per-request via the Monetization Gateway [monetization-gateway]. HYPOTHESIS: publishers otherwise use robots.txt (unenforceable), blanket WAF blocks (blunt, kills search referral), or a manual licensing deal (only viable at scale).
Why current solutions are bad
This is where the idea starts to die. The three native Cloudflare features are not 'bad' β€” they are free or bundled, first-party, and already in the customer's dashboard. The proposed product's entire value-add is (a) stitching three dashboards into one, (b) recommending which rule to set, and (c) a take-rate on money that flows through Cloudflare's own settlement rail. Cloudflare owns the rail, the data, and the customer relationship. A thin broker sitting on top of a vendor's stack, monetizing a spread on that vendor's payment rail, is the textbook definition of a business the platform absorbs. HYPOTHESIS but a confident one.
Proposed product
If built at all, do NOT build the broker. Build the narrow, defensible slice: a Cloudflare OAuth app that reads Attribution Business Insights across a publisher's zones and produces (1) a weekly 'AI crawler leakage report' β€” who crawled you, what it's worth, what changed, benchmarked against anonymized peers in your category, and (2) one-click recommended per-purpose rule sets ('allow search, block training, meter agents') applied through the API. Charge a flat $19–49/mo. Peer benchmarking is the only asset Cloudflare does not trivially have and the only thing that compounds β€” no single publisher can see the cross-site crawl comparison. Add x402 metering later, as a feature, not as the business model.
MVP version
OAuth app against Cloudflare (per [oauth-for-all]) requesting read on analytics/bot-management and write on bot rules. Pull Attribution Business Insights per zone on a schedule. Store per-crawler counts + Cloudflare's own value estimate in Postgres. Email a weekly diff report. Ship a 'recommended rules' page that writes per-purpose rules through the API. No payments, no x402, no take-rate, no dashboard beyond a single page. 2–3 weeks of solo AI-assisted work. Hard dependency: whether the Attribution Business Insights data is actually exposed via API to an OAuth app, and on which plan tiers. This is UNVERIFIED and is a go/no-go gate β€” check it in the first 48 hours before writing any other code.
30-day build
Days 1–2: read the Cloudflare API docs and confirm (a) Attribution Business Insights is API-readable, (b) an OAuth app can request that scope, (c) which plan tiers expose it. If any of the three is no, kill the project β€” there is no product. Days 3–7: interview 15 site owners in Cloudflare/webmaster/publisher communities. Ask what they currently do about AI crawlers and whether they have ever paid for a tool to manage it. If fewer than 3 say they'd pay $29/mo for a leakage report, kill it. Days 8–21: build the MVP above. Days 22–30: onboard 10 free design partners, generate real reports, learn whether the numbers are interesting enough to be worth money.
60-day build
Ship peer benchmarking once ~50 zones are connected (this is the moat and it needs density; below ~50 zones in a category the benchmark is noise and should not be shown). Convert design partners to paid at $29/mo. Publish the aggregate crawl data as a free public report β€” 'which AI companies crawl the web hardest, by category' β€” as the top-of-funnel content play. Target: 20 paying customers, ~$600 MRR.
90-day revenue plan
Realistic honest target: 40–75 paying customers at $29/mo = $1,200–2,200 MRR. That is real but it is not the outcome the original convergence description implies. HYPOTHESIS: the take-rate business generates approximately $0 in 90 days, because it requires AI crawler operators to voluntarily start paying per-request through your broker rather than through Cloudflare directly, and there is no evidence in any provided source that any crawler operator has agreed to pay anyone.
Distribution path
Content-led, no ad spend, no sales calls. The public aggregate crawl report is the wedge β€” it is exactly the kind of data journalism that gets picked up by HN, r/webdev, r/SEO, publisher newsletters, and Cloudflare's own community forum. Secondary: the Cloudflare app ecosystem listing itself, which per [oauth-for-all] is now open to any developer β€” this is a real, free, high-intent distribution channel and it is the single best thing about this idea. Founder fit is strong here: this is a data/report product with public-records-style aggregation, sold on demonstrated value.
Pricing hypothesis
$29/mo flat for the leakage report + rule automation. $79/mo for multi-zone/agency. Do not charge a take-rate at launch β€” it introduces payments, KYC-adjacent questions, stablecoin custody exposure, and a revenue model whose counterparty has never agreed to transact. Revisit metering in month 6 if and only if x402 crawl payments show observable volume.
Technical difficulty
Low-to-moderate. OAuth flow, scheduled API polling, Postgres, a diff-and-email pipeline, one write endpoint. No ML, no scale problem, no infrastructure. The hard part is not technical; it is that the technical part being easy is precisely why the product is easy to copy.
Legal / regulatory risk
Low for the report product. Meaningfully higher for the broker version: taking a percentage spread on stablecoin payments between two parties makes you a money transmitter in the eyes of some US regulators, or at minimum forces a serious legal opinion. This alone should push the founder to the report product and away from the take-rate. Also: Cloudflare's OAuth app terms will govern what you may do with customer analytics data β€” the peer-benchmarking feature depends on aggregating across customers, and that must be explicitly permitted in the app terms and in your own ToS. Verify before building the moat on it.
Platform dependency
Total. This product cannot exist without Cloudflare. It reads Cloudflare data, writes Cloudflare rules, installs through Cloudflare OAuth, and (in the original design) settles on a Cloudflare payment rail. Cloudflare can end the business with a changelog entry β€” either by shipping the feature natively (they have already shipped three quarters of it) or by revoking the OAuth scope. Treat this as a 12–24 month cash-generating asset, not a company. That framing is fine for a founder who needs revenue in 30–90 days; it is fatal for anyone building to hold.
Founder fit
Genuinely strong on the report product. This is complaint-mining and public-records-style aggregation applied to crawler logs: pull a data source nobody has bothered to normalize, benchmark it, sell the interpretation. It is a data/report micro-SaaS with no sales calls, no ad spend, and a self-serve install β€” sold on demonstrated value, which is exactly how this founder sells. Weak on the broker version, which requires convincing OpenAI and Google to route payments through a solo operator's spread.
Breakout potential
Low as a company, moderate as a cash asset. The ceiling on 'weekly crawler report for mid-size publishers' is maybe $10–30k MRR before it saturates or gets absorbed. The peer-benchmark dataset is the only thing that could become interesting β€” an authoritative independent measurement of which AI companies crawl what, which is a media/data asset with a life beyond the tool. HYPOTHESIS.
Final recommendation
CONDITIONAL BUILD β€” but build a materially different, smaller product than the one proposed. Kill the pay-per-crawl broker: the take-rate depends on a counterparty (AI crawler operators) that has never agreed to pay, requires enterprise supply-side origination a solo founder cannot do, and monetizes a spread on the platform's own rail, which is the most-absorbed business model in existence. Also, taking a percentage of stablecoin flows between two parties invites money-transmitter questions that are not worth answering for pre-revenue. What survives the kill attempt is the narrow slice: a $29/mo Cloudflare OAuth app that reads Attribution Business Insights, emails a weekly AI-crawler leakage report with cross-site peer benchmarking, and one-click applies per-purpose bot rules. That version has an existing reachable buyer, a two-to-three-week MVP, a zero-cost distribution channel in the newly-opened Cloudflare app directory, and a defensible-ish asset in the aggregated benchmark. Expect $1,200–2,200 MRR at day 90, a $10–30k MRR ceiling, and a 12–24 month life before Cloudflare ships it natively. That is a good trade for a founder who needs cash in 30–90 days and a bad trade for anyone else. Everything above is gated on one unverified fact: that Attribution Business Insights is readable through the API by a third-party OAuth app. Confirm that in 48 hours or walk away.
Next action
Within 48 hours: read the Cloudflare API and OAuth scope documentation and answer three questions in writing β€” (1) is Attribution Business Insights data exposed via API, (2) can a third-party OAuth app request that scope, (3) on which plan tiers. If all three are yes, immediately post in r/SEO, r/webdev, and the Cloudflare community forum asking site owners what they currently do about AI crawlers, and get 15 conversations. If any of the three is no, kill and move on.

Kill arguments (adversarial)

Competitors

β€’ Cloudflare (first-party) (link) β€” The dominant competitive threat. Already ships attribution, per-purpose bot rules on all tiers, and x402 settlement. Anything built here is a feature they can fold into the dashboard.
β€’ Cloudflare Monetization Gateway (link) β€” Directly competes with the broker concept β€” it already prices arbitrary resources per-request via x402, with no third-party broker needed.
β€’ Cloudflare per-purpose AI traffic rules (link) β€” Free/bundled on all tiers. Removes the 'granular bot control' half of the proposed product's value prop before launch.
β€’ Content licensing marketplaces (e.g. TollBit, ScalePost, Human Native) β€” HYPOTHESIS β€” not referenced in the provided sources and therefore unverified. If they exist as described, they are funded, already have crawler-side relationships, and occupy exactly the broker position. Verify before building.

Source citations (facts)

β€’ Unlocking the Cloudflare app ecosystem with OAuth for all β€” FACT: Developers can register and self-manage OAuth apps against Cloudflare accounts, so a third-party app can act on a user's Cloudflare resources with delegated auth. This is what makes a self-installing publisher app possible and is the distribution channel for the recommended product.
β€’ Unmasking the crawls with Attribution Business Insights β€” FACT: Site owners can now see which AI crawlers hit them, how much, and an estimate of that traffic's value β€” data previously not exposed. This is the data source the recommended leakage-report product reads, and its API availability is the project's go/no-go gate.
β€’ Your site, your rules: new AI traffic options for all customers β€” FACT: Per-bot-purpose rules (search vs agent vs training) are available to all Cloudflare tiers, not just enterprise. This both enables the one-click rules feature and undermines the product, since the control layer is already free.
β€’ Announcing the Monetization Gateway: charge for any resource behind Cloudflare via x402 β€” FACT: Arbitrary resources behind Cloudflare can be priced per-request and paid by autonomous agents via x402 stablecoin settlement. HYPOTHESIS: no source shows any AI crawler operator actually paying through this rail, which is why the take-rate model is unfunded speculation.

Actions