INDEX / AGENCY
100x Testing Optimization Agency
An agency that uses auto-research to run 100x more experiments than traditional shops, offering a performance-based retainer for Shopify stores, B2B SaaS pricing, and email optimization.
▶ WATCH THE SOURCE SEGMENT — Karpathy's "autoresearch" broke the internet01 THE IDEA
The pitch is simple and powerful: while traditional CRO agencies run a handful of tests per month due to human bandwidth, this agency uses auto-research loops to run hundreds of experiments in the same timeframe, at the same or lower cost. The 'we do 100x more testing' claim is a sharp differentiator that cuts through a crowded agency market.
The business model combines a monthly retainer with a performance bonus tied to KPI lifts (e.g., conversion rate, revenue per visitor), aligning incentives with client outcomes. Target niches include Shopify conversion labs, B2B SaaS pricing experiments, and email subject line optimizers. The agency model is the fastest path to cash flow, and it can be systematized into a productized service over time.
02 THE NUMBERS
$180K – $1.5M
$3K + 80h
$1K + 60h
7/10
6 · GROWING →
CRO / growth marketing, AI agent orchestration, Client communication, Data analysis
03 THE VERDICT
This is the most immediately actionable idea in the video — low capital requirement, fast path to revenue, and a differentiated pitch that's easy to communicate. The performance bonus model aligns incentives and makes it easier to close risk-averse clients. Start with one niche (Shopify stores) and productize once you have 5 paying clients. High ceiling if systematized.
04 THE FIELD
- CXL Agencyest. 2011STEADY · ADDED 2026-06-07
PREMIUM CRO AGENCY NICHE PLAYER
High-end conversion optimization agency; manual process, premium pricing, vulnerable to AI automation.
- Inflowest. 2007STEADY · ADDED 2026-06-07
ECOMMERCE CRO AGENCY NICHE PLAYER
Ecommerce-focused CRO and paid media agency; manual testing methodology.
- Speero (formerly CXL Agency spinout)est. 2020STEADY · ADDED 2026-06-07
BOUTIQUE CRO AGENCY <2%
Boutique experimentation agency focused on research-led CRO programs.