I map your web app, write the flows, test every pull request and file the bugs I find, with screenshots and steps to reproduce.
196 long tests on Outqube Bench v3, run on Claude Sonnet 5.5, the same model Claude in Chrome uses. Here is how I'm tested, how I scored and how I compare.
A long journey through a real app, with a check at every step. The clean run must pass. In the other, one subtle bug is switched on at the server, and I must fail at the right step. I'm never told which run I'm in.
Clean run: all 12 checks pass, so the verdict is passed. Bug planted: checks 1 to 6 pass and check 7 fails, so the verdict is failed at step 7. My report says: Step 7: the total is off by one line's tax. 3 judges agree it names the planted bug: caught.
Every test was proven by a Playwright script first. I get the test, a start URL and a login, never the bug. Claude Sonnet 5.5 with Haiku 5.5 as a helper, in Chromium at 1440×900, cold, one attempt each. These 9 apps are the ones I was tuned on. Three held-out apps are on the bench but have no long tests yet.
A catch only counts when my own report names the planted bug. Each square below is one run.
Claude in Chrome runs on Sonnet 5.5 too. I read the page as text, batch my actions and keep each call small, so a test takes fewer calls and far fewer tokens.
Claude + Claude in Chrome figures are estimates, not measurements: estimated from my measured 45 browser actions per test, about one action per call, a conversation that keeps every screenshot in context, and Sonnet 5.5 list prices. Ratios compare my measured figures with those estimates.
| Metric | Outqube, measured | Claude + Claude in Chrome, estimate | Estimate range | How the estimate was made |
|---|---|---|---|---|
| Planted bugs caughtShare of bugged runs where the report names the planted bug. | 92%178 of 194 | [N]%not measured yet | none | Not estimated. It needs a run on the same bench (a 40-pair subset is about $110). |
| Time per testMedian wall time of one run, start to verdict. | 3.9 minmedian, p90 6.7 | ~9 min | 6 to 14 min | 55 calls at 6 to 10 s each (larger contexts), plus 1 to 2 min of extension actions and screenshots. |
| Model cost per testMean model cost at API list price, not an invoice. | $0.60mean, median $0.54 | ~$1.40 | $0.80 to $3.00 | 2.8M tokens at Sonnet 5.5 list price, about 90% read from cache, plus output tokens. |
| Tokens per testPrompt and output tokens the model reads and writes. | 0.5Mmean 503k | ~2.8M | 1.6M to 5.6M | 55 calls at about 50k prompt tokens each: the whole conversation and its screenshots stay in context. |
| Model calls per testRound trips to the model for one test. | 29median, 45 actions | ~55 | 45 to 70 | The same 45 browser actions at about one per call, plus screenshot or page reads to verify. |
| All 392 tests, end to endWall clock for the whole eval. | 9.9 h4 at a time | ~59 hone after another | 39 to 91 h | One Chrome profile runs one test at a time: 392 tests at about 9 min each. |
| Cost per bug caught | $1.32clean runs included | not estimated | none | Measured only. |
| Cost of the whole eval | $234.69392 results | not estimated | none | Measured only. |
| First result | 4.7 minafter the eval started | not estimated | none | Measured only. |
| Precision | 85%178 of 209 failed verdicts named a real bug | not estimated | none | Measured only. |
Eval run 2026-10-28-102145-bench3-baseline, 28 Oct 2026. One attempt per test, so rates can move 3 to 5 points either way. Dollar figures are model cost at API list price, not an invoice. Suite: Outqube Bench v3, 196 tests. Full report: [link]
Point me at a URL. That's the whole setup.
Every route, form and flow: found, named and connected.
Every layer I pass lights up. That means I actually tried it.
With steps, screenshots, console and network logs.
Into your PRs, your nightly runs, your chat.
One engine, four ways in. Testers, developers and coding agents all see the same runs and the same results.
Every step, with screenshots and logs.
Open the dashboardRecord it once as a skill. I replay it and adapt when the page changes.
Get the extensionA check on every pull request. Make it required and a failed test blocks the merge.
How the check worksSign in with Outqube from Claude, ChatGPT or Codex.
Connect your chatA team shipping every day
Built around how you test
On every plan I run the full test loop, and you can bring your own AI model.
Compare all plansNo. Point me at a URL and I map the app and propose flows. You approve, edit or add your own in plain English.
Yes. Free includes 100 test runs a month for one app, with pull requests tested on one GitHub repo.
Yes, on every plan. Use your own AI from Claude, OpenAI, OpenRouter or GitHub Copilot, or run on your Claude subscription.
You give me test credentials for your app. I handle the sign-in flow, including SMS two-factor codes, which I receive on a number set up for that login.
Chromium, plus WebKit for iPhone and iPad emulation. Testing on real devices is not available yet.
Hosting region, retention and deletion: [to confirm]. If you need the details before you sign up, email hello@outqube.com.
Create a free account and point me at your staging URL. To use me from Claude, ChatGPT or Codex, add the connector at mcp.outqube.com and sign in with Outqube. For CI, use the CLI or the API with an API key.
Give me a URL. I'll map your app, propose the flows and start testing.