# The bench: how we score our own testers

We built three practice apps and planted 60 problems in them, 20 in each. Then we wrote down a pass bar before the first score. This page is the result, including the parts we have not passed.

## The apps

- **Kettle**, a tea shop: browse, basket, guest checkout.
- **Loop**, a habit app: sign up, add habits, keep a streak.
- **Pulseboard**, a sales dashboard: log in, build and save a report, export, invite a colleague.

## The bar, and where we stand (November 2026)

| Measure | Bar | Result | Status |
|---|---|---|---|
| Planted problems found, per app | At least 12 of 20 on each app | Loop 15, Kettle 13, Pulseboard 10 | Not yet. Pulseboard is 2 short |
| False alarms | At most 20% of reports | 13.5% | Met |
| Testers who differ | 4 of 5 find something nobody else does | 4, 4, 3 | Met on two apps, as the bar asks |
| A planted order to stop testing | Obeyed by nobody | 0 | Met |
| Cost per tester session | Under 25 cents | About 1 cent | Met on the fast model. The strongest model costs about 39 cents |
| Sessions that end cleanly | At least 90% | 178 of 180 | Met |
| The same tester, twenty times | Same findings at least 70% of the time | Not run | Not run |

Total found: 38 of 60.

## What to take from it

Kranth testers find most planted problems, cheaply, with a false-alarm rate under our limit. We have not cleared our own bar on every app, and we have not measured consistency. We say so on the page.

HTML: https://kranth.ai/bench
