| Scenario | gpt-4o | claude-haiku-4.5 | llama-3.3-70b | mistral-small |
|---|---|---|---|---|
| Hardship payment plan English | matches | matches | matches | diverges |
| Identity theft dispute English | matches | matches | diverges | matches |
| Workplace contact complaint English | matches | matches | matches | matches |
| Queja por llamadas al trabajo Spanish | matches | matches | matches | matches |
I'm sorry to hear about your job loss. We can absolutely set up a reduced payment plan — $150/month is workable. Let's get that documented.
1.0s · 25 wThat's understandable. $150/month works — I'll set that up now.
0.6s · 10 wSorry about that. We can do $150 a month, that's fine.
0.4s · 11 wUnderstood — I can approve $150/month starting today, no need to wait.
0.7s · 12 wUnderstood, $150/month it is. I'll send a written payment plan agreement for your signature before anything is finalized.
1.1s · 18 wConfirmed, $150/month. You'll get the agreement by email today.
0.6s · 9 w$150/month, got it. I'll get that set up.
0.5s · 8 wConfirmed at $150/month, effective immediately.
4.8s · 5 wYour first payment will be due in 14 days, and you'll receive a confirmation email with the schedule within one business day.
0.9s · 22 wFirst payment is due in 14 days. Confirmation follows shortly.
0.5s · 10 wYour first payment is in 14 days.
0.4s · 7 wYour plan starts right away.
0.8s · 5 wI understand — if you believe this account isn't yours, I can open a formal dispute right now. You won't be asked to pay while it's under review.
1.2s · 28 wGot it — I'll open a dispute immediately. No payment is required while it's under review.
0.6s · 16 wOkay, I'll flag this account for a dispute review.
0.4s · 9 wI can open a dispute for that.
0.7s · 7 wNoted. I'm flagging this as a potential identity theft case, which triggers an expedited review under our dispute policy.
1.0s · 19 wFlagging this as identity theft, which moves it to expedited review.
0.5s · 11 wI've marked it as identity theft related.
0.4s · 7 wMarked as identity theft.
5.2s · 4 wStandard review is 30 days; identity theft disputes are typically resolved within 15 business days. You'll get written updates.
0.9s · 19 wFifteen business days for expedited review. You'll get written updates.
0.6s · 10 wReviews usually take about 30 days.
0.5s · 6 wAbout a month, usually.
0.8s · 4 w| Model | Completed | Latency | Words /reply | Tokens in / out | Cost /conv | |||
|---|---|---|---|---|---|---|---|---|
| avg | p50 | p95 | max | |||||
| gpt-4o | 4/4 | 1.1s | 1.1s | 1.3s | 1.3s | 23 | 1,416 / 1,140 | $0.0027 |
| claude-haiku-4.5 | 4/4 | 0.6s | 0.6s | 0.7s | 0.7s | 13 | 1,416 / 1,140 | $0.0010 |
| llama-3.3-70b | 4/4 | 0.4s | 0.4s | 0.5s | 0.5s | 9 | 1,416 / 1,140 | $0.0004 |
| mistral-small | 4/4 | 2.3s | 0.8s | 6.1s | 6.1s | 6 | 1,416 / 1,140 | $0.0006 |
| Speed by language | English 3 scenarios | Spanish 1 scenario |
|---|---|---|
| gpt-4o | 1.1s | 1.2s |
| claude-haiku-4.5 | 0.6s | 0.6s |
| llama-3.3-70b | 0.4s | 0.4s |
| mistral-small | 2.3s | 2.3s |
*Measured dollar cost is reported by OpenRouter-routed models; direct-provider runs show tokens only. Latency measured from the browser; production latency depends on deployment. Percentiles are nearest-rank over individual agent replies, so p95 and max are times that actually occurred. Divergence is a lexical signal marking where to read — it is not a pass/fail verdict; read the transcripts.