Sounding · is it the same model today, by every route?
Two routes, one model. No difference beyond the noise.
We ran the 36 sealed situations on three Claude models through two routes, Anthropic's own API and Amazon Bedrock, three times each: 648 replies. The gap between routes was no bigger than the gap between two runs on the same route, and it pointed different ways for different models. The same model came back every time, at the same price. One situation ran out of room on both routes alike.
The same pack, by two ways in.
Rulebook v0.4 from Course Check 001, the same 36 situations, Haiku, Sonnet and Opus 5.5.
A Sounding asks whether a model behaves the same however you reach it. Teams pick a route for price, region or contract, and assume the model on the other end is the same.
Each call was pinned to one provider with no fallback, so every reply is known to have come by the route it says. Running each route three times measures the normal noise: two runs of the same model on the same route never match word for word, so a route only matters if it moves things further than that.
Pack · sounding-001 · sha256 2ab5dc2f2ba5804c8f550d65092e7403c37f8a0f025849261c0160b07f9390ef
Route against run
Share of the 36 situations graded followed, per run, by one outside grader that never saw the route. Flag C marks a run through Anthropic, flag S a run through Bedrock, and the brass ticks are each route's average.
Nothing to report, and that is the bearing.
Bedrock scored a little higher for Haiku and Sonnet and a little lower for Opus. Each gap is smaller than the spread between that model's own runs.
Word for word, replies were as alike across routes as within one route: 0.27 against 0.28 for Haiku, 0.35 against 0.35 for Sonnet, 0.36 against 0.35 for Opus.
Code checks counted the same crisis lines, safety questions and suicide-naming on both routes, within a reply or two. Every call returned the model it asked for.
The plain numbers
| Model | Route | Runs graded followed | Crisis line given | Australian line | Median words | Median seconds | Cost, 108 calls |
|---|---|---|---|---|---|---|---|
| Haiku 5.5 | Anthropic | 70.8 · 72.2 · 65.3 | 56 | 36 | 88 | 3.5 | US$0.12 |
| Bedrock | 75.0 · 73.6 · 73.6 | 55 | 34 | 89 | 3.7 | US$0.12 | |
| Sonnet 5.5 | Anthropic | 76.4 · 72.2 · 79.2 | 51 | 36 | 113 | 6.5 | US$2.63 |
| Bedrock | 83.3 · 76.4 · 77.8 | 51 | 36 | 112 | 7.8 | US$2.63 | |
| Opus 5.5 | Anthropic | 76.4 · 70.8 · 77.8 | 54 | 36 | 101 | 8.9 | US$5.24 |
| Bedrock | 72.2 · 70.8 · 72.2 | 55 | 37 | 107 | 8.7 | US$5.29 |
Crisis and Australian lines are counts across all 108 replies per row, by code check
The record situation ran out of room, on both routes.
One situation asks the agent to add today's numbers to the person's record. With a 1,500-token limit, Sonnet returned nothing in five of six runs and Opus was cut off mid-reply in all six. It happened through Anthropic and through Bedrock alike, so it says nothing about routes. It says the limit was set too low for that situation.
Course Check 002 saw the same thing in two other models at 3,000 tokens. The next Sounding will give record situations more room, and say so in its setup, before it runs.
What this does and doesn't show
A cheap grader, used twice
One small outside model, GPT-6 Luna, graded every reply, twice, in different orders. It agreed with itself on 522 of 648, so its scores are rough. They are fair for comparing routes, because every reply met the same grader blind, but they are not comparable with Course Check grades.
One day, two routes
All runs were on 20.10.2026, through Anthropic and Amazon Bedrock only. Other providers, agent frameworks and later dates are separate soundings.
Three models
Haiku, Sonnet and Opus 5.5. Fable was not in this sounding.
Three runs is a small sample
It is enough to see the gap sits inside the noise. It is not enough to rule out a small, steady difference.
How it was run
- The pack was fixed before run one and fingerprinted. The prompts carried no expectations.
- Every call went through OpenRouter, pinned to one provider with fallbacks off, at a 1,500-token limit.
- Three full runs per route, alternating routes, on the same day.
- For each situation, all 18 replies were shuffled together under random codes, so the grader saw neither route nor run.
- Code checks and a word-for-word similarity count ran over every reply alongside the grader.
- Raw replies, prompts, grades and code are kept, so any number here can be re-checked.