A nonpartisan voter guide for a California city of about 96,000 registered voters. It reports where campaign money comes from. It matches each donor to a record in the county's list of registered voters, so a reader can see who is backing each candidate. It covered the June 2 2026 election and the three-week count that followed, and every number on it is footnoted to a public filing.
I moved to Torrance and wanted to know who was running for city council. That turned out to be hard, and partly by design. California city races are nonpartisan by law, so nobody runs as a Democrat or a Republican and the ballot says nothing about party. The politics don't stop. They just stop being written down.
What is left is scattered across county voter files behind paywalls, donor spreadsheets and council minutes buried in PDFs, and the local paper that used to read all of that for you has thinned to almost nothing. Mailers fill the gap. I got them: a party's colors, a party leader's photo beside a candidate's, an endorsement implied and never stated.
"City coverage has thinned, the public records are scattered, and mailers are built to persuade. Torrance Watch is the page in the middle." — torrancewatch.org/about
So the site had to be useful to a reader who does not trust it. Every claim on it should be checkable somewhere else. Rather than labelling a candidate as leaning one way, it publishes five separate party-related facts about each one, every fact traced to a public document. The fifth of those is the only one the site computes itself: the party mix of that candidate's donors, published as a group total and never as a list of people.
Three editorial rules are enforced by the software rather than remembered by me.
The donor party mix could point at one person if the group were tiny, so two rules apply. A candidate's mix is published only once enough donors have been matched that no single person can be picked out of the total, and the smallest party groups are reported together instead of separately.
LA County counts for three weeks after election day and no wire service covers city races, so explaining the count became the most-read work the site did. Explaining it was only possible because of one estimate. "Leads by 400" means nothing until you know whether 500 or 15,000 ballots are still to be counted, so the site had to be able to say how many were left.
The estimator starts from the county's own published turnout projection, subtracts the ballots already counted, and publishes a range for how many are left. It never publishes a vote count, a margin, or a winner, and when the inputs are missing the estimate does not appear on the page at all. A forecast that cannot say "I don't know" is not a forecast.
Declaring a winner followed the same discipline, and it worked differently in two places. The results page never declared one. Publishing a call there takes a deliberate editorial step, and the name gets checked against the official list of people in that race, so a typo stops the call from going out instead of putting a wrong name on the page.
The newsletter did call it, and said what the call rested on. On June 5, Chen was ahead by 359, and the last three batches of results the county had released were moving toward Kalani. The newsletter said Kalani was favored to overtake him, and labeled that a read of public data rather than an official count.
On June 7 it reported that she had passed him, and said the three council races were settled. It kept the treasurer's race open as the only one still in doubt, with the leader ahead by 286 and mailed ballots still to count. That race was certified on June 26, decided by 309 votes. A read you are willing to publish early is worth more than a verdict you publish once it is safe, provided you show what it rests on.
On election night the county results feed broke. The page fell back to showing zeros, which looked exactly like the normal page before the polls close, and it stayed that way for hours. A failure that looks exactly like health needs its own detector.
This page is the build record, not the site. Three things were hard. Four government systems describe the same people and none of them spell the names the same way. Thousands of judgment calls had to be handed to an AI language model that was never allowed to make the final call. And accuracy had to be measured against an answer key nobody had ever written down, which then turned out to be half the system's own output.
Astro 7 static · Cloudflare Workers · Postgres on the laptop, never at the edge · Node pipeline scripts · Claude Code as build partner and in-session reviewer
498 merged PRs and 653 commits, Apr 1 to Aug 19, one person. How many donors the system could match was settled in a day, on May 7. Everything after that was about knowing whether those matches were right, and the two biggest changes that came out of it both made the site publish less. First make it exist, then make it deserve belief. In August that discipline got its final test: the scoring settings themselves were put on trial against the evidence, and kept their job. That story closes the accuracy section.
Four government systems describe the same people without agreeing on spelling. "Jim Smith, donated $200" and "James Smith, registered on Maple Ave": one human or two? That question has to be answered defensibly for 2,543 donors against 6,411,869 voter registrations, and every confident answer has to show which facts it agreed on. The software that does this is called the matcher throughout this page.
Version 1 ran a list of rules in order and took the first hit. It was scrapped within a day, for three reasons that turned out to be the whole problem:
Version 2 scored the evidence, required a clear lead over the runner-up, and sent close calls to a queue for review. Doubt became a thing the system could express and route to a human. Every version after that bought better measurement rather than more matches.
Facts that agree earn points, weighted by how much each one narrows the field, because a shared zip code means little in a city of 96,000 and a shared uncommon first name means a great deal. Treating every agreement as an equal yes-or-no vote produces confident nonsense. The last name earns nothing at all: it is only how a record gets onto the shortlist in the first place, not evidence that the match is right.
The requirement it took three months to arrive at: a link needs a high enough total score, and a clear lead over the second-best possible match, and at least one fact beyond the name to agree. The lead test alone only asks whether anyone else came close, and for an unusual name nobody does, so a single candidate could clear it on the name alone. Being the only Emily Wang in the county is not evidence that you are the right Emily Wang.
Adding that third requirement moved the links the matcher makes on its own from 13 wrong in 134 to none wrong in 79, at a cost of about 100 links, which now go to a person instead. A name by itself is never enough to publish a link.
Where the 2,543 donors stand today:Some cannot match at all: 305 are organizations, and a few hundred live outside LA County, which is the only voter file this matches against.
Coverage, measured against the donors who could match: an organization has no voter registration, so counting companies in the total guarantees they show up as failures. Against the 2,238 individual donors, 1,569 of 2,238 are linked, and 1,182 of those were linked by the software alone, with no human reviewer.
The second line on the accuracy chart divides by the whole pool instead, organizations included, which is why it sits lower. That line is tracking one specific thing over time, how much the matcher settled on its own, and it is not a coverage figure.
Where the model is, and isn't:Across 3,283 decided queue rows:
A standing rule decides more than the model does. The rules handle the kinds of case that can be written down in advance. The model handles what is left over, working through the queue 50 rows at a time, with each row arriving as its own short brief of the evidence. It proposes and it explains, and it can never approve. The model is rented. The rules around it are the thing I actually built.
Where the panel fits: a panel here means several model runs on the same question that have to reach the same answer before it counts, which turns one opinion into a small vote. Panels appear at two points and they are doing different jobs.
Inside this queue, a panel settled 25 donor-match rows that the ordinary single runs had left undecided, and those sit within the 1,014 above. It is the escalation step on the review diagram: the row a single pass could not settle goes to several runs that have to agree.
Separately, and more importantly, a panel ran the blind re-check in the accuracy section, marking 200 already-published links without being shown what the matcher had picked. That second one is not a decision path at all. It is an audit, and it is described under how accurate is it.
That split was backwards on this page until Aug 17, and it is worth walking through, because it is exactly the failure this page is about: a number that looks fine from the inside and is wrong. The page credited the model with about 1,700 decisions and standing rules with about 460, roughly inverted.
Every decision in the queue is stamped with a marker naming what decided it, and the largest marker by far is claude-heuristic-donor-voter-match, at 1,149 rows. So any count that just looks for the word "claude" in the marker counts all 1,149 of those rows as the model's. The name says heuristic, which means a fixed rule of thumb, and a rule of thumb is a rule.
The split now comes from the same sorting code the accuracy report uses. That code decides what each marker means by reading the documentation of the code that writes it, rather than by the marker's name. And an automatic check compares the result against this page every time the site is built, so the two cannot drift apart again. The lesson generalizes past this project: a label that names a tool does not tell you what did the work.
The site is built from files packed with personal data, and a conventional site with a live database puts all of it one breach away from the internet. So all the data sits in one database on one laptop. The public site is a set of plain files with no database connection at all. Those files are written out on the laptop before anything is uploaded, and only from tables that have had the personal details stripped out. Once the site is live, there is no database behind it to break into.
How it got there, in order. The split was in the design from day one. Then came written rules about what may ever be published. Then reports that listed individual people were kept out of the code repository. Then database permissions were taken back in the places where new tables had been handing them out automatically. Last came an audit, which found a route into voter data that nothing was using but nothing was blocking either, closed it, and then proved the block worked by trying the same thing again on purpose.
An answer key is a set of cases where somebody already knows the right answer, so a system's output can be marked against it. Nobody has ever recorded which Torrance donors are which of 6.4 million registered voters, and no such key exists to buy.
So for the first three months the system was graded against its own earlier output. That is a genuinely useful test, and it catches a system drifting away from what it used to do. What it cannot do is tell you whether what it used to do was right, because a system compared to itself can only ever agree.
Building a real key meant deciding cases by hand, cheapest method first:
The key has grown as reviews come in, and it now holds 464 donors, every one decided by a person.
None of those labels came from the matcher's own output, which is the entire point. The old key held 159 rows the matcher itself had produced, filed as truth, and those were half of its 313 rows. They scored 100% in every run they ever appeared in, which is what it looks like when a test only ever asks a system whether it agrees with itself.
Removing them dropped the measured accuracy on the spot, and the drop tripped the automatic check that blocks the site from publishing when accuracy falls. That was the correct outcome, and the first number the project had that meant anything.
Every version of the scoring code is still in the project's history, so each one can be re-run against the new key and marked the same way. That is why the first chart on this page can show accuracy for May, months before anything was measuring accuracy at all.
Where it stands now:Unattended links are the ones the matcher publishes on its own, with nobody looking; they are the ones that need proving. Those came back 104 of 104 right against the answer key. Read that as no errors found in 104 checks rather than no errors anywhere, because a sample that size can only narrow the true rate to a range, and that range runs between 96 and 100 out of 100.
Most of those labels were written with the matcher's pick on screen, which measures agreement rather than tests it. So a second panel re-checked 200 published links without being shown what the matcher had chosen, and found 195 of 195 correct as of Aug 16. That is the stronger of the two numbers, and it is the reason the blind version was run.
Two more angles on the same question. The matcher used to publish a second, lower-confidence class of link on its own as well. That class was retired in August, and it scored 70 of 74 on the same key.
And every number above only grades the links the matcher actually made. None of them can see the person it never pulled up for consideration in the first place. That gap gets measured too: 394 of 401 is how often the right voter registration made it onto the shortlist of possible matches the matcher chooses from. The seven misses: two compound first names, one first-name variant, three surname spellings, and one where the verified answer itself looks wrong, which is its own kind of finding.
Checks that read what the matcher cannot: a second pass hunts for links that cannot be true, using only facts the scoring code never looks at.
The principle is simple and worth stealing: a check is only evidence if it reads something the thing it grades never saw. Contradicted addresses among unattended links stand at 34 in 1,182, down from 147 in 1,310 before a second agreeing fact was required. Two separate tests pointing the same way, and neither one was told what the other found.
The trap, and what it actually measures: take a link whose answer is known, delete the correct person from the list of possible matches, and make the matcher choose again. Anything it still commits to has to be wrong, because the right answer is no longer on the table. That is the appeal: it needs no labels at all, so it works long before an answer key exists.
It now does that 12 of 1,182 times, down from roughly one in nine. Read it precisely, because it answers a narrower question than it appears to: how often the matcher commits when the right person is missing entirely. It is not the share of published links that are wrong, and the two get confused easily enough that this section exists mostly to keep them apart.
The settings stood trial, and kept their job (Aug 19): the matcher's settings, meaning how many points each agreeing fact earns and the bar a total has to clear, were set by judgment in May and never checked against real evidence, because until August there was no honest evidence to check them against. With a real answer key in hand, they could finally be tested. The rules of the contest were written down and saved before anything ran: what would count as better, how ties would break, and which verified answers would stay sealed. The sealed ones were kept out of the search entirely, to be used once at the end.
An automated search then tried 28,532 alternative settings. Thousands beat the current ones on the cases they could study; the best raised correct automatic matches on the key from 100 to 147 without a single error there. Then the sealed answers were opened. Each of the two winning settings made one confident wrong match on them. Each one picked out a voter registration for a donor that a human reviewer had already decided was not on the voter file at all. The current settings made none.
So the current settings stayed, which was the pre-written rule. The winners had gained ground by tuning right up against the hardest mistake anyone had already found, and one sealed case sitting just past that line was enough to tip them into a wrong answer. A sealed test that can veto your winner is what makes it tuning instead of wishful thinking.
Then the grader itself got checked. An independent code review found that it was counting 33 rows nobody had ever verified as if they were confirmed answers. After the fix, everything was re-graded: same verdicts, on clean evidence. The settings the site runs on today are the ones it ran on in May, but "unchanged" now means measured and confirmed rather than never examined.
Every figure here is generated from the database and the accuracy report by scripts/build-record/derive-figures.mjs, and a build fails if this page and that file disagree.
| Number | What it is |
|---|---|
| 498 / 653 | merged PRs / commits, Apr 1 to Aug 19, one person |
| 6,411,869 / 54.5M | people in the voter file / rows of participation history, all of LA County |
| 2,543 | donors matched against it: 1,182 linked by the matcher alone · 177 waiting on review · 797 no match found · 387 settled by review, by hand, or by copying the answer from another record that turned out to be the same donor |
| 6,209 of 6,210 | reported contributions that are tied to a single cleaned-up donor record, after different spellings of the same name were merged into one |
| 464 | donors in the answer key, every one adjudicated by a person and none of them the matcher's own output |
| 104 of 104 | links the matcher made on its own, checked correct against that key. Read as "no errors found", not "no errors exist": a sample this size narrows the true rate to between 96 and 100 out of 100, and the key leans toward hard cases on purpose |
| 195 of 195 | a blind re-check of 200 already-published links, the reviewer never shown which registration the matcher had picked. The strongest accuracy figure on this page, as of Aug 16 |
| 394 of 401 | how often the right person even reaches the shortlist the matcher chooses from, which no figure above can see. The seven misses: two compound first names, a first-name variant, three surname spellings, and one verified answer that itself looks wrong |
| 70 of 74 | the lower-confidence class of link, which stopped publishing automatically in August. A second sample, drawn to represent all links rather than the easy ones, got it wrong 3 times in 13, and that is the number that retired it |
| 34 in 1,182 | links the matcher made on its own whose address contradicts every address the donor filed, down from 147 in 1,310 before a second agreeing fact was required |
| 12 of 1,182 | how often the matcher still commits to an answer after the correct person is deleted from its list of possible matches. Not the share of published links that are wrong |
| 28,532 | alternative settings tried in the Aug 19 trial. The best raised correct automatic matches on the key from 100 to 147, then made one wrong match on the sealed answers the search never saw. The shipped settings made none there, and stayed |
| ~26,000 | estimated human visitors, lifetime (range 23.6-26.9K, as of Aug 5). The raw 33,058 figure counts internet addresses rather than people, automated crawlers included, and was retired as the headline |
| 2,522 / ~227K | internet addresses seen on the site's biggest day, June 8, mid-count / social views across 29 posts, both as of Aug 5 |
| 309 / 96,606 | the margin, in votes, that decided the treasurer's race (44.8% turnout) / registered voters, the denominator behind every turnout figure |
Check what your answer key is made of before you trust it. For three months this system was graded against an answer key that was half its own output. It scored 100% every time. Of course it did: it was agreeing with itself. And nothing about that looks broken from the inside. The tests pass, the number is high, everyone is happy.
So before you trust a score, ask where the answers behind it came from. If your system produced any of them, throw those out, and add a check in code that keeps them out. This mistake sneaks back in dressed as good news, and memory is not a defense against it.
If you have no answer key at all yet, there is a cheap stopgap: delete the true answer, run the system anyway, and count how often it still says it found something. Every one of those is a mistake you can count without knowing any right answers.
If a model does judgment work for you, put it inside a contract. The model here reads one case file at a time and proposes an answer. It cannot approve anything. Its allowed answers are spelled out in advance, and every decision is stamped with who or what made it.
That stamp earned its keep: months in, the stamps showed a plain rule was doing most of the work everyone had credited to the model.
Then measure the parts you assumed were working. I asked the model to rate its own confidence on every row. When I finally checked those ratings against real outcomes, "medium" did worse than "low". The rating meant nothing, so I deleted it.
Write the contest rules before the contest, and keep a sealed test. When the matcher's settings finally got tuned, three things were written down before anything ran: what "better" would mean, how ties would break, and which verified answers would stay locked away from the search, to be used once at the end.
The search tried 28,532 alternative settings and found ones that looked clearly better. Then the sealed answers came out. Both winners got one wrong. The old settings got none wrong. So the old settings stayed. That was the rule, written in advance, and it is the sealed test doing its one job: telling you your improvement is a mirage before your readers find out.
One more habit worth stealing: have someone check the grader too. An outside review found mine counting unverified guesses as verified answers. The verdicts held up after the fix, but I only know that because somebody looked.
One person and Claude Code on a nine-week deadline. That was possible not because rigor was skipped but because rigor got cheap. You do not rebuild trust by asking for it. You rebuild it with systems willing to show their sources and publish their own error rate.
↓ Seven diagrams, starting with accuracy over time, then the master flow, four zooms and the reach chart.
The headline chart · every version of the scoring code, re-run against one answer key
Green is checked against the 464 donors a person resolved by hand. The 100% means no errors were found in 104 checks: a sample that size can only narrow the true rate to between 96 and 100 out of 100. The blind re-check described under “how accurate is it” is the stronger of the two tests, and it is the one to read.
The map of the whole system. Every diagram after it zooms into one stage.
Stage 3, told through time. Same box on the master flow, seven rewrites of it.
Stage 4, where a close call goes to be decided and by what.
Stage 5, one eval of several. Accuracy itself is the first chart on this page.
The seam between 6 and 7, and the only thing that crosses it.
From the reach dataset, pulled Aug 5 · every figure carries its definition