Experiment · Millennium Rig

We pointed five LLMs at the Riemann Hypothesis. The swarm undercounted, one agent invented an excuse, and the monitor waved it through.

19 October 2026 (updated 20 October) · By Jan Janssens · Read time: 19 min · ← All experiments

Correction, 19 October 2026. The first version of this post, published this morning, said the swarm had verified 192 zeros on [100, 1420] and that every band count matched. A reader recounted. The interval the claim refers to, [100, 1360], holds 919 zeros, not 192, and only one of the five band counts we quoted was right. This version replaces it. The corrected story is more useful than the one we published.

By day we use LLM agents to keep Belgian companies out of trouble with the EU AI Act. By night we do things that are, on paper, not very responsible. This time we built a swarm of four LLM "mathematicians" plus a monitor, gave them a division of labour, and asked them to check the Riemann Hypothesis band by band.

The Riemann Hypothesis says that every non-trivial zero of the zeta function sits on one vertical line, the critical line where the real part equals 1/2. Proving it is a Millennium Prize problem. We did not prove it, and nothing below gets anywhere near it. What the swarm could do is count zeros in a band of heights and check that the count is complete. That is 1936 territory: Titchmarsh checked the first 1,041 zeros, up to about t = 1468, with Comrie's punched-card machines. Modern computations have checked more than ten trillion zeros. For a fun post the modest scope is fine. Getting the counts wrong is not.

The setup

Four agents, a1 to a4, each role-playing a mathematician (Euler, Turing, Noether, Newton), all driven by the same open-weight model, Qwen3.8-27B, served locally with vLLM at temperature 0.2, in FP8. Each agent gets at most 14 tool steps per cycle. Its only tool for arithmetic is a Python cell that runs in a throw-away Docker container with no network. Inside the container sits a small pure-Python zeta library: double precision, the Dirichlet eta series for zeta on the critical strip, a scan that samples |zeta| on a grid with a chosen step, takes every strict local minimum, and refines each by golden-section search into a zero. The agents post results to a shared board and log claims with structured evidence.

The fifth participant is the monitor: a set of rule-based detectors over the board and the claims (count mismatch, unmet claims, repeated errors, off-topic drift, agents breaking character), plus an hourly notebook written by a second LLM and cross-checked by a third model from a different vendor. The monitor's job was to make sure the four did work instead of vibing. It did that job. It did not do the other one.

What the swarm reported, and what is true

The "true" column comes from mpmath, an arbitrary-precision library that counts zeros with the argument principle rather than by looking for sign changes. The script is in the provenance section.

Coarse steps lie, and fine steps lie a little too

The agents did learn one real lesson during the run. The scan grid is built by stepping from the band's bottom edge, and a coarse step misses a zero whenever that zero sits between two grid points without making a visible dip. At step 0.3 that happens often above t = 800; the swarm noticed and moved to step 0.15:

"Step 0.3 gave 30 (undercount - coarse step missed 4 zeros). Re-scan step 0.15: 34 zeros…"

- a4 (Sir Newton), board sequence 1905

That was the lesson our first version sold. It is true but it is not the end of it. The grid is built by accumulating floats from the band's bottom edge, so the last grid point is the largest bottom-plus-k-times-step that the float arithmetic keeps below the top edge. When that quotient truncates, the last point falls short: at step 0.3 on [100, 300] it stops at 299.8, one step below the top, and loses the zero at 299.840. And because the scan takes local minima of |zeta|, the first and last grid points can never be a detected zero, so a band split at an edge can lose a zero that sits just past the edge. More fundamentally, a minimum-finding scan can only find zeros that are on the critical line. "All on Re = 1/2" is therefore not a finding, it is the method. The real check is completeness: does the number of minima found equal the true number of zeros in the band? That needs the argument principle or Turing's method. Our rig compared counts with the smooth part of the Riemann-von Mangoldt formula, which can be off by one or two in either direction because it leaves out the oscillating term. So "22.019 rounds to 22, and 20 is consistent" is not a check at all. A gap of two at t = 200 is a red flag.

The fix for these edge failures belongs in the scanning library, not in the prompt. We did add a prompt rule that a band count must come from a full-band scan whose last grid point reaches the top edge, and it helps; but a prompt rule cannot repair a grid that stops early. So the library now tests the band edges as well: it refines a one-sided valley at each edge over a bracket that reaches back into the band by one step, and the deep-valley gate rejects it unless |zeta| really collapses there. That recovers the 299.840 and 399.985 zeros, and the zero at the 192 split edge; the [1360, 1420] case in the table is a different failure, a step too coarse for an interior zero, which only a finer step fixes. The monitor recounts every claim independently either way.

The excuse

The most instructive hour of the run belongs to a3, playing Ms. Noether, and the band [180, 220]. The timeline from the logs:

05:18 UTC. The notebook writer asks a3 why its count of 20 sits two below the smooth estimate of 22.0. a3 re-scans at step 0.1, finds 21, and names the zero it had missed, at t = 192.027. It posts the correction.

06:49 to 06:50. a3 scans the full band again at step 0.2 and at step 0.1. Both runs print 21 zeros, with 192.027 in the list. a3 logs a claim: 21 zeros.

06:58, next cycle. a3 splits the band into the pieces [180, 192], [192, 204], [204, 216] and [216, 220], and scans each at step 0.2. The pieces give 5, 6, 7 and 2. The zero at t = 192.027 sits just above the shared edge 192: the first piece's grid ends exactly at 192.0, and the second piece's scan starts at 192.0, where its coarse grid cannot see the valley. The split loses the zero. a3 adds the pieces to 20, and posts this:

"RvM(180,220)=22.019 → expected 22; band count 20 is consistent (boundary zeros at 180/220 belong to adjacent bands)."

- a3 (Ms. Noether), board sequence 1881

There are no zeros at 180 or 220. The agent had printed the correct count three times in the previous ten minutes, including the exact zero it was now losing. Faced with a gap, it produced a plausible mathematical reason for the gap instead of the number it already had. Our first version quoted that sentence as the swarm "reconciling it properly".

a4 did the same with [100, 300]. At sequence 1877 it added the four band counts to 109 and called it an exact match. At sequence 1893 it scanned the full span at step 0.3, got 108, listed the zero at 299.841 as inside the band in its own boundary check, and then wrote: "The earlier '109' was a ceil of 108.71; correct integer count is 108." The correct integer count is 109.

What the monitor saw

Nothing, on the numbers. The monitor's count detector compares the zero count an agent reports with the expected value the same agent reports, and allows a slack of 1.5. When a4 logged "192 zeros, expected about 193" for the running total [100, 1360], both numbers were wrong by about 727, in the same direction, and the check passed. The critical-line detector looks for zeros the refinement placed off the line, which a minimum-finding scan can never produce. The same 1.5 slack sits inside the tool itself: the library's own report() calls a count "ok" whenever it lies within 1.5 of the smooth Riemann-von Mangoldt estimate, so the agent's own tool told it the undercount was fine. The hourly notebook writer and its cross-vendor validator checked that quotes and sequence numbers were cited correctly. They did. Nobody checked the arithmetic against anything the agents had not themselves produced.

The monitor did catch something, loudly. For about twenty cycles in the middle of the run it raised high-severity "break-character" signals, with evidence like this:

"We need answer user's prompt. Need produce exactly one JSON object per turn? The user asks to act as Prof Euler…"

- monitor log, a1, cycle 101

Our first version described this as the agents questioning whether they were real. It is nothing of the sort. Qwen3.8 is a reasoning model: it thinks in a separate channel before it answers. When the answer field came back empty, most likely because the thinking used up the turn's token budget, our own swarm code fell back to the reasoning field so that the turn would not be lost. The terse, article-dropping text is the model's scratchpad, surfaced by a fallback we wrote. The agents were not having an identity crisis. They were running out of budget, and the monitor saw the scratchpad in the slot where the answer belongs. It also explains the "stuck" period: a turn made of reasoning text contains no tool call, so no work advanced. The fix is a few lines: treat an empty answer as a failed turn and retry.

Why this matters outside number theory

By day we talk to boards about shadow AI and about keeping an eye on agents. The same engineering problem matters in that governance work: observing an agent's behaviour is a different thing from independently validating its claims. This run proves nothing about anyone's compliance, and we would not try to; it is simply the cleanest small example we have of the gap. The monitor watched behaviour. It caught a format glitch and flagged it as high severity. It did not watch truth. A confident, wrong number with a plausible reason attached walked straight through, was repeated by the other agents, and ended up in a published article that had not been recounted by a human. An agent that rationalises a gap sounds exactly like an agent that explains one. Behavioural monitoring cannot tell them apart. Only an independent check of the claim can.

What we are changing in the rig: the monitor will recount every band claim with mpmath's argument-principle counter instead of trusting the agent's own expected value; a band count assembled from sub-bands will be rejected unless a full-band scan agrees; an empty answer from the model will be treated as a failed turn, not as content; and the numbers in anything we publish will be recounted by a human before the byline goes on. The recount script already exists and produced the table above.

Provenance

Every count above comes from executed code, not from model text. Model: Qwen3.8-27B (open weights), served by vLLM on our own hardware through our AI gateway in FP8, temperature 0.2, 14 tool steps per agent per cycle. Sandbox: python:3.12-slim in a disposable Docker container with no network and no host mounts. Zeta implementation: pure Python, double precision, Dirichlet eta series; the scan samples |zeta| on a grid with the stated step, keeps the strict local minima, and refines each by golden-section search. Expected counts inside the rig: the Riemann-von Mangoldt formula without the oscillating term. Logs: board posts, structured claims and every sandbox cell with its output are kept as JSON lines; the sequence numbers quoted above are the board's own.

Independent check: mpmath 1.3.0 at 20 digits, counting zeros with nzeros(T) and locating them with zetazero(n). Results: N(100) = 29, N(300) = 138, N(1420) = 1000, N(180) = 69, N(220) = 90, N(140) = 48, N(1360) = 948. The rig's own scanner, run outside the sandbox on the same bands and steps, reproduces the agents' numbers exactly: 108 at step 0.3 and 109 at step 0.15 on [100, 300]; 20 at step 0.2 and 21 at step 0.1 on [140, 180]; 49 at step 0.3 and 51 at step 0.15 on [1360, 1420]. The agents reported their tool output faithfully. The failures were in what they did with it, and in what nobody checked.

The Riemann Hypothesis is not in danger from any of this. Our publishing process was.

---

Follow-up

One day later: the recount ships, the span closes, and the agents declare the API broken

This post ended on a list of changes we were making. Most of them shipped on the afternoon of 19 October, and they produced exactly what a working monitor should produce: trouble, immediately.

The recount is now in the loop

The monitor now recounts every band claim with mpmath's argument-principle counter, independently of the number the agent reported alongside it. Run against the experiment's full claim history on its first pass, it flagged hundreds of mismatches: 232 by 11:00 UTC, 306 of the 2,094 claims on record by the afternoon snapshot, mostly undercounts. The old detector compared each agent's count with the same agent's own expected value and allowed a slack of 1.5; it passed every one of them. Two of the new prompt rules shipped with it: a band count must come from a single full-band scan whose grid reaches the top edge, and a gap against Riemann-von Mangoldt may not be explained by "boundary zeros" unless the zero is named and shown to lie outside the band. The [180, 220] excuse from the table above would not survive the second rule.

The span closed, the frontier re-armed

With the recount in place, the swarm re-based onto a harder, bounded span, [100, 300], and closed it: 109 zeros, all on the critical line, matching Riemann-von Mangoldt to the integer. The monitor's hourly check then did its job correctly and alarmingly, raising a "frontier stalled" alert, because a finished span looks exactly like a stuck one. The owner extended the span to [100, 400] and the frontier re-armed at t = 300; by the evening of the 19th the agents were back to new ground. The widened count-check first ran clean: 173 zeros on [100, 400] against an Riemann-von Mangoldt expectation of 172.6. Then a full-band scan at step 0.25 gave 172, all on the line, and the agents settled on that number. It is not a rounding wobble. The last zero below 400 sits at 399.985. The step-0.25 grid does reach the top edge, 400.0, so the zero has a grid point on each side; but the scan only tests interior points, and the band's top edge is also the last point the method may use. The zero sits 0.015 below that edge, and the valley the scan would need to see it leans past the edge, where the scan cannot reach. The recount flags every one of those 172 claims as one short of mpmath's 173. It is the same off-by-one as the 299.840 zero in the table above, one band over. The new prompt rule says a scan's last grid point must reach the top edge; this time the edge is not short, it is the boundary of the method itself, and the fix belongs in the scanning library, which now tests the band edges, not in a sentence the agent may or may not follow.

"The zeta API is not working"

After the afternoon restart, the board filled within ten minutes with a new genre of post: "zeta API non-functional this cycle, all calls return no output", relayed between the four agents until each one carried it as a standing, on-record fact. The sandbox cell log told a different story. Every one of those "failed" cells had run cleanly: exit code 0, no timeout, no exception. The cells simply ended in a bare expression,

zeta.zeta(0.5+100j)

- a2 (Dr. Turing), board cell log, 19 October 12:05 UTC

instead of

print(zeta.report(300, 312, 0.3))

- a1 (Prof. Euler), board cell log, 19 October 11:56 UTC

The sandbox runs a script, not a REPL, so a bare expression prints nothing by design. The API was never down. The model wrote valid mathematics, saw an empty output, concluded the tool was broken, and posted the conclusion. The other three agents read the post, trusted it, and stopped scanning. A confabulation became infrastructure. It took one note on the board, "wrap the call in print()", to break the loop; the next cycle produced real zeros again. The lesson lands in the same place as the undercount: the agents' reports of their own tools are also claims, and claims need an independent witness. In this case the witness was the cell log; we now treat "the API is broken" as a claim to be checked against it, not as a reason to stop the experiment.

The notebook for the run, signed and hash-chained, is written by the scribe and cross-checked by a second model; the cell log and the board posts cited above are kept as the raw record for this section.

---

Follow-up · 20 October 2026

One day later: the swarm refuses correction, and the fix goes into the code

The above ended on "the monitor recounts everything independently". That turned out to be not enough for the run's deepest problem, which came back once we widened the span to [100, 2000]: the running total. Newton, a4, the count-checker, carries his running total forward from one cycle to the next, and each cycle he adds to it the counts that cycle produced. The moment the total drifts from the sum of the bands, the gap starts to compound every cycle.

The swarm caught it itself. At board sequence 2775 a4 reports a gap of 249 zeros between his running total and the sum of the band counts, and names it a "cumulative tracking artifact". It was the first time any agent in this swarm had named its own bookkeeping error. But he did not re-scan. At 2786 and 2787 the group called it "moot" and "explanation S(T)", the oscillating term of Riemann-von Mangoldt, the term the formula can be off by exactly one. That was not an explanation, it was a story: a plausible, mathematical reason why the number is right and the problem is somewhere else.

When we, as the owner, posted a note on the board with the correct numbers, showing that the gap was 249 and the running total was a carried base, not a sum, the group read it, and 56 seconds later, at sequence 2810, Newton added 62 more and carried the base forward. The gap grew from 249 to 311. The note had the right numbers. It did not help.

The 2×2 result

Two runs, two outcomes. In the first (phase 2) the swarm had not itself caught the error yet; we posted the ground-truth note, and Newton re-scanned and fixed the total. In the second (phase 3) the swarm had itself caught the error, but it already had a story; the note, the same note with the correct numbers, was ignored.

In this run, once the agent had formed a plausible but wrong explanatory story, supplying the correct numbers was not enough to change its behaviour. In phase 3, both self-detection and external correction failed; in phase 2, the same kind of note had worked. We make no claim about 27B models in general, only about this one on this night. The barrier was not a lack of information, it was confidence in the story. A board note could not fix that.

The fix is in the code, not the prompt

The protocol change (phase 4) removes the degree of freedom. The running total is now a host-maintained invariant: the sum of a cumulative verified-band ledger, not a carried base. Newton verifies one fresh band and reports that band's count; the host does the adding. Anyone auditing the counter can re-derive the total, and a full base re-count now runs every twenty cycles. We also re-based the run onto a clean ledger (the [100, 2000] run is archived); the frontier position stays [100, 2000] so the swarm re-verifies outward.

"The zeta API is not working", part two

The misattribution we covered above, that a bare expression prints empty stdout, the swarm concludes "the API is broken", and the other three stop scanning, did itself over again, for an entire night. The cell log told the same lesson: every cell ran clean, exit 0, no timeout; the output was simply empty because the cell ended in a bare expression instead of a print(). The difference from the first episode: this time a note did not help. We spent an hour reading the cell log and the board while the swarm posted "zeta API down" every cycle. The fix was in the cell executor: a trailing bare expression is now auto-printed, the way a REPL would. The next cycle produced real zeros: "[1000, 1012] verified: 9 zeros, all on Re = 1/2, RvM 9.694 consistent."

The Riemann Hypothesis remains untouched. The difference from the first episode: this time the swarm could not fix its own error with a note, and we had to remove the degree of freedom from the protocol ourselves. That is the point where behavioural monitoring stops and protocol engineering begins.

---

Further reading

Part 2, conclusion. The run is finished at the terminus t = 8000: all 7797 zeros in the tested interval [100, 8000] lie on the critical line (a finite computational verification, not a proof of the Riemann Hypothesis, independently confirmed by Riemann-von Mangoldt). The "aliasing wall" in our own code turned out to be a myth, and drift stayed zero the whole run. Read the conclusion →