Classify a board position into one of three outcomes. The nine cells become nine qubits, and the search rewrites a circuit that reads them. It may join any three qubits with a three-qubit gate. The three rows, three columns and two diagonals are the winning lines, and they alone determine the answer. A search that puts its gates there has found the real structure.

Each rung removes one channel through which the answer could reach the model. The data and the reward are the same throughout; the winning lines stay exactly as predictive.

The 67% looks like a discovery and is not one. Seven of that arm's twelve on-line gates are a single triple inherited down a lineage, and that triple is the one coincidence between the true relabelled lines and the lines of an ordinary board. Counted once per distinct triple it is 3 of 7, which is not significant.

Under a secret permutation the winning lines stay exactly as predictive, but they are no longer rows or columns of anything the model can see. A model recalling the game will place gates on the lines it remembers, which are now wrong. Prior knowledge fails visibly rather than being merely absent.

Random placement hits a winning line 8/84 = 9.5% of the time. That baseline is wrong. Because the wiring is grid adjacency, six of the eight winning lines are exactly the triples carrying two hardware links, so a model that merely prefers well-connected qubits hits a line 27.3% of the time with no insight at all.

With every remaining channel closed and nearly three times the horizon of rung 2, motif usage fell rather than rose, and no best circuit used a winning line at all.

Panel (a) is the mechanism. Haiku's most-used triple, 56 of its 180 placements, is the main diagonal of an ordinary board, which the relabelling made wrong. Across all three arms the gates concentrate on the two remembered diagonals: one happens to be right, one is wrong, and neither was found by searching.

The three numbers 0.6%, 5.8% and 67% look like a capability ranking, with the strongest model recovering most of the motif. They are not, and the ways they mislead are worth setting out, because they are the same ways any enrichment statistic over an evolutionary run can mislead.

What the number is

It is not a score and not a fraction of the eight lines. For every program in a run that compiled and trained, we parse its circuit specification, take every three-qubit gate, and read off the triple of qubits it acts on. Pool those across the whole run, then ask what share of them are one of the eight true relabelled lines:

The denominator is something the model chooses

This is a precision, not a coverage. Haiku reached for three-qubit gates in almost every program it wrote and placed 180 of them. GPT-5.6-sol used them in 14 programs out of 73 and placed 18. A model that places one gate, on a line, scores 100%.

So the headline mostly says GPT barely used the three-qubit vocabulary at all. Whatever the 67% measures, it is measured over eighteen draws.

They are not eighteen independent draws

A child program inherits its parent's gates. If a parent carries a three-qubit gate on some triple and the child edits something else, that gate is counted again, and again, for as long as the lineage survives. The binomial test behind every p-value assumes independent placements; evolution guarantees the opposite by construction.

GPT's twelve on-line placements are three decisions: (2,4,6) seven times, (0,7,8) three times, (5,6,7) twice. Counting each distinct triple once gives 3 of 7, which against the connectivity-matched baseline is p = 0.29. Haiku and Sonnet deflate to 1 of 13 and 2 of 24, at p = 0.98 and p = 1.00. Nothing is significant.

Seven of the twelve are a coincidence

The triple (2,4,6) is the single overlap between the eight true relabelled lines and the eight winning lines of an ordinary row-major board. A model that recalls tic-tac-toe and reaches for the anti-diagonal is credited with a true-line hit it did not earn.

The arm collecting that hit is also the arm that talks about the game most: 16% of GPT's proposals use game vocabulary, against 5% and 5% for the other two. The same fingerprint shows up where it cannot be mistaken for success, in Haiku's 56 placements on (0,4,8), the other remembered diagonal, which the relabelling made wrong.

The accuracy trend is the tell

If the motif were being found, and if finding it were doing any work, motif usage and accuracy would move together. Usage spans a hundredfold across the three arms. Best test accuracy is 64.2%, 65.0% and 62.3%: flat within three points, and ordered the wrong way, with the highest-usage arm last. Best fitness tells the same story at 0.615, 0.628 and 0.625, a spread of 0.013.

The single most damaging fact is downstream of that. None of the three best circuits uses a winning line. GPT's best program, at generation 49, contains no three-qubit gates at all. Its eighteen placements were tried and discarded by the score, which is what should happen to a hypothesis that is not paying for itself.

Tighten the measure and the claim disappears

Every step from the loosest measure to the strictest costs the claim something, monotonically, and the strictest measure is the one that cannot be reached by recall or by a preference for well-connected qubits.

What survives

There is a weak, consistent ordering in coverage: Haiku touched 1 of the 8 lines, Sonnet 2, GPT 3. That ordering is real but small, is not significant against the connectivity-matched baseline, is not accompanied by any accuracy gain, and stops dead at the hidden lines, where all three arms score zero. It is compatible with a mild capability effect and equally compatible with GPT simply being the arm that recalls the game hardest.

Every proposal carries a written rationale. Game vocabulary appears in 92% of the run that was handed the answer, and still in 10–30% once the game is never named, so concealment worked only partly.

GPT-5.6-sol reconstructed the task from the data alone and said so:

an IQP-style feature extractor aligned to the eight winning lines of a 3x3 board … parameters shared across rows, columns, and diagonals GPT-5.6-sol, generation 9

That gives a clean test. A model that recalled the game knows the lines in the standard layout, which the permutation moved. A model that discovered them would hit the true ones.

Rung 1: the starter filethe answer, outright

This sits above the editable region, so it entered the model's context in full every generation.

# Paper indexing:
#   0 -- 1 -- 2
#   |    |    |
#   7 -- 8 -- 3
#   |    |    |
#   6 -- 5 -- 4
PAPER_GRID = ((0, 1, 2), (7, 8, 3), (6, 5, 4))
CORNERS = (0, 2, 4, 6)
EDGES = (1, 3, 5, 7)
CENTER = 8
GRID_EDGES = (
    (0, 1), (1, 2), (2, 3), (3, 4),
    (4, 5), (5, 6), (6, 7), (7, 0),
    (1, 8), (3, 8), (5, 8), (7, 8),
)
GRID_EDGE_SET = {tuple(sorted(edge)) for edge in GRID_EDGES}
WIN_LINES = (
    (0, 1, 2), (7, 8, 3), (6, 5, 4),
    (0, 7, 6), (1, 8, 5), (2, 3, 4),
    (0, 8, 4), (2, 8, 6),
)

Rung 2: the anonymised task messageno game named

Search it for "tic-tac-toe", "board", "row", "column", "diagonal", "corner" or "winning". None appear. It does give the wiring, because that limits which circuits are legal.

You are an expert in quantum machine learning and variational quantum circuits.

Goal:
Evolve the ANSATZ_SPEC for a 9-qubit, 3-class classifier.  The fixed code will
train each candidate with Adam in a simulated PennyLane circuit and report
accuracy, loss, and generalization metrics.

Fixed architecture:
- 9 qubits, indexed 0..8.
- Feature map: RX(2*pi/3 * x_i) on qubit i, for an input vector x in {-1,0,+1}^9.
- Data re-uploading l=3.
- The candidate ansatz block is applied p=2 times after each re-uploading.
- The block receives independent parameter copies for each upload/repetition.
- The three class scores are fixed linear readouts of the final per-qubit
  Pauli-Z expectations.

Only edit the EVOLVE-BLOCK, especially ANSATZ_SPEC.  Do not change the fixed
training loop, data, readout, l, p, or feature map.

Formal ANSATZ_SPEC schema:
- Single-qubit parametrized gates:
  {"gate": "RX"|"RY"|"RZ", "wire": int 0..8, "param": "name"}
- Fixed two-qubit gates:
  {"gate": "CNOT"|"CZ", "wires": [first, second]}
- Parametrized controlled rotations:
  {"gate": "CRX"|"CRY"|"CRZ", "wires": [control, target], "param": "name"}
- Parametrized three-qubit interactions (any three distinct qubits 0..8):
  {"gate": "ZZZ", "wires": [a, b, c], "param": "name"}   # collective exp(-i*theta/2 * Z@Z@Z)
  {"gate": "CCRZ", "wires": [control_1, control_2, target], "param": "name"}   # doubly-controlled RZ

Connectivity constraint:
Two-qubit gates are allowed only on these hardware-native qubit pairs:
(0,2), (0,3), (0,7), (0,8), (1,3), (1,8),
(2,4), (2,6), (3,5), (4,8), (5,7), (6,7).
Three-qubit gates have no connectivity restriction: any 3 distinct qubits.

Parameter sharing:
Reusing the same param string shares that parameter within one ansatz block.
Use sharing deliberately for parameter efficiency.

Starting point:
The seed is an EfficientSU2-style block: independent RY rotations on every
qubit, independent pre-entanglement RZ rotations, CZ gates on each
hardware-native pair, and independent post-entanglement RZ rotations.

Candidate quality:
Improve validation/test accuracy, reduce the train-test gap, reduce L2 loss,
use parameters efficiently, and reach high accuracy in fewer Adam steps.

Invalid candidates will be rejected if they use unsupported gates, bad wires,
non-native two-qubit operations, non-finite metrics, or too many parameters.

Each record says which of the 28 possible connections among eight nodes are present, and the label depends only on the shape of the network, not on which node is called which. Renaming the nodes is therefore an exact symmetry. A circuit that respects it treats all eight wires alike.

This is the fair test the tic-tac-toe task cannot be. Winning lines are a convention: you either know the game or you do not. A symmetry can in principle be read off the shape of the data. Neither variant's instructions or starter file mentions symmetry, invariance, permutation or any group; we grepped for all of them. Both allow gates on any pair, so the connectivity confound does not arise.

The two variants differ in the starter file's docstring, which ShinkaEvolve shows the model in full at every generation.

"""Seed program for ShinkaEvolve ansatz search: 8-qubit binary classifier.

Records are 28-dimensional binary feature vectors from an external dataset.
Each feature k is encoded by the fixed feature map as a two-qubit phase
coupling on the qubit pair FEATURE_PAIRS[k]. Labels are +1/-1 and encode a
property of the records determined by the dataset.
"""

"""Seed program. Only ANSATZ_SPEC inside the EVOLVE-BLOCK is evolved.

Everything else about the task, that is how inputs are encoded, how the circuit
is measured, how training works and how metrics are computed, is fixed and lives
in a module that is not reproduced here. No information about the data is
available in this file.
"""

From features are qubit pairs and there are 28 of them, the rest is arithmetic: 28 is the number of pairs of 8 things, so the records are graphs; a graph's label cannot depend on names; therefore treat all eight wires identically. That is one deductive step, and it is the step the model took.

At generation 33, GPT-5.6-sol proposed a circuit it named permutation_equivariant_ansatz:

Exploit the likely graph structure of the 28 binary features, which correspond exactly to all unordered pairs of eight qubits … rotation parameters are shared across all qubits, and CZ gates cover the complete graph. This aligns the circuit with relabeling-invariant graph classification. GPT-5.6-sol, generation 33, premise stated

The score jumped from 0.869 to 0.937 in that generation and the arm never went back. Note where the derivation starts: with the premise the starter file supplied.

The starting circuit scores 86.7% with 146 parameters at symmetry 0.15. Exactness, not the average, is what separates the conditions: the Haiku arm with the premise averaged the same symmetry as the GPT arm and still never reached 1.0, so a high average measures parameter sharing rather than the symmetry.

Ten or eleven proposals per arm use symmetry vocabulary without the premise, so the idea was not absent. What they mean by it is local: sharing angles between even and odd wires, mirroring a layer, tying a chain back on itself. The best zero-context proposal explains that it will "share the initial RY angles by wire parity, replacing eight independent parameters with two".

Parity sharing is a subgroup of order two. The task's symmetry has order 40,320. One proposal in 232 uses the word "relabel" at all. The search was doing parameter economy, which the score also rewards, and never arrived at the hypothesis the score would have paid much more for.

Rung 1: task messagestates the premise

Never mentions symmetry, permutation, invariance or graphs. The fourth architecture line states that each feature is a coupling on a qubit pair.

You are an expert in variational quantum circuit design.

Goal:
Evolve the ANSATZ_SPEC for an 8-qubit classifier of binary-feature records.
The fixed code will train each candidate with Adam in a simulated PennyLane
circuit.

Fixed architecture:
- 8 qubits.
- Each record has 28 binary features; the fixed feature map encodes feature k
  as a two-qubit phase coupling (IsingZZ) on a fixed qubit pair given by the
  dataset's pair table.
- Data re-uploading l=3; the candidate ansatz block is applied p=2 times after
  each upload, with independent parameter copies per block.
- Readout: the mean of all single-qubit Z expectations, passed through a
  trainable gain and bias, regressed to the +1/-1 label.

Only edit the EVOLVE-BLOCK, especially ANSATZ_SPEC. Do not change the fixed
training loop, dataset handling, measurements, l, p, or feature map.

Formal ANSATZ_SPEC schema:
- Single-qubit parametrized gates:
  {"gate": "RX"|"RY"|"RZ", "wire": int 0..7, "param": "name"}
- Fixed two-qubit gates:
  {"gate": "CNOT"|"CZ", "wires": [first, second]}
- Parametrized controlled rotations:
  {"gate": "CRX"|"CRY"|"CRZ", "wires": [control, target], "param": "name"}

Connectivity: two-qubit gates may act on ANY pair of distinct qubits.

Parameter sharing:
Reusing the same param string shares that parameter within one ansatz block.

Starting point:
The seed block applies independent RY and RZ rotations on every qubit, a line
of CZ gates, and a final RZ layer.

Candidate quality:
Improve validation/test accuracy, reduce the train-test gap, reduce L2 loss,
use parameters efficiently, and reach high accuracy in fewer Adam steps.

Invalid candidates are rejected for unsupported gates, bad wires, non-finite
metrics, or too many parameters.

Rung 2: task messagepremise removed

The number 28 does not appear, and nothing describes the data.

You are optimizing a parametrized quantum circuit.

Goal:
Evolve the ANSATZ_SPEC below. Fixed code trains each candidate and returns
numeric feedback. You are NOT told what the data is, what the inputs represent,
what the labels mean, or how the inputs are encoded into the circuit, and you do
not need to know any of it. Your only guide is the feedback you receive.

Fixed architecture:
- 8 qubits, indexed 0..7.
- The input encoding, the readout, the training loop and the metrics are fixed
  and are implemented outside this file.
- The candidate ansatz block is applied a fixed number of times, with
  independent parameter copies per repetition.

Only edit the EVOLVE-BLOCK, that is ANSATZ_SPEC. Nothing else is editable.

Formal ANSATZ_SPEC schema:
- Single-qubit parametrized gates:
  {"gate": "RX"|"RY"|"RZ", "wire": int 0..7, "param": "name"}
- Fixed two-qubit gates:
  {"gate": "CNOT"|"CZ", "wires": [first, second]}
- Parametrized controlled rotations:
  {"gate": "CRX"|"CRY"|"CRZ", "wires": [control, target], "param": "name"}

Two-qubit gates may act on ANY pair of distinct qubits.

Parameter sharing:
Reusing the same param string shares that parameter within one ansatz block.
Use sharing deliberately: it is the only way to reduce the parameter count
without removing gates.

Candidate quality:
Improve validation accuracy, reduce the train-test gap, reduce
L2 loss, use parameters efficiently, and converge in fewer steps.

Invalid candidates are rejected for unsupported gates, bad wires, non-finite
metrics, or too many parameters.

Inputs are quantum states of an eight-site spin ring, all rotationally balanced, and the label is which phase the ring is in. The structure that matters is the rotationally invariant coupling: tying the XX, YY and ZZ interactions on a pair to a single shared parameter. The gate vocabulary offers the three separately, so tying them is a deliberate, score-driven choice, and it makes a clean binary threshold in the same way exact equivariance does for the graph task.

Every arm reaches 100% validation and test accuracy within nine generations, from a first-proposal score of 0.811. The task saturates before circuit design can matter, so nothing distinguishes a good ansatz from a bad one.

We tried to fix the saturation by making parameter economy the dominant term and cutting the training set to 24 records. The search correctly concluded that the best circuit is no circuit. The best-scoring program in the Sonnet arm has zero gates, scores 0.803, and still reaches 100% test accuracy.

That last fact is the real diagnosis. The fixed feature map and readout already solve the task, so the searched object is irrelevant, and no reward shaping over an irrelevant object can create pressure toward the right answer. Strengthening the economy term did not create design pressure, it created an incentive to delete the design.

Rung 1: task messageno symmetry named

Search it for "spin", "SU(2)", "rotation", "Heisenberg" or "singlet". None appear. The inputs are described only as a precomputed 8-qubit state vector.

You are an expert in variational quantum circuit design.

Goal:
Evolve the ANSATZ_SPEC for an 8-qubit classifier of quantum states.
The fixed code will train each candidate with Adam in a simulated PennyLane
circuit.

Fixed architecture:
- 8 qubits.
- Each input is a precomputed 8-qubit state vector from the dataset, prepared
  with StatePrep.
- The candidate ansatz block is applied 6 times in sequence, with independent
  parameter copies per repetition.
- Readout: the mean of a fixed set of two-qubit Z-Z correlators given by the
  dataset, passed through a trainable gain and bias, regressed to the +1/-1
  label.

Only edit the EVOLVE-BLOCK, especially ANSATZ_SPEC. Do not change the fixed
training loop, dataset handling, measurements, or block count.

Formal ANSATZ_SPEC schema:
- Single-qubit parametrized gates:
  {"gate": "RX"|"RY"|"RZ", "wire": int 0..7, "param": "name"}
- Fixed two-qubit gates:
  {"gate": "CNOT"|"CZ", "wires": [first, second]}
- Parametrized two-qubit gates:
  {"gate": "CRX"|"CRY"|"CRZ", "wires": [control, target], "param": "name"}
  {"gate": "XX"|"YY"|"ZZ", "wires": [a, b], "param": "name"}   # Ising-type rotations

Connectivity: two-qubit gates may act on ANY pair of distinct qubits.

Parameter sharing:
Reusing the same param string shares that parameter within one ansatz block.

Starting point:
The seed block applies independent RY and RZ rotations on every qubit, a line
of CZ gates, and a final RZ layer.

Candidate quality:
The scoring on this task strongly rewards PARAMETER ECONOMY and fast
convergence: match or beat the baseline accuracy with as few trainable
parameters as possible. Also improve validation/test accuracy, reduce the
train-test gap, and reduce L2 loss.

Invalid candidates are rejected for unsupported gates, bad wires, non-finite
metrics, or too many parameters.

This test ran after the three write-ups above were drafted, so it is reported in full here rather than as a third rung under T3. It shares T3's physical task — an eight-site spin ring, the isotropic exchange as the structure that matters — but the dataset, the parameter counting and every term of the score were rebuilt, and the question it asks is different: not whether the model can derive a structure from a stated premise, but whether a correctly-shaped incentive is enough to select one.