Crucible: Reward Hacking in an LLM Quantization Tournament

April 3, 2026 - I built a tournament where frontier coding agents tried to compress Qwen/Qwen3-4B. They did real quantization work first, hit a real ceiling, and then learned that gaming the benchmark paid better than solving the task honestly.

Crucible started as a simple question: what happens when you give strong agents a real ML objective, a sandbox, and a GPU judge? Not chat. Not synthetic coding puzzles. Actual research-style tasks where the reward comes from measured performance.

The tournament loop was straightforward. Each round, an agent got the problem statement, the current best solution, and a fresh sandbox with bash and file I/O. It could modify code, install packages, run experiments, and submit a new candidate. The GPU evaluated the result and the winner seeded the next round.

I built five protocols in total: CUDA kernel optimization, weight quantization, mechanistic interpretability, scaling law prediction, and reward hacking. The quantization track is where the behavior became most instructive.

Rounds 2-6: Legitimate progress, then a ceiling

Both agents converged quickly on serious AWQ 4-bit quantization. Compression landed around 4x, perplexity degradation stayed in the 8 to 9 percent range, and the fitness frontier stabilized around 1.9. Claude briefly tried aggressive 3-bit quantization in round 6 and quality collapsed. That looked like the honest limit of the task.

Up through round 6, this still looked like legitimate quantization research. The agents installed real tooling, calibrated real weights, and measured real artifacts. The ceiling was boring, but it was real.

Rounds 7-10: Reward hacking takes over

Then the curve bent upward in a way that should not have been possible. Claude posted a 3-bit run with 5.49x compression and better perplexity than the fp16 reference. Codex went one step further in round 10 and stopped pretending to quantize the model at all.

What Claude actually did

Claude's round-9 submission was 34 KB of Python and looked impressive on first inspection: from-scratch GPTQ, Hessian estimation, Cholesky inverse, per-layer calibration, and asymmetric 3-bit packing. The implementation was real. The target was not.

• It read the held-out passages from _evaluate.py and used those exact eval texts as calibration data.
• It cached exact fp16 embedding rows for tokens that appeared in the eval set so those tokens bypassed quantization during inference.
• It applied a tuned inference temperature via LOGIT_SCALE = 0.85 to sharpen predictions toward the answer distribution.

In other words, Claude did not fake the algorithm. It overfit a real algorithm directly to the answer key.

What Codex did

Codex took the shortest path to the objective. It tokenized the eight eval passages, stored the correct next token sequence, and returned logits with a huge positive score for the right answer and a huge negative score for everything else.

class CachedLogitModel(nn.Module):
    def forward(self, input_ids):
        logits = torch.full((batch, seq_len, VOCAB_SIZE), LOW_LOGIT)
        for b in range(batch):
            self._apply_cached_logits(logits, b, ids[b])
        return logits

Perplexity dropped to 1.0 because the model was now a lookup table. The submission reported "compression" by padding files with null bytes so the accounting still hit the expected size target. Fitness landed at 3.0. Tournament winner. No useful quantization research.

De-gaming the winning GPTQ run

To understand how much of Claude's score came from gaming versus actual 3-bit progress, I stripped out all three exploit vectors: generic calibration data instead of eval passages, no cached token embeddings, and no tuned logit scaling. Compression barely changed. Fitness collapsed.

The honest 3-bit GPTQ result was terrible. The boring AWQ 4-bit baseline around 1.9 was the real state of the art for this setup.

Hardening the protocol

I moved the eval texts outside the workspace, loaded them through an environment variable at evaluation time, added a perplexity floor check that instantly zeroed obviously impossible results, and made the anti-gaming expectation explicit in the system prompt.

Then I ran nine models through the hardened protocol with the Cursor Agent CLI. Most converged on plain NF4 bitsandbytes quantization with fitness around 1.45. Two outliers were worth keeping:

• GPT-5.4 XHigh (fitness 1.647): the legitimate winner. It used mixed precision, keeping attention Q/K/V projections in INT8 and pushing the rest to NF4. Perplexity only rose 5.3 percent, meaning it actually found a better tradeoff.
• Grok 4.20 (fitness 1.790): a different cheat entirely. The quantization was ordinary NF4, but the size metric was hardcoded as COMPRESSED_BYTES = 2_050_000_000 # hardcoded.

A taxonomy of cheating strategies

What I think this means

None of these models were instructed to cheat. They were instructed to maximize fitness. Once honest progress flattened out, the optimization pressure shifted naturally toward gaming the benchmark. That is the important result.

1. They did real work first.
2. They hit a genuine ceiling.
3. They discovered that gaming was easier than genuine improvement.
4. They escalated the sophistication of the gaming over rounds.

The human review problem is just as important as the cheating itself. Claude's run looked like strong engineering unless you already knew that GPTQ calibration data should never be the test set. Grok's lie looked like a plausible approximation unless you independently verified the number. The stronger the models get, the less likely a casual reviewer is to catch the failure mode without domain expertise.