An NES CPU compiled into Qwen weights, no training required

The nesformer project runs a real Nintendo game's processor inside a standard Qwen3.5 checkpoint, and nobody trained it. A compiler wrote the weights directly. The model carries the NES's 6502 CPU core, its 2 KB of RAM, and a cartridge's program ROM, while a small host program handles the graphics, controller, and timing. It predicts the processor's execution log one token at a time, and the result matches a reference emulator exactly across millions of instructions.

The construction is the interesting part. Each attention-style memory head keeps a 128 by 128 state matrix, and the project treats every row as a memory cell, so RAM and registers live inside the model's state rather than in external storage. Seven layers form the circuit: one tracks state and counters, one handles the program counter and instruction fetch, and the rest decode, compute, and set flags. The checkpoint loads with a plain from_pretrained call and needs no custom code. Of its 466 million parameters, only 9.3 million are nonzero, so the shipped weights compress to 3.3 MB.

It is slow. A real NES runs at about 700,000 instructions per second, but nesformer manages roughly 1,650 tokens per second at best, between 150 and 290 times slower, so a game plays at about one to six seconds per frame. It supports only simple cartridges and official opcodes, and there is no sound. But it runs, verified frame by frame, and that is the point.

Why it matters

If you think of a transformer only as something you train on data, this is a concrete counterexample: the same architecture can be hand-compiled into an exact program, which is a useful mental model when you reason about what these networks can actually represent.