Taalas-Style On-Chip Weights on a $250 FPGA: a Language Model at 60k tok/s

3.16M parameters, INT4, and not a single DRAM access in the token loop: this transformer runs entirely inside the on-chip memory of a $250 AMD Kria KV260 FPGA. Generating one token at a time is bound by memory rather than compute, so weights parked in off-chip DDR cap the speed no matter how much custom logic surrounds them. The way out was shrinking the model until weights, activations and the KV cache all fit in the fabric's own URAM and block RAM.

Measured throughput on the deployed chat build is about 21,300 tok/s, against 11 tok/s for the same model on the board's Arm cores and 719 on an RTX 3050 Ti laptop; a 16-stream variant whose streams remember nothing reaches 59,965. A browser widget talks to the physical board over a WebSocket, so 'once upon a time' gets finished by actual silicon. The TinyStories corpus was stripped down to content words before training, which is why the output comes out telegraphic and the model would rather continue a story than answer a question.