TinyAttention
FlashAttention, built in hardware and run in your browser.
A synthesizable SystemVerilog engine for attention. It computes the output one tile at a time, so the N×N score matrix never goes to memory. The RTL is verified bit-exact against a golden model, and the same RTL runs here as WebAssembly.
The schedule
Each cell is a 4×4 tile of the attention matrix. Blue is computing, green is done, faded tiles are masked and never computed. The full matrix is never stored.
Output, HBM traffic, tile counts and cycle counts match the golden model on every run in the test sweep.
At N = 32, FLASH moves 1536 bytes against 7296 for the naive schedule, with the same result.
The Verilator model is compiled to an 83 KB (gzip) WebAssembly module. You step it clock by clock.
Start with what attention computes, then the online softmax that makes the tiling possible.