The fused layer engine
How QKV, attention and FFN combine into a single NPU dispatch, eliminating five of every six per-layer xclbin calls. Throughput here is historical and unsourced.
The per-layer tax
Before the fused engine, every transformer layer paid for the same work in separate installments. QKV projection, attention and the feed-forward each went out as its own xclbin dispatch, and each dispatch carried launch overhead that did no math.
Most of a layer's time was spent talking to the NPU, not computing on it. Five of every six per-layer xclbin calls existed to be eliminated.
One dispatch per layer
| operation | before | after |
|---|---|---|
| QKV projection | separate call | fused |
| attention | separate call | fused |
| feed-forward | separate call | fused |
| xclbin calls per layer | 6 | 1 |
QKV, attention and FFN now combine into a single NPU dispatch: 1 call per transformer layer instead of 6. Five of every six per-layer xclbin calls are gone.
What it buys
The win is not a faster multiply, it is less idle time. The NPU receives a continuous stream of work instead of a queue of requests, and the launch latency that used to be overhead becomes compute.
The caveat
The throughput figures attached to this engine are historical and unsourced, quarantined in benchmarks/latest.json _unverified. The architecture is current; the numbers attached to it here are not evidence.
The pattern
Fusing was the difference between a working engine and a fast one. It is the architecture the site ships today, and the discipline of marking which parts of a claim are still standing.