Measured, not projected.

Kernel-level microbenchmarks and end-to-end inference on real models, on Strix Halo. Every number below is real and labeled.

Kernel-level microbenchmarks.

bar width on a linear scale, relative to 433 tok/s

validated

Q1 GEMV (fused) ✓

433 tok/s ROCm HIP

Fused TQ2 (QKV + GU) ✓

420 tok/s ROCm HIP

BitNet TQ2_0 (native) ✓

420 tok/s ROCm HIP

Q1_0 binary ✓

380 tok/s ROCm HIP

TQ2 GEMV ✓

367 tok/s ROCm HIP

IQ1_S dequant + GEMV ✓

45 tok/s ROCm HIP

Synthetic 28-layer weight buffer. IQ1_M dequant is bit-exact vs the llama.cpp reference (50/56-byte block sizes fixed in the reader).

End-to-end inference.

real model · real prompts · Strix Halo

Qwen3-0.6Bnative NPU engine · XDNA 2

2.3 tok/s (435 ms/tok)

M=32 open kernels, BS=1. The old 7.4 tok/s figure was an invalid fake batch (issue #111) - top-K candidates as sequential tokens, non-causal. True batch (BS=8) measures 235-237 ms/tok; prefill 475 ms/9 tok.

Qwen3-0.6BGGML-Vulkan · Q4_K_M

373 tok/s

validated

BlackMamba 1.5BMamba1 HIP · Strix Halo

79.4 tok/s

Full decode, alternating SSM / MoE dispatch. Re-validated 2026-07-26 after shfl_xor_sync kernel fixes.

llama.cpp ROCmPrismML · third-party

229 tok/s

comparison point, not our engine

ZAYA1-8B-shapedAVX-512 CPU · portable path

2.5 tok/s

Real generate() loop, not a synthetic kernel. Steady-state 5-token average; first-token 4.37 tok/s.

Read the ternary numbers right. The TQ2 / TQ1 rows are kernel-level compute for ternary-native models (BitNet / Bonsai). They say nothing about dense-model quality: TQ2-quantizing a dense model destroys it (ppl 2.6e8 vs Q4NX 62). Dense models run Q4NX, and the engine never converts them silently.