research · field notes
Public notes on the work. Papers we steal from, benchmarks we run, things we ship. The earliest posts carry measured figures we no longer trust, and each one is marked with a correction.
serial · 2026
The embedded Lemonade server core is re-vendored to v11.7.0 — new models, register/options/stats/metrics endpoints live, and a repeatable re-vendor loop so every upstream release lands without touching engine code. The engine's own HF coverage stays at 100%: 552 architecture tokens, 1,774 HF arch strings, 317,310/317,310 checkpoints mapped.
A two-node agent fleet on home hardware: two DeepSeek Harness agents on one LAN, both fed by 1bit.MONSTER. 35B at 166 tok/s on a Radeon 8060S, 8B at 96 tok/s on an RX 9070 XT, plus Qwen3-VL vision. Six engine bugs fixed along the way, with methodology.
AMD's official FastFlowLM numbers, decoded: 13.65 tok/s decode and 221.96 tok/s prefill fully on the NPU (Kraken Point), verified against the official Qwen config. Our own first pass: 75.65 tok/s via llama.cpp Vulkan on Strix Halo.
We extracted AMD's entire NPU model zoo: all 37 pre-built FLM models from ROCm/FastFlowLM v0.9.46, including the Qwen3.5 Omni multi-modal C++ source and Qwen3.6-MoE-35B with 256 experts. Auto-detection straight from the Q4NX header.
Zamba2 hybrid SSM and ZR1 reasoning, all 1BP. All four models converted and validated end-to-end on ZINC GPU: Zamba2 1.2B/2.7B/7B (Mamba2-hybrid) and ZR1-1.5B, 26 tok/s on Strix Halo. Two architectures, one engine.
Ternary and binary inference on NPU + GPU. Full support for Q1_0, TQ1, TQ2, IQ1_S/M and BitNet GGUF on both GPU (HIP/Vulkan) and NPU (XDNA 2), verified exact on real Strix Halo hardware. 4,200 lines across 31 files.
Train and fine-tune LLMs on Radeon, Instinct and Ryzen AI. Native AMD GPU support for training, fine-tuning and RL: 2x faster, 70% less VRAM, runs in 3GB. Complementary to our inference side.
Open-source token-level routing for multi-backend LLM inference on AMD Strix Halo. NPU draft plus GPU verify, with real logprobs on the wire.
AMD Strix Halo ships a 50 TOPS NPU. AMD's FLM runtime gets 11 tok/s and ROCm gets 41 tok/s; we drive the same NPU through a fused-layer engine. Throughput here is historical and unsourced.
DSpark was projected to hit 572 tok/s on Strix Halo. End-to-end measurement on the NPU disproved it: 0.1 to 0.2 tok/s at 0% draft acceptance. How and why, and what it would take to fix. Figures here are historical and unsourced.
The 97 tok/s number was real. The output behind it never was. Three silent bugs, LM head substitution, weight transpose and activation clipping, were producing garbage at full speed.
How a single tiny C++ binary auto-detects 73+ models across six backends, NPU, ROCm, Vulkan, CUDA, Metal and CPU, from one Q4NX header parse.
Seven engine versions in four days took NPU inference from 1,930 ms/tok to 3.4 ms/tok, a 72x speedup through batch decode, fused dispatch and INT8 GEMM. Throughput here is historical and unsourced.
Benchmarking the XDNA 2 NPU: historical NPU benchmark notes from the first month of the project. Figures quarantined as unsourced.
How QKV, attention and FFN combine into a single NPU dispatch, eliminating five of every six per-layer xclbin calls. Throughput here is historical and unsourced.
One person, a free Chess license and a C++ compiler turned AMD's locked-down NPU into an open-source inference engine. Throughput figures here are historical and unsourced.
A pi.dev-compatible coding agent CLI with 7 commands, NPU-native inference, package management, extensions, skills, themes and a systemd service. Ships as part of the 1bit MONSTER monorepo.
A 24x speedup in one session, from 244 to 10 ms/tok, beating FLM Kraken Point by 46%. OpenMP attention plus LM head. Full C++23, zero Python.
An early local run with one Lemonade endpoint over two compute lanes, iGPU llama.cpp ROCm plus NPU FLM, on Linux. Historical local run, kept for the record.
The notes will land. The engine is already there and it is free.
Read the source on GitHub