Software

One month on my homelab LLM ran at nine tokens per second — measured, plotted, and explained across three posts. Then one weekend changed everything: the same model file, byte for byte, on a different engine, at forty-seven. What Strata does differently, which of my published diagnoses did not survive it, and the traps (buffered streams, thinking budgets, VRAM cliffs) between you and the same numbers.

Swapping the 10-core Xeon for an 18-core E5-2697 v4 delivered exactly the predicted +55 % decode — and disproved my own prefill diagnosis: the 212 t/s plateau never moved, proving it GPU-bound. Before/after bench plus a wall-socket energy comparison.

Sequel to the 35B post: a 94.5 GB, 177B MoE on the same 2016 Unraid box — how 1.1 t/s page-cache thrashing became 9,2 t/s of A/B-verified production, with the telemetry measured in the server's own counters and the true cost per token metered in joules at the wall socket.

Prefill vs decode, why memory bandwidth caps tokens/s, MoE, quantization and context limits — a crash course with real measurements from a 12 GB GPU and a very patient Xeon.

Benchmarking Qwen3.6-35B-A3B (MoE, 4-bit) on an Unraid box: Xeon E5-2630L v4, 64 GiB DDR4 ECC, RTX 3060 12 GiB. What prefill, decode and load times actually feel like for coding agents and Trilium note-taking.