One month on my homelab LLM ran at nine tokens per second — measured, plotted, and explained across three posts. Then one weekend changed everything: the same model file, byte for byte, on a different engine, at forty-seven. What Strata does differently, which of my published diagnoses did not survive it, and the traps (buffered streams, thinking budgets, VRAM cliffs) between you and the same numbers.
local-ai
Tag: local-ai#
Eighteen Cores and a Stubborn Plateau: The Flash-Next CPU Upgrade Bench
Swapping the 10-core Xeon for an 18-core E5-2697 v4 delivered exactly the predicted +55 % decode — and disproved my own prefill diagnosis: the 212 t/s plateau never moved, proving it GPU-bound. Before/after bench plus a wall-socket energy comparison.
Chasing Flash-Next: bringing a 95 GB Qwen model to a 96 GB homelab
Sequel to the 35B post: a 94.5 GB, 177B MoE on the same 2016 Unraid box — how 1.1 t/s page-cache thrashing became 9,2 t/s of A/B-verified production, with the telemetry measured in the server's own counters and the true cost per token metered in joules at the wall socket.
Why Local AI Feels Slow: A Beginner's Guide to Tokens, Prefill and the Memory Wall
Prefill vs decode, why memory bandwidth caps tokens/s, MoE, quantization and context limits — a crash course with real measurements from a 12 GB GPU and a very patient Xeon.
A 35-billion-parameter model on a 12 GB GPU: measuring real-world speed for coding and note-taking
Benchmarking Qwen3.6-35B-A3B (MoE, 4-bit) on an Unraid box: Xeon E5-2630L v4, 64 GiB DDR4 ECC, RTX 3060 12 GiB. What prefill, decode and load times actually feel like for coding agents and Trilium note-taking.