
We evaluate CPU-only distributed inference of DeepSeek-R1-Distill-Llama-8B (dense 8B, Q40) on a four-node Raspberry Pi 5 (16 GB) cluster running distributed-llama. A clean-room measurement at the optimal thread count sustains 8.32 tok/s short-context decode (n=20, 95% CI [8.30, 8.34]), bit-exact — to our knowledge the best published figure for this model on Raspberry Pi 5, 29% above the prior four-node record (6.43 tok/s) and 37% above the Seeed Studio configuration (6.06 tok/s). We attribute the gain by layer: the production binary is byte-identical to clean upstream, so the source/kernel changes contribute approximately zero to dense decode (they add 86% to prefill and 15.2% to a 30B Mixture-of-Experts model in the companion report). Dense decode is bound by memory bandwidth; the gains live in the hardware and operating-system layers, not the application kernels. Using distributed-llama's per-token timing we show decode falls roughly linearly with context length — t(L) = 117 + 0.0174·L ms/token, R² = 0.999, prompt-independent — to about 5.0 tok/s at a 5,000-token context, so the 8.32 figure is the short-context peak. The four-thread optimum is context-independent (≈13% over three threads at every length). We additionally describe a bit-exact, adaptive prompt-lookup speculative decoder and a controlled dense-vs-MoE contrast on identical silicon. All figures are bit-exact (SHA-256 of the generated token-id stream at temperature 0, seed 42); no overclocking, quantisation change, or model substitution. Raw per-token timing data, benchmark harness and LaTeX sources are included.
CPU inference, KV-Cache, memory bandwidth, distributed-llama, distributed inference, Raspberry Pi, Raspberry Pi 5, edge AI, large language models, DeepSeek-R1
CPU inference, KV-Cache, memory bandwidth, distributed-llama, distributed inference, Raspberry Pi, Raspberry Pi 5, edge AI, large language models, DeepSeek-R1
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
