Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Report . 2026
License: CC BY
Data sources: ZENODO
ZENODO
Report . 2026
License: CC BY
Data sources: Datacite
ZENODO
Report . 2026
License: CC BY
Data sources: Datacite
addClaim

Shackled to the Machine — Breaking the DeepSeek-R1-8B Public Record by +29% on Four Raspberry Pi 5

Authors: Correa Villa, Daniel;

Shackled to the Machine — Breaking the DeepSeek-R1-8B Public Record by +29% on Four Raspberry Pi 5

Abstract

We evaluate CPU-only distributed inference of DeepSeek-R1-Distill-Llama-8B (dense 8B, Q40) on a four-node Raspberry Pi 5 (16 GB) cluster running distributed-llama. A clean-room measurement at the optimal thread count sustains 8.32 tok/s short-context decode (n=20, 95% CI [8.30, 8.34]), bit-exact — to our knowledge the best published figure for this model on Raspberry Pi 5, 29% above the prior four-node record (6.43 tok/s) and 37% above the Seeed Studio configuration (6.06 tok/s). We attribute the gain by layer: the production binary is byte-identical to clean upstream, so the source/kernel changes contribute approximately zero to dense decode (they add 86% to prefill and 15.2% to a 30B Mixture-of-Experts model in the companion report). Dense decode is bound by memory bandwidth; the gains live in the hardware and operating-system layers, not the application kernels. Using distributed-llama's per-token timing we show decode falls roughly linearly with context length — t(L) = 117 + 0.0174·L ms/token, R² = 0.999, prompt-independent — to about 5.0 tok/s at a 5,000-token context, so the 8.32 figure is the short-context peak. The four-thread optimum is context-independent (≈13% over three threads at every length). We additionally describe a bit-exact, adaptive prompt-lookup speculative decoder and a controlled dense-vs-MoE contrast on identical silicon. All figures are bit-exact (SHA-256 of the generated token-id stream at temperature 0, seed 42); no overclocking, quantisation change, or model substitution. Raw per-token timing data, benchmark harness and LaTeX sources are included.

Keywords

CPU inference, KV-Cache, memory bandwidth, distributed-llama, distributed inference, Raspberry Pi, Raspberry Pi 5, edge AI, large language models, DeepSeek-R1

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average