Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Report
Data sources: ZENODO
addClaim

Ruler Score Discrepancies in Llama-3.1-8B Benchmark Evaluations Across Studies

Authors: Assignee Research;

Ruler Score Discrepancies in Llama-3.1-8B Benchmark Evaluations Across Studies

Abstract

This report synthesises findings from 1 peer-reviewed paper addressing the following research question: Benchmark archaeology: investigate Ruler score discrepancy for Llama-3.1-8B --- reported 1.9\%--85.6\% (spread 83.7pp) across 2 papers. Sources: 'MTraining: Distributed Dynamic Sparse At' (1.9\%);. 8 claims were extracted from source literature; 8 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 9.0/10. This report is a machine-generated literature synthesis and does not constitute original research.Research goal: Benchmark archaeology: investigate Ruler score discrepancy for Llama-3.1-8B — reported 1.9%–85.6% (spread 83.7pp) across 2 papers. Sources: 'MTraining: Distributed Dynamic Sparse At' (1.9%); 'ReST-KV: Robust KV Cache Eviction with L' (85.6%). Identify evaluation protocol differences (few-shot, prompting, preprocessing).Autonomous literature synthesis. Automated review score: 9.0/10. Full text and citation available at Assignee Research.

Powered by OpenAIRE graph
Found an issue? Give us feedback