Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2026
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2026
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2026
License: CC BY NC
Data sources: ZENODO
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY NC
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY NC
Data sources: Datacite
versions View all 4 versions
addClaim

An Exploratory Study of Inherited Bias in AI-Assisted Evaluation: Thirteen Corrections, Zero Content Changes

Authors: Lee, Taekyung;

An Exploratory Study of Inherited Bias in AI-Assisted Evaluation: Thirteen Corrections, Zero Content Changes

Abstract

This study reports an exploratory single-case study in which 82 creative works were submitted to an AI (Claude Opus 4.5) as anonymous text for evaluation. During the evaluation, 13 bias episodes were identified. Each time the human corrector logically challenged a deduction rationale grounded in information external to content, the AI revised its evaluation criteria and the score shifted—without any change to the content itself (1,790 to 2,255 out of 2,400, +25.9% as a descriptive within-session shift). The biases were grouped into five provisional categories: (1) reputation/authority, (2) diffusion/market, (3) format/medium, (4) tool/authorship, and (5) action/realization. Five preliminary design implications are proposed, including pre-evaluation debiasing prompts, iterative debiasing protocols, multi-model cross-validation, standardized human corrector roles, and bias audit reports. This study proposes that AI blind evaluation becomes effective not as a one-time blinding technique but when combined with an iterative correction attitude. The present design cannot fully distinguish bias reduction from sycophantic agreement or rubric renegotiation. Replication with independent evaluators and multiple AI models is needed. Keywords: AI bias, blind evaluation, halo effect, bias inheritance, human-AI collaborative assessment, cognitive bias correction, dialogic correction

Keywords

AI bias, dialogic correction, human-AI collaborative assessment, bias inheritance, blind evaluation, cognitive bias correction, halo effect

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average