Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2026
License: CC BY SA
Data sources: ZENODO
ZENODO
Preprint . 2026
License: CC BY SA
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY SA
Data sources: Datacite
addClaim

Large Language Models on Wikipedia Editing: The Stylistic Mask Effect in a Randomized Controlled Intervention in Health Education

Authors: Corrale de Matos, Hector Gabriel; Wasmann, Jan-Willem; Peschanski, João Alexandre; Alvarenga, Kátia de Freitas; Jacob, Lilian Cassia Bornia;

Large Language Models on Wikipedia Editing: The Stylistic Mask Effect in a Randomized Controlled Intervention in Health Education

Abstract

The effects of large language models (LLMs) on content, references, and language quality in Wikipedia assignments remain unclear. This randomized controlled study assessed how model assistance influences the quality of Wikipedia edits created by undergraduate audiology students. Thirty-six participants were assigned to two groups: Group 1 (G1) edited without model support, and Group 2 (G2) edited with ChatGPT support. Twenty blinded expert reviewers evaluated a sample of 30 texts (15 per group) using a six-item Likert-scale instrument covering Content, References, and Language. Inter-rater reliability was high (Gwet's AC2 = 0.80). The analysis suggests that LLM assistance may lead to a significant improvement in Language (β = 0.400; p = 0.023; Cliff's δ = 0.400). G2 improved by 0.68 ± 0.47, while G1 improved by 0.28 ± 0.57. The analysis found no significant differences for Content (β = 0.167; p = 0.214) or References (β = −0.267; p = 0.859), although G2 scores in the latter category trended lower. Output counts were not associated with quality improvement in either group. This asymmetry is proposed as a potential Stylistic Mask Effect. It describes a gap between surface language gains and limited substantive development. LLM assistance appears to risk substituting constructive contribution with linguistic polish. This record contains the preprint manuscript. The manuscript has been submitted for peer review and has not yet been published. Dataset and code are made available to support transparency of the reported analyses.

Keywords

Encyclopedias as Topic, Audiology/education, Writing, Natural Language Processing, Education, Medical, Undergraduate

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
Related to Research communities