Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2026
License: CC BY
Data sources: ZENODO
https://doi.org/10.2139/ssrn.6...
Article . 2026 . Peer-reviewed
Data sources: Crossref
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
addClaim

Quality Challenges Do Not Escalate Source Fabrication in Large Language Models: A Pre-Registered Cross-Platform Factorial Study

Authors: Zoltan Varga;

Quality Challenges Do Not Escalate Source Fabrication in Large Language Models: A Pre-Registered Cross-Platform Factorial Study

Abstract

Background. Large language models (LLMs) frequently fabricate non-existent URLs and citations. A prominent hypothesis holds that rejection framing-prompts expressing dissatisfaction and demanding better sources-amplifies this behaviour by increasing the pressure on the model to produce specific references (schema activation lock hypothesis).&nbsp; <div> Methods. We conducted a pre-registered 2×5×7 mixed factorial study across two LLM platforms (ChatGPT gpt-4o; Perplexity sonar-pro) and five rejection-framing arms (BASE, SJR, UPR, NSAF, ECC), with seven conversation depths (k=0-6). Twenty fictional entities per platform (1,400 observations total) were used as probes. The primary outcome was the Fabricated Source Rate (FSR): fabricated URLs divided by total URLs cited. A two-tier validation protocol distinguished URL non-existence (HTTP Tier 1) from content misattribution (TF-IDF Tier 2, 30% stratified sample).&nbsp; </div> <div> Results. The primary hypothesis (H1: SJR rejection framing amplifies FSR relative to BASE, OR≥1.5) was not supported on either platform (ChatGPT: OR=1.00, p=0.998; Perplexity: OR=1.19, p=0.822). Uncertainty-preserving framing (UPR) produced complete suppression of fabrication at k≥1 on both platforms (0/240 positive events), as did gap-anchoring framing (NSAF) on Perplexity (0/120 events). Platform architecture was the dominant determinant of FSR: ChatGPT averaged FSR=0.075, Perplexity FSR=0.008 (9.2× difference). Content-level validation revealed that Perplexity's near-zero HTTP fabrication rate masks near-universal content misattribution (95.5% of HTTP-validated URLs pointed to topically related but entityunspecific pages). The expansion framing arm (ECC) showed a significant platformdifferential interaction: amplifier on ChatGPT, suppressor on Perplexity (OR=0.157, p=0.042).&nbsp; </div> <div> Conclusions. Quality challenge framing does not amplify URL fabrication; rather, the unrestricted baseline request produces maximal fabrication pressure. Uncertainty-preserving and gap-acknowledging framings provide robust, architecture-independent protection. Parametric and retrieval-grounded systems exhibit distinct failure modes that require different mitigation strategies. </div>

Keywords

retrieval-augmented generation, fictional entities, prompt framing, generalised estimating equations, source fabrication, large language models, GEE, hallucination, confabulation, URL validation, rejection framing, pre-registered study, CEE

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average