Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2025
License: CC BY
Data sources: ZENODO
ZENODO
Dataset . 2025
License: CC BY
Data sources: Datacite
ZENODO
Dataset . 2025
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

A Variant Character Dataset for Historical Narratives of Middle and Late Imperial China

Authors: Lee, Jiwon; Jung, Youngim;

A Variant Character Dataset for Historical Narratives of Middle and Late Imperial China

Abstract

This dataset, titled “A Variant Character Dataset for Historical Narratives of Middle and Late Imperial China,” addresses the widespread presence of variant characters in pre-modern Chinese texts, a significant challenge in computational analysis. In historical narratives written in Classical Chinese from the Tang to the Qing dynasties—much like the majority of pre-modern Chinese literature—there exist variant characters that share the same meaning but differ in appearance. Such orthographic variations can undermine the reliability of natural language processing tasks such as text similarity detection, keyword extraction, and analysis of historical text reuse. Moreover, modern encoding standards (e.g., Unicode) that capture minor visual differences in characters, as well as OCR technology that can mistakenly conflate these variations, often exacerbate the issue. To mitigate these challenges, this dataset compiles 2,723 variant–representative character mapping pairs that can be used to normalize text for computational tasks. It consolidates data from multiple authoritative sources, including Unihan Variants (Unicode Consortium), variant tables published by the State Council of the People’s Republic of China, and variant lists extracted from Wikisource and Kanripo corpora of Classical Chinese literature. After collecting initial candidates, duplicates and purely modern simplified characters were removed to retain only those relevant to historical narratives. Variant sets were then merged and streamlined, ensuring no character pair is duplicated. A prioritization scheme based on Taiwanese and Korean standard forms was implemented to select a single “representative” character for each set. This enables a “many-to-one” mapping, where each variant converges on a single standardized character. The resulting 1:1 pairs between each variant and its representative character are particularly valuable for improving text mining reliability in research on Middle and Late Imperial China. By applying these mappings, scholars can more accurately track word frequencies, investigate intertextual connections among historical works, and improve the quality of OCR outputs for further analysis. Additionally, the dataset holds significant potential for reuse in other East Asian contexts—such as Korea, Japan, and Vietnam—where historical texts are also replete with variant characters inherited from classical Chinese orthography. The open-access license (CC0) further allows researchers to adapt the dataset to different time periods, literary genres, and research agendas, fostering collaborative refinement over time. In sum, A Variant Character Dataset for Historical Narratives of Middle and Late Imperial China serves as an essential resource for anyone seeking to apply computational methods to pre-modern Chinese texts. By harmonizing variant characters into a unified standard, this dataset helps bridge the gap between the rich diversity of historical Chinese orthography and the structured requirements of modern computational analysis.

Keywords

classical chinese characters, Middle and Late Imperial China, historical narratives, Unicode, variant characters

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average