Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset . 2026
License: CC BY
Data sources: ZENODO
ZENODO
Dataset . 2026
License: CC BY
Data sources: Datacite
ZENODO
Dataset . 2026
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

TCGA-242-Pathology-Benchmark: External Validation Dataset for "The Digital Registrar"

Authors: Chang, Kai-Po; Chow, Nan-Haw; Tatonetti, Nicholas;

TCGA-242-Pathology-Benchmark: External Validation Dataset for "The Digital Registrar"

Abstract

This dataset contains the Independent External Validation (IEV) cohort used in the study: Digital Registrar: A Schema-First Framework for Multi-Cancer Privacy-Preserving Pathology Abstraction via Local LLMs. Diagnostics. 2026; 16(11):1644. It comprises 242 de-identified surgical pathology reports selected from The Cancer Genome Atlas (TCGA) program, extracted by Kefeli, J. & Tatonetti, N. (2025, please see reference) covering five distinct categories: Breast, Colorectal, Esophagus, Stomach, and "Others" (including synchronous bilateral cancers). This benchmark serves as a "Gold Standard" for evaluating Large Language Models (LLMs) on clinical extraction tasks, providing raw text, expert-verified ground truth annotations, and baseline model outputs. If you use this code or the dataset in your research, please cite: Chow N-H, Chang H, Chen H-K, Lin C-Y, Liu Y-L, Tseng P-Y, Shiu L-J, Chu Y-W, Chung P-C, Chang K-P. Digital Registrar: A Schema-First Framework for Multi-Cancer Privacy-Preserving Pathology Abstraction via Local LLMs. Diagnostics. 2026; 16(11):1644. https://doi.org/10.3390/diagnostics16111644 BibTeX @Article{diagnostics16111644, AUTHOR = {Chow, Nan-Haw and Chang, Han and Chen, Hung-Kai and Lin, Chen-Yuan and Liu, Ying-Lung and Tseng, Po-Yen and Shiu, Li-Ju and Chu, Yen-Wei and Chung, Pau-Choo and Chang, Kai-Po}, TITLE = {Digital Registrar: A Schema-First Framework for Multi-Cancer Privacy-Preserving Pathology Abstraction via Local LLMs}, JOURNAL = {Diagnostics}, VOLUME = {16}, YEAR = {2026}, NUMBER = {11}, ARTICLE-NUMBER = {1644}, URL = {https://www.mdpi.com/2075-4418/16/11/1644}, ISSN = {2075-4418}, ABSTRACT = {Background/Objectives: Free-text surgical pathology reports hinder automated cancer registry entry and secondary analytics. This study introduces a clinically governed schema layer for interoperability, testing whether a locally-deployable Large Language Model (LLM) pipeline can deliver robust registry-grade extraction across institutions. Methods: We developed a College of American Pathologists (CAP)-aligned clinical ontology encompassing 10 cancer types, 192 per-organ scalar fields, key biomarkers, and nested structures for lymph nodes and margins. Encoded via Declarative Self-improving Python (DSPy) signatures with grammar-constrained decoding using DSPy v3.2.1, this model-agnostic pipeline was benchmarked on 893 internal reports against a pathologist-adjudicated gold standard. External validation utilized 242 The Cancer Genome Atlas (TCGA) reports. Hardware feasibility was confirmed on a single 48-gigabyte (GB) Graphics Processing Unit (GPU), ensuring suitability for privacy-preserving on-premises deployment. Results: Using the gpt-oss-20b model, the framework achieved 92.0% macro-mean exact-match accuracy on internal data, demonstrating near-perfect run-to-run reliability. Critical prognostic indicators, including breast estrogen receptor/progesterone receptor (ER/PR) (98.7%) and margin positivity (>93%), maintained high fidelity. On the external TCGA cohort, accuracy was 77.5%, rising to 88.0% after excluding structurally silent fields absent in older narratives. Operationally, the model processed reports in 40–70 s, optimally balancing speed and accuracy. Conclusions: This schema-first abstraction layer successfully decouples clinical logic from specific Artificial Intelligence (AI) models. By reliably transforming narrative reports into machine-readable structures, it establishes a portable privacy-preserving foundation for automated cancer surveillance, institutional data reuse, and future multimodal clinical systems.}, DOI = {10.3390/diagnostics16111644} }

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
Related to Research communities
Cancer Research