Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao Closed Access logo, derived from PLoS Open Access logo. This version with transparent background. http://commons.wikimedia.org/wiki/File:Closed_Access_logo_transparent.svg Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao Journal of Plastic R...arrow_drop_down
image/svg+xml Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao Closed Access logo, derived from PLoS Open Access logo. This version with transparent background. http://commons.wikimedia.org/wiki/File:Closed_Access_logo_transparent.svg Jakob Voss, based on art designer at PLoS, modified by Wikipedia users Nina and Beao
Journal of Plastic Reconstructive & Aesthetic Surgery
Article . 2025 . Peer-reviewed
License: Elsevier TDM
Data sources: Crossref
versions View all 2 versions
addClaim

This Research product is the result of merged Research products in OpenAIRE.

You have already added 0 works in your ORCID record related to the merged Research product.

Evaluating Large Language Model’s accuracy in current procedural terminology coding given operative note templates across various plastic surgery sub-specialties

Authors: Mia J. Carrarini; Hilary Y. Liu; Catherine K. Perez; Francesco M. Egro;

Evaluating Large Language Model’s accuracy in current procedural terminology coding given operative note templates across various plastic surgery sub-specialties

Abstract

Manual CPT coding from operative notes is a time-intensive process that adds to the administrative burden in healthcare. Large Language Models (LLMs) offer a promising solution, but their accuracy in assigning CPT codes based on full operative note templates remains largely untested. Thus, this study evaluates the ability of three LLMs - GPT-4, Gemini, and Copilot - to generate accurate CPT codes from operative note templates across diverse plastic surgery procedures.Twenty-six deidentified operative note templates from six plastic surgery subspecialities were entered into each LLM using a standardized prompt requesting appropriate CPT codes. Model outputs were compared to surgeon-verified codes and categorized as correct (all codes accurate), partially correct (some correct codes with errors), or incorrect (no correct codes). Accuracy was analyzed overall and by subspeciality using Extended Fisher's Exact Tests (significance set at p<0.05).There was a significant difference in overall coding accuracy between the models (p = 0.02176). Gemini and Copilot had the highest accuracy rates (19.2% each), though Copilot produced more partially correct outputs (53.8%). GPT-4 had the lowest accuracy (7.7%). Subspeciality analysis showed Gemini performed best in aesthetic surgery (60%), while Copilot was most accurate in general reconstruction (42.9%). None of the models correctly coded breast reconstruction or craniofacial trauma procedures. Frequent errors included misidentification of procedural details and inappropriate bundling of CPT codes.LLMs show potential for automating CPT coding but currently lack the contextual understanding required for reliable accuracy. Continued human oversight and model refinement are essential for future success of LLM CPT coding.

Related Organizations
  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    1
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
1
Average
Average
Average
Upload OA version
Are you the author of this publication? Upload your Open Access version to Zenodo!
It’s fast and easy, just two clicks!