Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ https://doi.org/10.3...arrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
https://doi.org/10.31234/osf.i...
Article . 2025 . Peer-reviewed
License: CC BY
Data sources: Crossref
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
https://doi.org/10.31234/osf.i...
Article . 2025 . Peer-reviewed
License: CC BY
Data sources: Crossref
versions View all 2 versions
addClaim

This Research product is the result of merged Research products in OpenAIRE.

You have already added 0 works in your ORCID record related to the merged Research product.

Implementation Report: Integrating Natural Language Processing Approaches in Child and Adolescent Mental Health Services Research Using Normalisation Process Theory

Authors: Huilin Chen; Jennifer Chapman; En-Nien Tu; Siyu Zhou; Jasmine Laing; Emma Fergusson; Andrey Kormilitzin;

Implementation Report: Integrating Natural Language Processing Approaches in Child and Adolescent Mental Health Services Research Using Normalisation Process Theory

Abstract

Objective: This implementation report documents the integration of an NLP-based data extraction tool within the Oxford Health NHS Foundation Trust’s Child and Adolescent Mental Health Services (CAMHS) to identify and analyse treatment options and outcomes for anxiety in autistic and ADHD youth using electronic health records (EHRs). We evaluate the implementation process through the lens of Normalisation Process Theory (NPT) to understand how the documented interventions were adopted and to identify factors influencing its normalisation into routine research practice.Methods: A multidisciplinary team implemented an NLP pipeline to automatically extract information on presenting problems, treatments, and outcomes (e.g., engagement, clinical outcomes, side effects) from pseudonymised EHR free-text notes. The implementation followed an iterative seven-phase roadmap: (1) defining research questions; (2) identifying key constructs; (3) co-developing an annotation schema; (4) training human annotators; (5) resolving annotation discrepancies; (6) large-scale annotation of EHR data; and (7) NLP model development and evaluation. Throughout, we applied NPT to guide implementation and evaluate progress. Quantitative outcomes (such as volume of data processed, annotation reliability, and NLP model performance) and qualitative feedback were collected to assess implementation success.Results: The NLP intervention was successfully deployed in a real-world clinical research setting, extracted data from 454,127 clinical notes across 5,906 patients. The annotation process achieved a high inter-annotator agreement (Krippendorff’s α = 0.86; 95% CI 0.83–0.88) after iterative schema refinement, and the trained NLP model demonstrated acceptable accuracy in identifying medications (F1-score = 0.9; 95% CI 0.89, 0.91) and anxiety-related information (F1-score = 0.79; 95% CI 0.77, 0.8). The NPT-based evaluation revealed that coherence of a shared understanding of the NLP tool’s purpose and value was established among the team. Cognitive participation was generally strong, clinicians and researchers showed growing interest and involvement in the NLP-enabled research workflow. Collective action was facilitated by clear protocols, though challenges such as variability in data quality and the learning curve for new technology impeded smooth operation at times. Through reflective monitoring, the team identified needs for ongoing training and infrastructure improvements. No major adverse event or harm was reported, though additional time was required for data access and computational setup, which considerably extended the project timeline.

Related Organizations
  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
hybrid
Related to Research communities