ABSTRACTBackgroundThe Injury Severity Score (ISS) is a commonly used trauma assessment tool. An accurately calculated ISS is fundamental when used for the classification of the injury severity of trauma patients and subsequent evaluation of a trauma center's performance. This study aimed to analyze the accuracy of a preliminary ISS of trauma patients in the resuscitation room.MethodsA preliminary ISS assessed by clinicians during the primary assessment of trauma patients at the Trauma Center of Rigshospitalet, Denmark in the time period January 2019–May 2024 was recorded in a trauma database and compared with definitive ISS assessed by certified Abbreviated Injury Scale (AIS) coders. Clinicians were not AIS‐certified. All trauma patients were clinically assessed by a trauma team. The primary outcome of the study was the interrater agreement of the preliminary and definitive ISS, evaluated using Cohen's Kappa and a Bland–Altman plot for visual representation. Cases with missing or invalid data were excluded.ResultsIn total, 3623 trauma patients with preliminary and definitive ISS were registered. The majority of trauma patients were adult 2858 (79%), and male 2433 (67%). Penetrating trauma was sustained by 588 (16%) patients while 3032 (84%) suffered blunt trauma. The Cohen's Kappa between the preliminary and the definitive ISS value was 0.51 (95% CI 0.50–0.53), suggesting a moderate overall agreement. The lowest agreement was found in the subgroup of seriously (ISS 15–24) injured patients, 0.31 (95% CI 0.27–0.35). The Bland–Altman plot showed acceptable agreement, although it seemed there was an increasing difference in ISS with increasing mean ISS. No indication of other bias or systematic mistakes was identified.ConclusionThis study found a moderate but overall acceptable level of agreement between preliminary and definitive ISS in trauma patients. In the most severe cases, the preliminary ISS showed a tendency to underestimate injury severity. These findings suggest that the accuracy of preliminary ISS diminishes in cases of severe trauma, highlighting the need for cautious interpretation in critically injured patients. Preliminary ISS remains a valuable tool in clinical settings for trauma severity classification.