Objective This study assesses whether neural networks trained on electronic health record (EHR) data can anticipate what individual clinical orders and existing institutional order set templates clinicians will use more accurately than existing decision support tools. Materials and Methods We process 57,624 patients-worth of clinical event EHR data from 2008-2014. We train a feed-forward neural network (ClinicNet) and logistic regression applied to the traditional problem structure of predicting individual clinical items as well as our proposed workflow of predicting existing institutional order set template usage. Results ClinicNet predicts individual clinical orders (precision=0.32, recall=0.47) better than existing institutional order sets (precision=0.15, recall=0.46). The ClinicNet model predicts clinician usage of existing institutional order sets (avg. precision=0.31) with higher average precision than a baseline of order set usage frequencies (avg. precision=0.20) or a logistic regression model (avg. precision=0.12). Discussion Machine learning methods can predict clinical decision-making patterns with greater accuracy and less manual effort than existing static order set templates. This can streamline existing clinical workflows, but may not fit if historical clinical ordering practices are incorrect. For this reason, manually authored content such as order set templates remain valuable for purposeful design of care pathways. ClinicNet’s capability of predicting such personalized order set templates illustrates the potential of combining both top-down and bottom-up approaches to delivering clinical decision support content. Conclusion ClinicNet illustrates the capability for machine learning methods applied to the EHR to anticipate both individual clinical orders and existing order set templates, which has the potential to improve upon current standards of practice in clinical order entry.

See README file Files that end in .pkl are pickle files, they can be loaded using Python's pickle module (see pickle module documentation). Files that end in .h5 are hdf5 files, they can be loaded using Python's h5py module (see h5py module documentation). Example for loading .h5 files: h5f = h5py.File(filename, 'r') data = h5f['key'].value h5f.close()

See manuscript Methods section.