Abstract Background and study aims Patient safety during a colonoscopy highly depends on endoscopist competence. Endoscopic societies have been calling for an objective and regular assessment of the endoscopists, but existing assessment tools are time-consuming and prone to bias. We aimed to develop and gather evidence of validity for a computerized assessment tool delivering automatic and unbiased assessment of colonoscopy based on 3 dimensional coordinates from the colonoscope. Methods Twenty-four participants were recruited and divided into two groups based on experience: 12 experienced and 12 novices. Participants performed twice on a physical phantom model with a standardized alpha loop in the sigmoid colon. Data was gathered directly from the Olympus ScopeGuide system providing XYZ-coordinates along the length of the colonoscope. Five different motor skill measures were developed based on the data, named: Travel Length, Tip Progression, Chase Efficiency, Shaft movement without tip progression, and Looping. Results The experinced had a lower travel length (P < 0.001), tip progression (P < 0.001), chase efficiency (P = 0.001) and looping (P = 0.006), and a higher shaft movement without tip progression (P < 0.001) reaching the cecum compared with the novices. A composite score was developed based on the five measurements to create a combined score of progression, the 3D-Colonoscopy-Progression-Score (3D-CoPS). The 3D-CoPS revealed a significant difference between groups (experienced: 0.495 (SD 0.303) and novices –0.454 (SD 0.707), P < 0.001). Conclusion This study presents a novel, real-time computerized assessment tool for colonoscopy, and strong evidence of validity was gathered in a simulation-based setting. The system shows promising opportunities for automatic, unbiased and continuous assessment of colonoscopy performance.