ABSTRACT Finding a balance between the time‐to‐market and quality of a delivered product is a daunting task. The optimal release moment is not easily found. We propose to use historical project data to monitor the progress of running projects. From the data we inferred a formula providing a rough indication of the number of defects given the effort spent thus far (effort‐to‐defect formula). Furthermore, we provide a worst case bound to the allowed number of residual defects at the end of a project in order to achieve the required level of quality. For this purpose we slightly modified a well‐known reliability growth model by Bishop and Bloomfield. It turned out that the software in Philips’ MRI scanners has a defect rate of 1 per 1175 device‐years of observation. This coincides with the second highest safety integrity level ( SIL3 ) as defined in the IEC 61508 standard. The highest level ( SIL4 ) is only attainable by applying redundancy. Finally, we combine the effort‐to‐defect formula with the reliability growth model to monitor the progress of a project and to determine when the required level of quality will be reached. We show that a common fault distribution, the Rayleigh model, is not necessarily the best model for predicting the number of residual defects in the system. Using a well‐known data analysis approach called exploratory data analysis ( EDA ) we obtained an alternative model based on the Normal curve. We have evaluated the Rayleigh model and our model based on the Normal curve at Philips Healthcare MRI . The Normal curve predicts defects over time better than the Rayleigh model in the case of Philips Healthcare MRI . Furthermore, time series models ( ARIMA ) are also useful for accurately describing the defect trend, but are not suitable for long‐term predictions. Finally, cost estimation models ( COCOMO ) lack the predictive capabilities of models fitted on the data using EDA . Their capabilities are limited compared to a model derived from data which reflects the constitutional knowledge of the actual realization of systems within a specific company. However, they can still be used to advantage when there is limited or no data available to use as a basis for lifting from gut feel to order of magnitude. © 2013 Wiley Periodicals, Inc. Syst Eng 17