Purpose: The intensity modulated radiation therapy (IMRT) patient‐specific quality assurance (QA) (referred to as QA in this paper for simplicity) process is a time and resource intensive effort in every clinic. The use of a global QA tolerance criterion for all treatment sites may be too tight for some complex sites increasing false negatives and rejections of QA measurements which typically results in wasted efforts, treatment delays, and decreased efficiency. At the same time, other sites requiring a less complex plan might have a high false positive leading to approvals of QA measurements that actually need to be rejected. This work is an effort to adopt statistical tools to1. develop a tool to identify statistical variations in the process, monitor trends, detect outliers, and proactively identify drifts in the overall QA results;2. analyze the results of the QA process, identify similarities and differences between treatment plans of different treatment sites, and evaluate the possibility of site‐specific tolerance levels for QA approval tolerances.Methods: The analysis was performed for QA measurements made using two ion chamber points. A custom software tool was developed for data processing and analysis. This tool facilitated QA data collection, retrieval, visualization, real‐time feedback, and advanced statistical analysis of the data. Statistical techniques based on analysis of variance were used to evaluate the need for site‐specific tolerances and statistical process control was used to study statistical variations in the process.Results: A retrospective analysis of the QA process variability was performed in order to identify site‐specific tolerances for the QA measurements and to reduce false positive and false negative QA results. From the data, it can be seen that the treatment sites are significantly different and need site‐specific tolerance levels for QA approval. The in‐house developed tool was used to further monitor the QA process using individual (I), standard deviations (S), and exponentially weighted moving averages charts for process variability studies.Conclusions: The authors have studied the analysis of variance on ion chamber measurements made for IMRT treatment plans on different sites, identified similarities and differences between different sites, and thereby evaluated the need for site‐specific tolerances for QA acceptance policy. The authors have proposed a way to calculate the appropriate tolerances for different treatment sites and illustrated the clinical usage. Variability at each step of the process increases the uncertainty in the process. The authors have explained the different approaches taken to reduce the variability at each step of the entire process. This process can be used for the benefit during/as part of an IMRT commissioning process in any clinic. The authors have also developed a tool to automate the process of data collection, analysis, and monitoring the process quality via standard deviations and EWMA charts.