Due to structural limitations, the processes of boring holes are performed in a low-rigidity machining system, which predetermines their susceptibility to vibrations. The article is devoted to the study of the process of boring holes on CNC machines, and the subject of the study is the effect of the cutting mode on the stability of the machining. The mathematical model of the machining system is presented in the form of a two-mass dynamic system, which forms a closed loop structure with negative feedback by elastic displacement. In addition, positive feedback is taken into account through the delay argument function, which represents machining along traces. It has been proven that this process provokes the emergence of regenerative oscillations in the machining system. The application of the system’s approach made it possible to obtain a mathematical model in the form of state variables, which is acceptable for the use of numerical modeling methods in both time and frequency space. An applied engineering program for determining the stability diagram in "cutting depth - spindle speed" coordinates has been created. The program uses a new criterion of stability of systems closed through positive feedback loop with a delay argument function. For the first time, the validity of such a criterion was proved for systems described by differential equations of the fourth order. The importance of taking into account the results of the study in the form of a stability lobes diagram when assigning a cutting mode, especially in the area of high speeds, is proven. Thus, according to the results of the experiments, a change in speed of only 7% from 2150 rpm to 2320 rpm with the same cutting depth of 0.4 mm allows the process to become stable. The use of the created program is possible in the system of automatic control of the online cutting mode when the machine is equipped with vibration sensors with appropriate systems for identifying the dynamic parameters of the machining system, which will significantly increase the machining efficiency.