Abstract A good knowledge of in situ stresses, including their orientations and magnitudes at depth, is of critical importance in extracting unconventional oil and gas resources. In this paper, an efficient semi-analytical model is developed for accurately predicting the maximum horizontal principal stress magnitude based on the wellbore breakout shapes, which, unlike the vertical and minimum horizontal stresses, cannot be measured directly. In this paper, a semi-analytical model is developed based on the complex variable method to calculate the stresses and displacements around a non-circular wellbore and the Mohr–Coulomb criterion is used to evaluate the failure zones around the wellbore. The non-circular wellbore shape is directly used in the model for stress calculation, and the model requires only the outline coordinates of the hole, the elastic modulus of the material, and the magnitude and direction of the far-field stresses. The stresses obtained from the proposed semi-analytical model based on the complex variable method are compared against those obtained from the commercial finite element package ABAQUS, and the model is in excellent agreement with the finite element model for stress fields, showing its accuracy and efficiency. In addition, the model runs in a few seconds on a laptop computer, thus providing an effective technique for calculating stresses and displacements around a non-circular wellbore. It can be found that the stress distribution around the non-circular wellbore predicted from this model is different from those around a circular wellbore, especially when the wellbore has a large elongation. The proposed method will be useful to study the effect of the wellbore shape on the stress distribution around the wellbore and predict the evolution of wellbore breakout.