AbstractThe influence of scour on the vertical response of rock‐socketed piles in a layered saturated rock–soil mass is studied through the finite element‐boundary element coupling method. By utilizing the theoretical solution of layered saturated media as the kernel function, the boundary element method (BEM) is used to deduce the boundary integral equations of rock–soil mass. Meanwhile, the pile is modelled with the three‐node bar element and the global stiffness matrix of the pile is assembled by the finite element method (FEM). Then according to the scour type, the flexibility matrix of the foundation is recombined and extended, which is consistent with the stiffness matrix dimension of the pile. In consideration of the force balance and continuity conditions at the pile‐soil‐rock interface, the FEM–BEM coupling equation is established to solve the interaction between layered rock–soil mass and rock‐socketed pile that considers different scour types. A corresponding MATLAB program is compiled to verify the correctness of the present theory by comparisons with the existing literature solutions, field tests and the finite element solutions. The influences of the scour type, scour depth, socketed length as well as pile‐rock mass modulus ratio and rock mass stratification on the vertical response of a rock‐socketed single pile are further discussed through several numerical examples.