Acoustic manipulation based on acoustic radiation force (ARF) is an important structure manipulation method, which has a wide application prospect in particle manipulation. The calculation of ARF received by regular particles has been extensively studied. In this paper, the scheme of ARF acting on general structure is proposed. The analytical relationship between the ARF and the velocity potential distribution of the scattering on a specified envelope surface is established directly. The irregular surface integration is extended to a custom surface, which is easy to be processed. The results of the analytical solution and the finite element simulation are given to verify its simplicity and accuracy. This solution has a significant improvement compared with the traditional calculation methods based on definitions, especially avoiding the integration upon complex surfaces. The proposal of this complete scheme makes it possible to calculate the analytical ARF acting on structure with any surface description and lays a complete theoretical foundation for the wide application of ARF in structure and particle manipulation.