In this paper, we propose a new set of fractional-order continuous orthogonal moments for image representation. These are called fractional-order Jacobi moments (FrJMs) and are defined from the fractional-order orthogonal Jacobi polynomials. We also propose a method for the fast and precise calculation of FrJMs based on recursive calculations of fractional-order Jacobi polynomials and on the separability property of FrJMs. Then, we will derive invariants of FrJMs with respect to rotation, scale and translation (RST) in order to apply them for classification tasks. Just as important, we have presented a systematic parameter selection method for finding the optimal fractional parameter values with respect to pattern recognition applications. Finally, an experimental and comparative study was carried out to test the capacity of FrJMs for the image reconstruction, extraction of global and local image characteristics, invariance to RST, sensitivity to noise, ability to recognize similar grayscale images and the computational times of the new descriptors. The proposed descriptors outperformed the recent orthogonal moments with fractional orders.