Rational function model (RFM) faces difficulty in extracting accurate geometric information from remotely sensed images, which is mainly due to the problems of overparameterization and ill-posedness. These problems can be addressed via variable selection methods, in which an optimum subset of rational polynomial coefficients is identified via an optimization algorithm, usually metaheuristic methods [e.g., genetic algorithm and particle swarm optimization (PSO)]. In this letter, we propose a PSO-based method that benefits from a novel cost function. The proposed cost function applies a figure condition analysis, where the sum of estimated errors for the entire image’s pixels is regarded as the cost value. The main advantages of the proposed method, in comparison to its alternatives of the same type, are as follows: 1) in contrast to other metaheuristic-based methods, it can be applied even with a limited number of control points (CPs); 2) since the proposed cost function is a global and continuous one, it yields an appealing RFM from the generalization capability viewpoint (i.e., it leads to satisfying positional accuracies for the entire image, even for those pixels far from CP); and 3) the method is much more stable, which means that it gives very similar results in successive runs. Our experiments, conducted over four real data sets, demonstrate that the proposed method addresses the aforementioned problems, namely, overparameterization and ill-posedness. In addition, it outperforms the typical PSO-based method by 37% on average and also achieves RFM’s structures that are more reliable and more stable than those identified by its alternative.