Ordinal regression is a supervised learning problem where training samples are labeled by an ordinal scale. The ordering relation and nonmetric property of the label set distinguish it from the multiclass classification and metric regression. To better exploit the inherent structure in the label and benefit from the hidden information in data distribution, we propose a novel ordinal regression model, which is named as nonparallel support vector ordinal regression (NPSVOR) to emphasis the utilization of nonparallel proximal hyperplanes. The new model constructs a hyperplane for each rank such that the patterns of this rank lie in the close proximity while maintaining clear separation with the other ranks. Since the learning of hyperplanes can be carried out independently, NPSVOR can be trained in parallel. Furthermore, we design an efficient solver at the same time for training the hyperplanes in NPSVOR based on the alternating direction method of multipliers. Extensive experimentation demonstrates that NPSVOR yields a large and statistically significant improvement in terms of generalization performance and training speed against nine baselines.