To operate a bike-sharing system efficiently, system operators need to accurately predict how many bikes are to be rented and returned throughout the city. In this paper, we propose a Hierarchical Consistency Prediction (HCP) model to predict the citywide bike usage in the next period. First, an Adaptive Transition Constraint (AdaTC) clustering algorithm is proposed to cluster stations into groups, making the rent and transition at each cluster more regular than those at each single station. Second, a Similarity-based efficient Gaussian Process Regressor (SGPR) is proposed to respectively predict how many bikes are to be rented at different-scale locations, i.e., at each station, each cluster, and in the entire city. Besides largely improving the training and online prediction efficiency, our regressor considers external impacted factors, addresses the data unbalance issue, and better captures the non-linearity in spatio-temporal data. Third, we design a General Least Square (GLS) formulation to collectively improve those obtained predictions via a mutual reinforcement way. GLS makes the final predictions for rent more reasonable. Considering the causality between rent and return, a Transition based Inference (TINF) method is designed to infer the citywide bike return demand based on the predicted rent demands. Experiments on real-world data are conducted to confirm the effectiveness of our model.