Road anomaly detection has attracting increasing attention in recent years due to its significant role in the public transportation of modern cities. A few methods has been proposed to detect road anomaly with inertial sensors (e.g., accelerometer and gyroscope), which usually utilize classification techniques by extracting time and frequency domain features from inertial sensor data. However, existing methods are time consuming since these methods perform on the whole datasets. In addition, few of them pay attention to the self-similarity of the data when vehicle passes over the road anomalies. In this paper, we propose QDetect, a road anomaly detection system with less data-dependency via querying and re-comparing. Specifically, QDetect consists of two phases: 1) Query filter. This phase is designed to roughly extract road anomaly segments by matching existing labelled anomalies; 2) Re-comparison on suspicious anomalies to identify their anomaly types. We have conducted comprehensive experiments on two real-world data sets, and the results show that our method outperforms some existing methods in both detection performance and running time. We expect to lay the first step to some new thoughts to the field of real-time road anomalies detection in subsequent work.