The need for movement smoothness quantification to assess motor learning and recovery has resulted in various measures that look at different aspects of a movement's profile. This paper first shows that most of the previously published smoothness measures lack validity, consistency, sensitivity, or robustness. It then introduces and evaluates the spectral arc-length metric that uses a movement speed profile's Fourier magnitude spectrum to quantify movement smoothness. This new metric is systematically tested and compared to other smoothness metrics, using experimental data from stroke and healthy subjects as well as simulated movement data. The results indicate that the spectral arc-length metric is a valid and consistent measure of movement smoothness, which is both sensitive to modifications in motor behavior and robust to measurement noise. We hope that the systematic analysis of this paper is a step toward the standardization of the quantitative assessment of movement smoothness.