Drill string vibration is a widely studied topic. This paper developed a real-time measurement system near the drilling bit and extracted the lateral vibration, longitudinal vibration time series of bottom rotating drill string. In order to reconstruct the phase space, we estimated the delay time with mutual information and calculated the embedding dimension through Cao’s method. Finally, the chaotic characterization of the system is analyzed by calculating the correlation dimension and the largest Lyapunov exponent. The results show that such system can exhibit positive finite-time Lyapunov exponents and a clear convergence toward the correlation dimension, which is a strong indicator for the chaotic behavior of the system. It is expected that the new dynamics found in this paper could be of potential implication to the control methods of the drill string vibration.