Multiscale wavelet-based representation of data has been shown to be a powerful tool in feature extraction from practical process data. In this paper, this characteristic of multiscale representation is utilized to improve the prediction accuracy of the popular linear auto-regressive with exogenous variable (ARX) model by developing a multiscale ARX (MSARX) modeling algorithm. The idea is to decompose the input-output data, construct multiple ARX models at multiple scales using the scaled signal approximations of the data, and then using cross validation, select among all MSARX models the one which best describes the process. Also, the MSARX modeling algorithm is shown to improve the parsimony of the estimated models, as ARX models with a fewer number of coefficients are needed at coarser scales. This advantage is attributed to the down-sampling used in multiscale decomposition of data. The main advantage of the MSARX algorithm is that it inherently accounts for the presence of noise in the data by the application of low pass filters used in the decomposition of the input-output data, which in turn improves the model robustness to measurement noise in the data and thus enhances its prediction. These prediction and parsimony advantages of the developed MSARX modeling algorithm are demonstrated using a simulated second order process.