Accurate detection of anomalies in multivariate time series data has attracted much attention due to its importance in a wide range of applications. Since it is difficult to obtain accurately labeled data, many unsupervised anomaly detection algorithms for multivariate time series data have been developed. However, building such a system is challenging since it requires capturing temporal dependencies in each time series and must also encode the inter-correlations between different pairs of time series. To meet this challenge, we propose a Multi Scale Convolutional Variational Autoencoder (MSCVAE) to detect anomalies in multivariate time series data. Firstly, multi scale attribute matrices are constructed from multivariate time series to characterize multiple levels of the system states at different time steps. Then, given the attribute matrices, a convolutional variational autoencoder is employed to generate reconstructed attribute matrices, and also an attention-based ConvLSTM network is used to capture the temporal patterns. In addition, a new ERR-based threshold setting strategy is developed to optimize anomaly detection performance instead of relying on the traditional ROC-based threshold setting strategy with an imbalanced dataset. Finally, the proposed framework is assessed by means of experiments on four datasets. The experimental results show that our proposed framework is superior to competing algorithms in terms of model performance and robustness, demonstrating that our model is effective in detecting anomalies in multivariate time series.