Hilbert-Huang Transform (HHT) is a kind of signal analysis method suitable for non-stationary signal, and the combination of HHT and vector signal processing is called vector HHT, by which the instantaneous azimuth of the target can be estimated. In vector HHT signal processing method, the existence of intermittent signal and noise leads to mode mixing in the results of empirical mode decomposition (EMD). Mode mixing influences exact extraction of vibration mode, which causes that the signal which should be in the same order of intrinsic mode function (IMF) distributes in different IMFs. In vector HHT signal processing method, the two vibration speed components need to be multiplied by sound pressure respectively, so mode mixing will lead to great error in the estimation of azimuth of the target. To overcome the influence of mode mixing on vector HHT, this paper proposes a kind of vector HHT signal processing method based on scale filtering. The new method can effectively eliminate mode mixing resulting from intermittent signal and noise, and exact decomposition results can be achieved. The improved vector HHT method is used in processing acoustic vector signal, and the data obtained from a certain sea experiment in Dalian city is analyzed by the new method. The waterfall plot accurately shows the azimuth of the target ship, which proves the validity of the method.