Video compression technology is an importa nt research part to the intelligent user interface for interactive multimedia system using technologies and services such as image processing, pattern recognition, computer vision and cloud computing service. Recently, high-efficiency video coding (HEVC) has been established as the demand of very high-quality multimedia service like ultrahigh definition video service. High-efficiency video coding (HEVC) standard has three units such as coding unit (CU), prediction unit (PU) and transform unit. It has too many complexities to improve coding performance. We propose a fast algorithm which can be possible to apply for both CU and PU parts. To reduce the computational complexity, we propose CU splitting algorithm based on rate---distortion cost of CU about the parent and current levels to terminate the CU decision early. In terms of PU, we develop fast PU decision based on spatio-temporal and depth correlation for PU level. Finally, experimental results show that our algorithm provides a significant time reduction for encoding with a small loss in video quality, compared to the original HEVC Test Model (HM) version 10.0 software and the previous algorithm.