The automatic segmentation of the left ventricle in magnetic resonance (MR) images is the basis of computer-aided diagnosis systems. To accurately extract the endocardium and epicardium of the left ventricle from MR images, a method based on a dilated dense convolutional network (DDCN) has been proposed in this article. First, to reduce memory consumption, computing time and the class imbalance between the target and background, a clustering algorithm that combines the prior knowledge of the spatial relationship between the slices has been proposed to crop the region of interest (ROI). Then, the DDCN model with 8 dilated convolutional layers and dense connections, which is efficient with respect to its memory consumption and training time, has been proposed to delineate the endocardium and epicardium. To compare the DDCN model with other algorithms, 30 sequences of the MICCAI 2009 left ventricle segmentation challenge database are used to train the proposed model and the other 15 sequences are used for testing. The performance of the proposed method is evaluated by the percentage of “good” contours (PGC), average Dice metric (ADM) and average perpendicular distance (APD). Our results show that for the endocardial and epicardial contours, the PGCs are 99.49%±1.99% and 100%±0%, the APDs are 1.50±0.34 mm and 1.31±0.22 mm, and the ADMs are 0.93±0.03 and 0.96±0.01, respectively, which indicates that our method provides contours with great agreement with the ground truth. In addition, the comparison results show that our method exhibits outstanding performance and possesses promising potential to be used in computer-aided diagnosis systems for cardiovascular disease.