Convolutional Neural Networks (CNNs) are widely used deep learning models for solving various tasks such as computer vision, speech recognition, among others. However, CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time-consuming and challenging. To address these issues, this study proposes an Improved Differential Evolution of Convolutional Neural Network algorithm, namely IDECNN, to design CNN layer architectures for image classification task.