Automatic emotion detection is a prime task in computerized human behaviour analysis. The proposed system is an automatic emotion detection using convolution neural network. The proposed end-to-end CNN is therefore named as ENet. Keeping in mind the computational efficiency, the deep network makes use of trained weight parameters of the MobileNet to initialize the weight parameters of ENet. On top of the last convolution layer of ENet, the authors place global average pooling layer to make it independent of the input image size. The ENet is validated for emotion detection using two benchmark datasets: Cohn-Kanade+ (CK+) and Japanese female facial expression (JAFFE). The experimental results show that the proposed ENet outperforms the other existing methods for emotion detection.