Artificial intelligence and machine learning (AI-ML) algorithms gave a new direction to the problem of imageclassification. All practical applications are nowadays applying AI-ML algorithms for image classification. Convolutional neural network (CNN) and its varieties rapidly became researcher’s first choice for computer vision related applications. Recently many implementations of hardware accelerators for CNN are reported in literature. The current work exploits the opportunities for improvements and proposes a novel very large scale integrated circuit (VLSI) architecture for classic CNN model for classification of gray-scale images. The proposed architecture is validated using field gate programmable array (FPGA) platform for classification of handwritten digits and hand gestures. The architecture is implemented on both Artix7 and Zynq FPGA board. This work achieves 96% classification accuracy for digits detection and 97% accuracy for gesture images using same CNN model with pre-defined filters in the convolution stage. Proposed architecture consumes less hardware resources compared to state-ofthe- art works by using a single vector multiplication unit (VMU) for both convolution-pooling stage and fully connected network. Architecture supports parallel convolutionand pooling operation and achieves processing speed of ≈16 μs per image frame of size 28×28. Also, the architecture is scalable and supports deep learning where more number of convolution-pooling stages may be used.