Sign Language is a language in which we make use of hand movements and gestures to communicate with people who are mainly deaf and dumb. This paper proposes a system to recognize the hand gestures using a Deep Learning Algorithm, Convolution Neural Network (CNN) to process the image and predict the gestures. This paper shows the sign language recognition of 26 alphabets and 0-9 digits hand gestures of American Sign Language. The proposed system contains modules such as pre-processing and feature extraction, training and testing of model and sign to text conversion. Different CNN architecture and pre-processing techniques such as greyscale, thresholding, skin masking, and Canny Edge Detection were designed and tested with our dataset to obtain better accuracy in recognition)