Sign language is a unique type of communication language which is essential for bridging the commuication gap between deaf and dumb people. In each sign language, there are various signs with variations in palm size, shape and motion and placement of hand which plays a major in each sign. A large number of applications have been put forward by various researchers. In the past few years, in these applications many remarkable changes have been made using deep learning concepts. Throughout this survey, we analysed these applications of hand gesture recognition using deep learning concepts from the last few years. Although there were many notable improvements in the accuracy in hand gesture recognition, there are still many complications that needs to be resolved. We put forward a taxonomy to clasify the proposed apllications for future lines of research in the field. Our objective is to develop an application that can recognize hand gestures and signs. We will train that model in a way that sign language will be converted into text and audio. This will help people communicate with people who are deaf and blind. The application will recognize hand gestures by comparing the input with pre-existing datasets formed using the American sign Language. Here the input will be in the form of a real-time video of hand signals of sign language. We will convert those signs into text as well as audio as output for users to recognize the signs which are captured by camera and presented by the sign language speaker.