Nepali Sign language is used by the differently able Nepali speaking community of Sikkim, hilly regions of North Bengal, some parts of Uttrakhand, Meghalaya and Assam in India and the major part of Nepal. This dataset is the first Nepali Sign language dataset prepared to represent consonant and vowel signs in the form of videos. Dataset was collected with the help of 14 volunteers, categorized as 5 Native (>25 years and >5 years of using Nepali Sign Language) and 9 beginners learning Nepali Sign Language. Dataset has 630 total videos containing 1205 gestures for 36 consonants and 13 vowels of Nepali Sign Language out of which 516 Videos is for consonant that has 972 gestures and 114 videos for vowel that has 234 gestures. Data was collected in two types of environment Prepared environment - Black background by putting a black chart paper at the background of the wall where the gesture is performed. Unprepared environment- No black chart paper was used to change the background while performing the gesture. Real world environment indoor and outdoor has also been considered. Lighting condition – Natural lighting condition has been considered while capturing the data at indoor and outdoor i.e no extra light was focused on the volunteer performing the gesture. Bright and Dark condition for indoor videos are referring the videos captured by turning the room lights and off respectively.