This paper presents a lip localization based visual feature extraction method to segment lip region from image or video in real time. Lip localization and tracking is useful in many applications such as lip reading, lip synchronization, visual speech recognition, facial animation etc. To synchronize lip movements with input audio we need to first segment lip region from input image or video frame. This paper presents a novel color based approach for localization of lips, which is an early stage for tracking lips in real time. A phoneme is a basic unit of speech and a viseme is a visual representation of phoneme or shape of mouth while utterance of particular phoneme. The main goal of our work is to implement a system for synchronizing lips with the input speech. To extract visual features i.e. visemes from input video frame or image we have used HSV and YCbCr color model along with various morphological operations. We have developed algorithm to work with normal lighting conditions and natural facial images of female and male.