In this paper we presented an intangible hand gesture based computer mouse control system. Our approach uses a novel skin color segmentation technique to control mouse movement. The system uses morphological operations like structuring elements and blob counting. Our system can remove other skin like objects from the background. One of the key features of the system is its simplicity. We have successfully tested our system for an intangible interface between human hand and computer mouse without any complex processing. Various mouse operations like cursor movements, right click and left click have been performed. The system has been implemented using webcam and MATLAB.