The effectiveness of biometric authentication based on face mainly depends on the method used to locate the face in the image or video. This paper presents a hybrid system for faces detection, in a color image or video, in unconstrained cases, i.e. situations in which illumination, pose, occlusion and size of the face are uncontrolled. To do this, the new method of detection proposed in this system is based primarily on a technique of automatic learning by using the decision of three neural networks, a new method of feature extraction based on the principal of energy compaction in the DC coefficient using the discrete cosine transform and a technique of segmentation by skin color to reduce the space of research and to accelerate the process of detection. A whole of pictures (faces and no faces) are transformed to vectors of data which will be used for entrain the neural networks to separate between the two classes while the discrete cosine transform is used to reduce the dimension of the vectors, to eliminate the redundancies of information, and to store only the useful information in a minimum number of coefficients. The experimental results have showed that this hybridization of methods will gave a very significant improvement of the rate of the recognition, quality of detection and the time of execution.