The approaches based on background difference are the most used with fixed cameras to perform motion detection, because of the high quality of the segmentation achieved. However, real time requirements prevent most of the algorithms proposed in literature to exploit the background difference with Pan-Tilt-Zoom (PTZ) cameras in real world applications. Nevertheless, using color information to detect motion yields a sensible improvement in terms of accuracy of segmented masks, by helping reducing camouflage and detecting shadows. To our knowledge, the algorithm we have conceived is the first to exploit color information and perform real time alignment and background difference based on background mosaicing, with PTZ cameras. In addition, no prior information regarding scene or camera parameters have been used for either spatial or tonal alignment. Accurate experiments performed on indoor and outdoor sequences allow to assess both quality and performance of the method we devised.