Based on K-means and a two-layer pyramid structure, a fast algorithm is proposed for color image segmentation. The algorithm employs two strategies. Firstly, a two-layer structure of a color image is established. Then, an improved K-means with integer based lookup table implementation is applied to each layer. The clustering result on the upper layer (lower resolution) is used to guide the clustering in the lower layer (higher resolution). Experiments have shown that the proposed algorithm is significantly faster than the original K-means algorithm while producing comparable segmentation results.