We present BVRI colors of 13 Jovian and 8 Saturnian irregular satellites obtained with the 2.56m Nordic Optical Telescope on La Palma, the 6.5m Magellan Baade Telescope on La Campanas, and the 6m MMT on Mt. Hopkins. The observations were performed between December 2001 to March 2002. Nearly all of the known irregular satellites can be divided into two distinct classes based on their colors. One, the grey color class, has the similar colors to the C-type asteroid, and the other, the light red color class, has colors similar to P/D-type asteroids. We also find at least one object, the Jovian irregular J XXIII Kalyke, that has colors similar to the red colored Centaurs/TNOs, although its classification is unsecure. We also find that there is a correlation between the physical properties and dynamical properties of the irregular satellites. Most of the dynamical clusters have homogeneous colors, which points to single homogeneous progenitors being cratered or fragmented as the source of each individual cluster. The heterogeneous colored clusters are most easily explained by assuming that there are several dynamical clusters in the area, rather than just one.

Submitted to Icarus, 43 pages including 5 figures