Linkages between albedo, surface morphology, melt pond distribution, and properties of first‐year and multiyear sea ice have been studied at two field sites in the North American Arctic between 1998 and 2001. It is shown that summer sea‐ice albedo depends critically on surface melt‐pond hydrology, controlled by melt rate, ice permeability, and topography. Remarkable short‐term and interannual variability in pond fraction varying by more than a factor of 2 and hence area‐averaged albedo (varying between 0.28 and 0.49 over the period of a few days) were observed to be forced by millimeter to centimeter changes in pond water level. Tracer studies show that the depth of the snow cover, by controlling the amount of superimposed ice formation in early summer, critically affects the retention of meltwater at the ice surface and hence affects pond coverage. Ice roughness as determined by deformation and aging processes explains a significant portion of the contrasts in pond coverage and albedo between ice of different ages, suggesting that a reduction in multiyear ice area and sea‐ice residence time in the Arctic Ocean is accompanied by large‐scale ice albedo decreases. Our work indicates that ice‐albedo prediction in large‐scale models with conventional methods is inherently difficult, if not impossible. However, a hydrological model, incorporating measured statistics of ice topography, reproduces observed pond features and variability, pointing toward an alternative approach in predicting ice albedo in numerical simulations.