Color values in an image are related to image irradiance by a nonlinear function called radiometric response function. Since this function depends on the aperture and the shutter speed, image intensity of a same object may vary during the acquisition of an image sequence due to auto exposure feature of the camera. While this is desirable to make optimal use of the limited dynamic range of most cameras, this causes problems for a number of applications in computer vision. In this paper we propose a method for estimating the radiometric response function and apply it to radiometrically align images so that the color values are consistent for all images of a sequence. Our approach computes the response function, exposure and white balance changes between images (up to some ambiguity) for a moving camera without any prior knowledge about exposures. We show the performance of our algorithm by estimating the response function from synthetic images and also from real world data, using it to radiometrically align the images.