Many hyperspectral sensors collect data using multiple spectrometers to span a broad spectral range. These spectrometers can be fed by optical fibers on the same or separate focal planes. The Modular Imaging Spectrometer Instrument (MISI), a 70 band line scanner built by the Rochester Institute of Technology, is configured in this manner. Visible and near infrared spectrometers at the primary focal plane are each fed by their own optical fiber. The spatial offset between the two fibers on the focal plane causes an inherent misregistration between the two sets of spectral bands. This configuration causes a relatively complicated misregistration which cannot be corrected with a simple shift or rotation of the data. This mismatch between spectral channels has detrimental effects on the spectral purity of each pixel, especially when dealing with the application of sub-pixel target detection. A geometric model of the sensor has been developed to solve for the misregistration and achieve image rectification. This paper addresses the issues in dealing with the misregistration and techniques used to improve spectral purity on a per pixel basis.