An accurate sky polarization field map is a prerequisite for polarization navigation applications. In this article, a detector for sky full-polarization imaging detection is described, the major error-influencing factors (MEIFS ) are obtained, and the error propagation is modeled and analyzed. We reveal the relationship between the error of the inversed Stokes vector and the condition number of the detector matrix, which shows that the error of the inversed Stokes vector is affected by the Stokes vector of the incident light itself and the MEIFS together, with the MEIFS playing a decisive role. With the MEIFS optimized, the impact of detector error on the inversed Stokes vector is attenuated. A control equation for system calibration is also deduced which can establish the connection between the detector matrix design and calibration process. The work in this article provides a reference for optimization and calibration of sky full-polarization imaging detectors.