This paper presents a flexible approach for calibrating omnidirectional single viewpoint sensors from planar grids. These sensors are increasingly used in robotics where accurate calibration is often a prerequisite. Current approaches in the field are either based on theoretical properties and do not take into account important factors such as misalignment or camera-lens distortion or over-parametrised which leads to minimisation problems that are difficult to solve. Recent techniques based on polynomial approximations lead to impractical calibration methods. Our model is based on an exact theoretical projection function to which we add well identified parameters to model real-world errors. This leads to a full methodology from the initialisation of the intrinsic parameters to the general calibration. We also discuss the validity of the approach for fish-eye and spherical models. An implementation of the method is available as OpenSource software on the author's Web page. We validate the approach with the calibration of parabolic, hyperbolic, folded mirror, wide-angle and spherical sensors.