In the presence of astigmatism a focal point typically becomes the well-known interval of Sturm with its pair of axially-separated orthogonal line singularities. The same is true of nodal points except that the issues are more complicated: a nodal point may become a nodal interval with a pair of nodal line singularities, but they are not generally orthogonal, and it is possible for there to be only one line singularity or even none at all. The effect of astigmatism on principal points is the motivation behind this paper. The three classes of cardinal points are defined in the literature in a disjointed fashion. Here a unified approach is adopted, phrased in terms of rays and linear optics, in which focal, nodal and principal points are defined as particular cases of a large class of special structures. The special structures arising in the presence of astigmatism turn out to be described by mathematical expressions of the same form as those that describe nodal structures. As a consequence everything that holds for nodal points, lines and other structures now extends to all other special points as well, including principal points and the lesser-known anti-principal and anti-nodal points. Thus the paper unifies Gauss's and Listing's concepts of cardinal points within a large class of special structures and generalizes them to allow for refracting elements which may be astigmatic and relatively decentred. A numerical example illustrates the calculation of cardinal structures in a model eye with astigmatic and heterocentric refracting surfaces.