Galaxies modeled as singular isothermal ellipsoids with an axis ratio distribution similar to the observed axis ratio distribution of E and S0 galaxies are statistically consistent with both the observed numbers of two-image and four-image lenses and the inferred ellipticities of individual lenses. However, no four-image lens is well fit by the model (typical $χ^2/N_{dof} \sim 20$), the axis ratio of the model can be significantly different from that of the observed lens galaxy, and the major axes of the model and the galaxy may be slightly misaligned. We found that models with a second, independent, external shear axis could fit the data well (typical $χ^2/N_{dof} \sim 1$), while adding the same number of extra parameters to the radial mass distribution does not produce such a dramatic improvement in the fit. An independent shear axis can be produced by misalignments between the luminous galaxy and its dark matter halo, or by external shear perturbations due to galaxies and clusters correlated with the primary lens or along the line of sight. We estimate that the external shear perturbations have no significant effect on the expected numbers of two-image and four-image lenses, but that they can be important perturbations in individual lens models. However, the amplitudes of the external shears required to produce the good fits are larger than our estimates for typical external shear perturbations (10-15% shear instead of 1-3% shear) suggesting that the origin of the extra angular structure must be intrinsic to the primary lens galaxy in most cases.

38 pages, 9 figures, submitted to ApJ