We construct and analyze generalized Gaussian quadrature rules for integrands with endpoint singularities or near endpoint singularities. The rules have quadrature points inside the interval of integration, and the weights are all strictly positive. Such rules date back to the study of Chebyshev sets, but their use in applications has only recently been appreciated. We provide error estimates, and we show that the convergence rate is unaffected by the singularity of the integrand. We characterize the quadrature rules in terms of two families of functions that share many properties with orthogonal polynomials but that are orthogonal with respect to a discrete scalar product that, in most cases, is not known a priori.