For many choices of radial basis kernels, the interpolation problem using shifts of these kernels is always nonsingular, independent of the dimension of the underlying Euclidean space and nearly independent of the geometry of the given data points (some mild geometric conditions are sometimes required). Further, this usually works for any large number of the data points. However, if there are many data points and the interpolation matrices are therefore high-dimensional (and normally not sparse), the numerical problems of solving these linear systems become nontrivial. Not only are the matrices normally not sparse, but they are often ill-conditioned with eigenvalues which tend to zero dramatically fast if any two of the data points become close. Therefore preconditioning is of the essence when the linear systems are solved numerically and in practical applications. This means that suitable bases for the linear spaces spanned by the translates of the radial basis functions must be found. This is the theme of the present paper. Among other subjects, general preconditioning methods are discussed with examples; new bases are defined and their non-negativity established; the problem of scaling is discussed and preconditioning with mean values of the coordinates of the points is proposed as a new method.