This paper deals with a multilevel construction of hierarchical matrix approximations to the inverses of finite element stiffness matrices. Given a sequence of discretizations Aℓxℓ = f ℓ, ℓ = 0,..., n, where A0x0 = f 0 denotes the coarse grid problem, we will compute A0-1 exactly and then use interpolation to obtain an H-matrix approximation Aℓ+1-H from the approximate H-matrix inverse Aℓ-H on the next coarser grid. We develop an exact interpolation scheme for the inverse of tridiagonal matrices as they appear in the finite element discretization of one-dimensional differential equations. We then generalize this approach to two spatial dimensions where these efficiently computed approximations to the inverse may serve as preconditioners in iterative solution methods. We illustrate this approach with some numerical tests for convection-dominated convection-diffusion problems. © 2006 Society for Industrial and Applied Mathematics.