A randomized algorithm for computing a data sparse representation of a given rank structured matrix $A$ (a.k.a. an $H$-matrix) is presented. The algorithm draws on the randomized singular value decomposition (RSVD), and operates under the assumption that algorithms for rapidly applying $A$ and $A^{*}$ to vectors are available. The algorithm analyzes the hierarchical tree that defines the rank structure using graph coloring algorithms to generate a set of random test vectors. The matrix is then applied to the test vectors, and in a final step the matrix itself is reconstructed by the observed input-output pairs. The method presented is an evolution of the "peeling algorithm" of L. Lin, J. Lu, and L. Ying, "Fast construction of hierarchical matrix representation from matrix-vector multiplication," JCP, 230(10), 2011. For the case of uniform trees, the new method substantially reduces the pre-factor of the original peeling algorithm. More significantly, the new technique leads to dramatic acceleration for many non-uniform trees since it constructs sample vectors that are optimized for a given tree. The algorithm is particularly effective for kernel matrices involving a set of points restricted to a lower dimensional object than the ambient space, such as a boundary integral equation defined on a surface in three dimensions.