We develop a greedy (pursuit) algorithm for a distributed compressed sensing problem where multiple sensors are connected over a de-centralized network. The algorithm is referred to as distributed parallel pursuit and it solves the distributed compressed sensing problem in two stages; first by a distributed estimation stage and then an information fusion stage. Along with worst case theoretical analysis for the distributed algorithm, we also perform simulation experiments in a controlled manner. We show that the distributed algorithm performs significantly better than the stand-alone (disconnected) solution and close to a centralized (fully connected to a central point) solution.