In this paper, we present an approach to obstacle avoidance for a group of unmanned vehicles moving in formation. The goal of the group is to move through a partially unknown environment with obstacles and reach a destination while maintaining the formation. We address this problem for a class of dynamic unicycle robots. Using Input-to-State Stability we combine a general class of formation-keeping control schemes with a new dynamic window approach to obstacle avoidance in order to guarantee safety and stability of the formation as well as convergence to the goal position. An important part of the proposed approach can be seen as a formation extension of the configuration space obstacle concept. We illustrate the method with a challenging example.