Organizations rely on crowds for a variety of reasons, e.g. in order to evaluate (Amazon), create content (Threadless) but also to solve given problems (InnoCentive and OpenIDEO). Several studies have examined how to organize problem‐solving activities. However, most papers have examined the crowdsourcing process using a partial perspective and a wide‐ranging outlook has been missing. This study uses a computer‐based simulation model and anecdotal case studies of InnoCentive and OpenIDEO, in order to study the underlying drivers of collective problem solving behavior. Results suggest that dynamics between the number of users, number of iterations and different selection mechanisms impact the ability to find an optimal solution to the given problem.