While the importance of institutions for explaining cross-country income differences is widely recognized, comparatively little is known about the origins of economic institutions. One strand of the literature emphasizes cultural differences while another points at exogenous environmental factors such as mortality and climate. Both are supported by some empirical evidence. I reconcile the two schools of institutional origins by proposing a theory of self-selection of colonists to different geographic destinations. Exogenous characteristics such as climate, mortality and factor differences determine which type of settler decides to move to a particular colony. Settler type, in turn, shapes the institutional quality of the new country. The model is used to confirm observed regularities reported by previous researchers. Furthermore, robust new evidence is presented in support of this selection process. The results suggest that any theory of colonial development that does not take selection into account will be incomplete.