This paper examines the relationship between household structures, the institutions that shape them and physical and human capital accumulation using household and individual data from China, Indonesia, Côte d'Ivoire and Ghana. Household structures differ greatly across countries and are very diverse within countries. In the two African countries studied a large share of the population live in extended households and/or polygamous ones. Such household structures are the exception or even absent in the Asian cases, where nuclear monogamous households prevail. This paper finds that polygamy is negatively related to capital accumulation. Wealth per capita is significantly lower in polygamous households even after controlling for income, age and literacy of the household head. A first analysis of the possible channels suggests that the larger size of polygamous households plays an important role. A similar result is found for education: enrolment rates are never higher but frequently lower in these households. The diversity across countries demonstrates that polygamy has very different meanings across societies. Extended households are also examined. The analysis shows that those households that accommodate inactive members of the extended kin group are wealthier than other, comparable households. This result is consistent with accommodation of kin group members acting as a vehicle for solidarity that could also be regarded as a private "tax on success". The implicit transfers embedded in such mechanisms, including fostering, are very high compared to monetary and in-kind transfers and have often been overlooked in the analysis of social relations.