Controlling the capacity of a learning system in a way that does not depend on the dimensionality of the hypothesis space provides the key for effectively using large neural networks and decision trees, ensemble methods and kernel-induced feature spaces. This extended abstract will provide an overview of recent work in this direction, based on the concepts of margin and margin distribution.