In kernel-based machines, the integration of a number of different kernels to build more flexible learning methods is a promising avenue for research. In multiple kernel learning, a compound kernel is build by learning a kernel that is a positively weighted arithmetic mean of several sources. We show in this paper that the only feasible average for kernel learning is precisely the arithmetic average. We investigate general families of averaging processes and how they relate to the development of kernels. Specifically, a number of multivariate and univariate kernels are developed based on the notion of generalized means. These results can be used in more general kernel optimization procedures.