AbstractThis paper proposes a novel framework for joint orientation distribution function estimation and tractography based on a new class of tensor kernels. Existing techniques estimate the local fiber orientation at each voxel independently so there is no running knowledge of confidence in the measured signal or estimated fiber orientation. In this work, fiber tracking is formulated as recursive estimation: at each step of tracing the fiber, the current estimate of the orientation distribution function is guided by the previous. To do this, second‐and higher‐order tensor‐based kernels are employed. A weighted mixture of these tensor kernels is used for representing crossing and branching fiber structures. While tracing a fiber, the parameters of the mixture model are estimated based on the orientation distribution function at that location and a smoothness term that penalizes deviation from the previous estimate along the fiber direction. This ensures smooth estimation along the direction of propagation of the fiber.In synthetic experiments, using a mixture of two and three components it is shown that this approach improves the angular resolution at crossings. In vivo experiments using two and three components examine the corpus callosum and corticospinal tract and confirm the ability to trace through regions known to contain such crossing and branching. Magn Reson Med, 2010. © 2010 Wiley‐Liss, Inc.