Diffusion MRI (dMRI) is the unique technique to infer the microstructure of the white matter in vivo and noninvasively, by modeling the diffusion of water molecules. Ensemble Average Propagator (EAP) and Orientation Distribution Function (ODF) are two important Probability Density Functions (PDFs) which reflect the water diffusion. Estimation and processing of EAP and ODF is the central problem in dMRI, and is also the first step for tractography. Diffusion Tensor Imaging (DTI) is the most widely used estimation method which assumes EAP as a Gaussian distribution parameterized by a tensor. Riemannian framework for tensors has been proposed successfully in tensor estimation and processing. However, since the Gaussian EAP assumption is oversimplified, DTI can not reflect complex microstructure like fiber crossing. High Angular Resolution Diffusion Imaging (HARDI) is a category of methods proposed to avoid the limitations of DTI. Most HARDI methods like Q-Ball Imaging (QBI) need some assumptions and only can handle the data from single shell (single b value), which are called as single shell HARDI (sHARDI) methods. However, with the development of scanners and acquisition methods, multiple shell data becomes more and more practical and popular. This thesis focuses on the estimation and processing methods in multiple shell HARDI (mHARDI) which can handle the diffusion data from arbitrary sampling scheme. There are many original contributions in this thesis. First, we develop the analytical Spherical Polar Fourier Imaging (SPFI), which represents the signal using SPF basis and obtains EAP and its various features including ODFs and some scalar indices like Generalized Fractional Anisotropy (GFA) from analytical linear transforms. In the implementation of SPFI, we present two ways for scale estimation and propose to consider the prior E(0)=1 in estimation process. Second, a novel Analytical Fourier Transform in Spherical Coordinate (AFT-SC) framework is proposed to incorporate many sHARDI and mHARDI methods, explore their relation and devise new analytical EAP/ODF estimation methods. Third, we present some important criteria to compare different HARDI methods and illustrate their advantages and limitations. Fourth, we propose a novel diffeomorphism invariant Riemannian framework for ODF and EAP processing, which is a natural generalization of previous Riemannian framework for tensors, and can be used for general PDF computing by representing the square root of the PDF called wavefunction with orthonormal basis. In this Riemannian framework, the exponential map, logarithmic map and geodesic have closed forms, the weighted Riemannian mean and median uniquely exist and can be estimated from an efficient gradient descent. Log-Euclidean framework and Affine-Euclidean framework are developed for fast data processing. Fifth, we theoretically and experimentally compare the Euclidean metric and Riemannian metric for tensors, ODFs and EAPs. Finally, we propose the Geodesic Anisotropy (GA) to measure the anisotropy of EAPs, Square Root Parameterized Estimation (SRPE) for nonnegative definite ODF/EAP estimation, weighted Riemannian mean/median for ODF/EAP interpolation, smoothing, atlas estimation. The concept of reasonable mean value interpolation is presented for interpolation of general PDF data.

Ph.D. thesis jointly from University of Nice Sophia Antipolis and Chinese Academy of Sciences.