A robust hyper spectral unmixing algorithm that finds multiple sets of end members is introduced. The algorithm, called Robust Context Dependent Spectral Unmixing (RCDSU), combines the advantages of context dependent unmixing and robust clustering. RCDSU adapts the unmixing to different regions, or contexts, of the spectral space. It combines fuzzy and possibilistic clustering and linear unmixing to learn multiple contexts and the optimal end members and abundances for each context. RCDSU uses fuzzy membership functions to partition the data, and possibilistic membership functions to identify noise and outliers. An extension of RCDSU to deal with the case of an unknown number of contexts is also proposed. The performance of the proposed work is evaluated using simulated and real hyper spectral data. The experiments show that the proposed methods can handle noisy data and identify an "optimal" number of contexts and appropriate end members within each context.