The presence of slums in a city is an indicator of poverty and its proper delimitation is a matter of interest for researchers and policy makers. Socio-economic data from surveys and censuses are the primary source of information to identify and quantify slumness within a city or a town. One problem of using survey data for quantifying slumness is that this type of data is usually collected every ten years and is an expensive and time consuming process. Based on the premise that the physical appearance of an urban settlement is a reflection of the society that created it and on the assumption that people living in urban areas with similar physical housing conditions will have similar social and demographic characteristics (Jain, 2008; Taubenb¨ock et al., 2009b); this paper uses data from Medellin City, Colombia, to estimate slum index using solely remote sensing data from an orthorectified, pan-sharpened, natural color Quickbird scene. For Medellin city, the percentage of clay roofs cover and the mean swimming pool density at the analytical region level can explain up to 59% of the variability in the slum index. Structure and texture measures are useful to characterize the differences in the homogeneity of the spatial pattern of the urban layout and they improve the explanatory power of the statistical models when taken into account. When no other information is used, they can explain up to 30% of the variability of the slum index. The results of this research are encouraging and many researchers, urban planners and policy makers could benefit from this rapid and low cost approach to characterize the intra-urban variations of slumness in cities with sparse data or no data at all.