Current level-2 condition assessment methods for critical infrastructure assets mostly rely on human visual investigation of visible damages and patterns at the structure surface, which can be a costly, time-consuming, and subjective exercise in reality. In this article, a novel method for crack detection is proposed via salient structure extraction from textured background. This method first extracts strong edges and distinguishes them from strong textures in a local neighborhood. Then, the spatial distribution of texture features is estimated to detect cracks as salient structures that are not widely spread across the whole image. The outputs from these two key steps are fused to calculate the final structure saliency map for generation of the crack masks. This method was validated on a data set with 704 images and the outcome revealed an average f-measure of 75% in detecting the concrete cracks that is significantly higher than two other baseline methods.