AbstractUrban facades regularly contain interesting variations due to allowed deformations of repeated elements (e.g., windows in different open or close positions) posing challenges to state‐of‐the‐art facade analysis algorithms. We propose a semi‐automatic framework to recover both repetition patterns of the elements and their individual deformation parameters to produce a factored facade representation. Such a representation enables a range of applications including interactive facade images, improved multi‐view stereo reconstruction, facade‐level change detection, and novel image editing possibilities.