We address the problem of contour inference from partial data, as obtained from state-of-the-art edge detectors. We argue that in order to obtain more pereeptually salient contours, it is necessary to impose generic constraints such as continuity and co-curvilinearity. The implementation is in the form of a convolution with a mask which encodes both the orientation and the strength of the possible continuations. We first show how the mask, called the “Extension field” is derived, then how the contributions from different sites are collected to produce a saliency map. We show that the scheme can handle a variety of input data, from dot patterns to oriented edgels in a unified manner, and demonstrate results on a variety of input stimuli. We also present a similar approach to the problem of inferring contours formed by end points. In both cases, the scheme is non-linear, non iterative, and unified in the sense that all types of input tokens are handled in the same manner.