In this paper we address the problem of identifying which of various possible spatial residue-residue neighbor pairs are plausible physical contacts without reference to the native structure side chain geometry. We propose an algorithm that eliminates most of the implausible physical contacts from the fold models. This algorithm exploits the correlations between the amino acid side chain rotamers and the direction of the physical contacts between the amino acid side chains. We use this algorithm to "filter" the score of the sequence-to-structure alignment. Filtering is dynamic, in the sense that the set of neighbor pairs contributing to the alignment score varies during threading. Whether or not a neighbor pair contributes to the score depends on the threaded amino acids. This score filtering improves the accuracy of the predicted sequence-to-structure alignment.