Protein S and C4b-binding protein (C4BP) form a tight complex (Kd approximately 0.6 nM) the physiologic purpose of which is unknown. The participation of protein S in this complex was investigated using site-specific mutagenesis. Normal recombinant human protein S (rHPS) and five specifically mutated protein S analogs were expressed in transformed human kidney 293 cells and the following properties were characterized: solution-phase C4BP binding, ability to be cleaved by thrombin, ability to act as a cofactor in the activated protein C-catalyzed inactivation of factor Va, and gamma-carboxyglutamic acid content. In some cases, beta-hydroxyaspartic acid plus beta-hydroxyasparagine content was also determined. Binding studies indicated that while clearly important for a high affinity interaction, the amino acid sequence Gly605-Ile614 identified by Walker (Walker, F J. (1989) J. Biol. Chem. 264, 17645-17648) does not account for all the binding energy of the HPS-C4BP interaction. All mutants perturbed in this region or lacking it altogether displayed reduced C4BP binding, and some retained anticoagulant cofactor function. Neither human factor X nor human steroid-binding protein had any measurable ability to compete with plasma HPS for C4BP binding. Furthermore, bovine protein S and a rHPS analog with bovine sequence from Gly597-Trp629 bound to human C4BP with the same affinity as did HPS, and both proteins substituted effectively for HPS as a cofactor for activated protein C in an otherwise human anticoagulation system. Together these results suggest that optimal binding of protein S to C4BP requires the putative alpha-helix Gly605-Ile614, as well as other undetermined regions of protein S, and that the regions of HPS responsible for C4BP binding and activated protein C cofactor function are structurally isolated.