Abstract A good physician-patient relationship is essential for the delivery of quality health care and is associated with better health outcomes. This study explored the association between patients' object relations and the physician-patient relationship. Primary care patients (n = 72) and physicians (n = 21) participated in the study. Patients' early memories were obtained prior to their medical visit and were coded with the Social Cognition and Object Relations–Global Method (SCORS-G). Patients and physicians independently rated the quality of the physician-patient encounter. Results indicated that object relations (SCORS-G Cognitive-structural and Self factors) incrementally predicted physician-patient relationship ratings, as rated by the patient, above and beyond psychological distress, somatization, and level of physician training. Patient education, psychological distress, and level of physician training predicted physician ratings of the medical encounter. The authors discuss the discrepancy between physician and patient ratings as they relate to patients' object relations.