Background: The role of epicardial adipose tissue (EAT) in heart failure with preserved ejection fraction (HFpEF) remains to be defined. Methods: A consecutive series of outpatients with chronic heart failure—heart failure with reduced ejection fraction (HFrEF) and HFpEF and/or diastolic dysfunction—had EAT assessed by echocardiographic measurement and related to indices of cardiac structure and function. Results: Epicardial fat thickness was significantly (p < 0.05) greater in HFpEF (N = 141) with a mean of 6.7 ± 1.6 mm compared with a mean of 5.1 ± 1.0 mm in HFrEF (n = 40). After adjusting for the relationship with BMI, in HFpEF, epicardial fat was significantly (p < 0.05) negatively correlated with left ventricular internal diameter end diastole (LVIDd), left ventricular internal diameter end systole (LVIDs), left ventricular (LV) end-diastolic volume (EDV) index, lateral e’, septal e’, right atrial (RA) volume index, and hemoglobin (Hgb). The association with Hgb was no longer significant after adjusting for the effect of age. HFpEF was associated with smaller LVIDd, LVIDs, LV EDV indexes, and left atrial (LA) and RA volume indexes. Conclusions: Epicardial fat is significantly (p < 0.05) greater in HFpEF than HFrEF. Epicardial fat is associated with smaller cardiac chamber sizes in HFpEF suggesting that epicardial fat acts as a constraint to cardiac dilation.