Birefringence imaging, including polarization sensitive optical coherence tomography (PS-OCT), can provide valuable insight into the microscopic structure and organization of many biological tissues. In this paper, we report on a method to fabricate tissue-like birefringence phantoms for such imaging modalities. We utilize the photo-elastic effect, wherein birefringence is induced by stretching a polymer sample after heating it above its glass-transition temperature. The cooled samples stably exhibit homogeneous birefringence, and were assembled into phantoms containing multiple well-defined regions of distinct birefringence. We present planar slab phantoms for microscopy applications and cylindrical phantoms for catheter-based imaging and demonstrate quantitative analysis of the birefringence within individual regions of interest. Birefringence phantoms enable testing, validating, calibrating, and improving PS-OCT acquisition systems and reconstruction strategies.