High structural susceptibility of multiferroic BiFeO3 (BFO) makes it a potential replacement of current Pb-based piezoelectrics. In this study, a tetragonal phase is identified based on a combination of x-ray diffraction, scanning transmission electronic microscopy, x-ray absorption spectroscopy, and Raman spectroscopy when BFO is grown on yttria-stabilized zirconia (YSZ) substrates. To distinguish the discrepancy between this tetragonal phase and common cases of monoclinic BFO, piezoelectric force microscopy images and optical property are also performed. It shows a lower electrostatic energy of ferroelectric domains and a large reduction of band gap for BFO grown on YSZ substrate comparing to the well-known one grown on LaAlO3 substrate. Our findings in this work can provide more insights to understand the structural diversity of multiferroic BFO system for further applications.