Abstract An octagonal photonic crystal fiber (O-PCF) structure with eight air-holes on the first ring is proposed based on a unit isosceles triangle. The propagation characteristics and cut-off behaviors of the O-PCF and the standard hexagonal PCF (H-PCF) are numerically investigated by combining the vector boundary method and the effective area method. The phase boundaries for cut-off, single-mode, and multi-mode operations between the O-PCF and H-PCF are calculated and compared. It is found that under the same pitch Λ and air filling fraction (AFF) of the air-holes the O-PCF has significantly wider wavelength range operating in single-mode region, more circular-like field distribution, and less confinement loss than the H-PCF.