In this work, we propose a novel architecture for building a robust integrated photonic neuromorphic accelerator based on a crossbar array design. Our architecture is based on an asymmetric multimode Y-coupler. A Y-coupler has the inherent benefit of high fabrication tolerance and broad optical bandwidth. From simulations, we show that our proposed Y-coupler has high coupling efficiency. Using modal decomposition analysis of our coupler, we numerically estimate the energy efficiency performance of a large-scale photonic network and show a 10% improvement in energy efficiency for large-scale photonic networks with high fabrication tolerance and broadband application and a small footprint.