In millimeter wave (mmWave) communication systems, existing hybrid beamforming solutions generally require a large number of high-resolution phase shifters (PSs) to realize analog beamformers, which still suffer from high hardware complexity and power consumption. Targeting at this problem, this article introduces a novel hardware-efficient hybrid precoding/combining architecture, which only employs a limited number of simple phase over-samplers (POSs) and a switch (SW) network to achieve maximum hardware efficiency while maintaining satisfactory spectral efficiency performance. The POS can be realized by a simple circuit and simultaneously outputs several parallel signals with different phases. With the aid of a simple switch network, the analog precoder/combiner is implemented by feeding the signals with appropriate phases to antenna arrays or RF chains. We analyze the design challenges of this POS-SW-based hybrid beamforming architecture and present potential solutions to the fundamental issues, especially the precoder/combiner design and the channel estimation strategy. Simulation results demonstrate that this hardware-efficient structure can achieve comparable spectral efficiency but much higher energy efficiency than that of the traditional structures.