Abstract: As the resource density of cloud servers increases, cloud providers deploy hundreds of VMs concurrently on a single server, requiring a high-performance, scalable, flexible and high-density I/O virtualization method. Hardware assisted virtualization such as device pass-through with SR-IOV can achieve near-native performance, however, at the expense of flexibility and a limited device count. Traditional software-based I/O virtualization systems tend to dedicate additional computing cores for higher performance, but suffer from critical scalability problems especially in high-density cloud. In this paper, the proposed Hyper-Density I/O virtualization (HD-IOV) system tries to achieve pass-through level performance without scalability and flexibility limitations in previous works. HD-IOV is a software-hardware co-designed I/O virtualization solution. The core insight of HD-IOV is to decouple virtualization and resource management logic from hardware devices to software, reducing device complexity and enabling more flexible hardware resource management. DMA transactions and interrupts are sent directly to guest VMs without VM exits. Isolation is achieved by leveraging an existing PCIe feature, allowing IOMMU to enforce queue pair level isolation. Extensive experiments show that HD-IOV achieves similar performance as SR-IOV for both network and accelerator devices. Furthermore, HD-IOV supports maximally 2.96x higher device count with 2.9x faster median device initialization time, which is critical for emerging container and lightweight VM systems. The artifact can also be found on Github: https://github.com/Maphist0/hdiov-ae