Orthogonal frequency-division multiple access (OFDMA) has attracted great attention as a key technology for uplink enhancement for Wi-Fi, since it can effectively reduce network congestion and channel access delay. Unfortunately, the traditional random access protocol of Wi-Fi seldom allows these benefits to be achieved, especially in dense network environments, as the access point (AP) rarely gains the channel access needed to trigger OFDMA uplink transmissions due to severe frame collisions. To address this problem, we propose a new channel access scheme called Contention-Free Channel Access for 802.11ax (CFX). In the proposed scheme, users can access the channel without contention, since they are guaranteed a transmission opportunity immediately after another user’s transmission. To realize CFX on top of the existing Buffer Status Report/BSR Poll (BSR/BSRP) exchange protocol of 802.11ax, we develop an additional scheme based on shared channel access that helps the AP to obtain the buffer status of users and manage a contention-free channel access schedule. In addition, in order to appropriately utilize the savings from the reduced frame collisions, we conduct sum throughput maximization using an actor-critic proximal policy optimization (PPO)-based deep reinforcement learning approach. The results of an extensive evaluation show that CFX not only significantly improves the uplink performance of Wi-Fi in terms of throughput and channel access delay but can also dynamically adjust the parameters in response to changes in the network status.