Sixth-generation (6G) wireless networks are expected to meet all the demands of the next decade, a feasibility that is onlypossible with advances in network design and management. This paper first proposes a unified resource management framework for a 6G-based network architecture that includes an Open Radio Access Network (O-RAN) deployment and then defines a hierarchical network energy control and resource management approach. Leveraging the openness of O-RAN, the proposed novel Hierarchical Reinforcement Learning (HRL)-based scheme provides two-level centralized agent policy evaluation and decentralized agent real-time scheduling optimization, thereby minimizing system energy consumption while ensuring Quality of Service (QoS). To validate the performance of this management approach, we propose a Deep Reinforcement Learning (DRL)- based algorithm for each level. Simulation results demonstrate the effectiveness of this solution in terms of throughput, userexperienced data rate, and energy consumption.