Rate-Splitting Multiple Access (RSMA) has recently emerged as an effective technique for increasing network capacity by smartly controlling the tradeoff between decoding and treating interference as noise. In this paper, aligned with the need for sustainable wireless networks, we study the energy-efficient power and rate allocation of the common and private messages in the downlink of a network where rate-splitting is adopted. The corresponding energy efficiency maximization problem is transformed into a multi-agent Deep Reinforcement Learning (DRL) problem, based on which each private stream transmitted in the downlink constitutes a different DRL agent. The formulated DRL problem is solved by using Deep Q-Learning (DQL) algorithm and training a single Deep Q-Network (DQN) from the cumulative experiences gained from the DRL agents and their exploration of the environment, i.e., the exploration of different private-message power allocations. Numerical results obtained via modeling and simulation verify the effectiveness of the proposed DQL algorithm, demonstrating that it concludes solutions that outperform existing approaches from the literature in the achieved energy efficiency.