In this letter, we propose an optimal direct load control of renewable powered smaller base stations (SBSs) in a two‐tier mobile network based on dynamic programming (DP). We represent the DP optimization using Graph Theory and state the problem as a Shortest Path search. We use the Label Correcting Method to explore the graph and find the optimal ON/OFF policy for the SBSs. Simulation results demonstrate that the proposed algorithm is able to adapt to the varying conditions of the environment, namely renewable energy arrivals and traffic demands. The key benefit of our study is that it allows elaborating on the behavior and performance bounds of the system and gives guidance for approximated policy search methods.