The main objective of this paper is to introduce a transfer learning-enhanced deep reinforcement learning (DRL) methodology that is able to optimize the geometry of any airfoil based on concomitant aerodynamic and structural integrity criteria. To showcase the method, we aim to maximize the lift-to-drag ratio CL/CD while preserving the structural integrity of the airfoil—as modeled by its maximum thickness—and train the DRL agent using a list of different transfer learning (TL) strategies. The performance of the DRL agent is compared with Particle Swarm Optimization (PSO), a traditional gradient-free optimization method. Results indicate that DRL agents are able to perform purely aerodynamic and hybrid aerodynamic/structural shape optimization that the DRL approach outperforms PSO in terms of computational efficiency and aerodynamic improvement and that the TL-enhanced DRL agent achieves performance comparable to the DRL one, while further saving substantial computational resources.