Unmanned aerial vehicle decision-making issues are increasingly being addressed using reinforcement learning (RL) (UAVs). The current advances in RL-based algorithms for UAV applications, encompassing both single-agent and swarm scenarios, are thoroughly reviewed in this work. First, the basic concepts of RL and its variants are introduced, followed by an overview of the state-of-the[1]art RL algorithms that have been applied to UAV navigation, path planning, and obstacle avoidance. The study then examines real-time learning concerns, model selection, and exploration-exploitation trade-offs, as well as challenges and potential for employing RL in UAV systems. In order to further the use of RL in UAVs, future research initiatives are also suggested. They include creating hybrid methods that integrate RL with other methodologies and incorporating human feedback and domain expertise into the learning process. Overall, this work demonstrates the potential of this approach to improve the autonomy, adaptability, and resilience of UAV systems and serves as a significant resource for researchers and those interested in applying RL to UAVs.