The accelerated uptake of unmanned aerial vehicles (UAVs) has significantly altered communication and data exchange landscapes but has also introduced substantial security challenges, especially in open-access UAV communication environments. To address these, Elliptic curve cryptography (ECC) offers robust security with computational efficiency, ideal for resource-constrained Internet of Drones (IoD) systems. This study proposes a Secure and Efficient Three-Way Key Exchange (SETKE) protocol using ECC, specifically tailored for IoD. The SETKE protocol’s security was rigorously analyzed within an extended Bellare–Pointcheval–Rogaway (BPR) model under the random oracle assumption, demonstrating its resilience. Formal verification using the AVISPA tool confirmed the protocol’s safety against man-in-the-middle (MITM) attacks, and formal proofs establish its Authenticated Key Exchange (AKE) security. In terms of performance, SETKE is highly efficient, requiring only 3 ECC scalar multiplications for the Service Requester drone, 4 for the Service Provider drone, and 3 for the Control Server, which is demonstrably lower than several existing schemes. My approach achieves this robust protection with minimal communication overhead (e.g., a maximum payload of 844 bits per session), ensuring its practicality for resource-limited IoD environments. The significance of this work for the IoD field lies in providing a provably secure, lightweight, and computationally efficient key exchange mechanism vital for addressing critical security challenges in IoD systems.