Abstract The 5 th Generation (5G) and Beyond 5G (B5G) are expected to be the enabling technologies for Internet-of-Everything (IoE). The quality-of-service (QoS) for IoE in the context of uplink data delivery of the content is of prime importance. The 3 rd Generation Partnership Project (3GPP) standardizes the Narrowband Internet-of-Things (NB-IoT) in 5G, which is Low Power Wide Area (LPWA) technology to enhance the coverage and to optimize the power consumption for the IoT devices. Repetitions of control and data signals between NB-IoT User Equipment (UE) and the evolved NodeB/Base Station (eNB/BS), is one of the most prominent characteristics in NB-IoT. These repetitions ensure high reliability in the context of data delivery of time-sensitive applications, e.g., healthcare applications. However, these repetitions degrade the performance of the resource-constrained IoT network in terms of energy consumption. Device-to-Device (D2D) communication standardized in Long Term Evolution-Advanced (LTE-A) offers a key solution for NB-IoT UE to transmit in two hops route instead of direct uplink, which augments the efficiency of the system. In an effort to improve the data packet delivery, this study investigates D2D communication for NB-IoT delay-sensitive applications, such as healthcare-IoT services. This study formulates the selection of D2D communication relay as Multi-Armed Bandit (MAB) problem and incorporates Upper Confidence Bound (UCB) based Reinforcement Learning (RL) to solve MAB problem. The proposed Intelligent-D2D (I-D2D) communication methodology selects the optimum relay with a maximum Packet Delivery Ratio (PDR) with minimum End-to-End Delay (EED), which ultimately augments energy efficiency.