
Abstract Purpose Autonomous navigation of catheters and guidewires can enhance endovascular surgery safety and efficacy, reducing procedure times and operator radiation exposure. Integrating tele-operated robotics could widen access to time-sensitive emergency procedures like mechanical thrombectomy (MT). Reinforcement learning (RL) shows potential in endovascular navigation, yet its application encounters challenges without a reward signal. This study explores the viability of autonomous guidewire navigation in MT vasculature using inverse reinforcement learning (IRL) to leverage expert demonstrations. Methods Employing the Simulation Open Framework Architecture (SOFA), this study established a simulation-based training and evaluation environment for MT navigation. We used IRL to infer reward functions from expert behaviour when navigating a guidewire and catheter. We utilized the soft actor-critic algorithm to train models with various reward functions and compared their performance in silico. Results We demonstrated feasibility of navigation using IRL. When evaluating single- versus dual-device (i.e. guidewire versus catheter and guidewire) tracking, both methods achieved high success rates of 95% and 96%, respectively. Dual tracking, however, utilized both devices mimicking an expert. A success rate of 100% and procedure time of 22.6 s were obtained when training with a reward function obtained through ‘reward shaping’. This outperformed a dense reward function (96%, 24.9 s) and an IRL-derived reward function (48%, 59.2 s). Conclusions We have contributed to the advancement of autonomous endovascular intervention navigation, particularly MT, by effectively employing IRL based on demonstrator expertise. The results underscore the potential of using reward shaping to efficiently train models, offering a promising avenue for enhancing the accessibility and precision of MT procedures. We envisage that future research can extend our methodology to diverse anatomical structures to enhance generalizability.
FOS: Computer and information sciences, Artificial intelligence, Computer Science - Machine Learning, Catheters, Computer Science - Artificial Intelligence, 610, Machine Learning (cs.LG), Computer Science - Robotics, Machine learning, Humans, Computer Simulation, Thrombectomy, Endovascular intervention, Algorithms [MeSH] ; Endovascular Procedures/instrumentation [MeSH] ; Machine learning ; Endovascular Procedures/methods [MeSH] ; Thrombectomy/methods [MeSH] ; Inverse reinforcement learning ; Humans [MeSH] ; Autonomous navigation ; Catheters [MeSH] ; Artificial intelligence ; Thrombectomy/instrumentation [MeSH] ; Computer Simulation [MeSH] ; Mechanical thrombectomy ; Surgery, Computer-Assisted/methods [MeSH] ; Feasibility Studies [MeSH] ; Original Article ; Endovascular intervention, Endovascular Procedures, Inverse reinforcement learning, Artificial Intelligence (cs.AI), Surgery, Computer-Assisted, Feasibility Studies, Original Article, Mechanical thrombectomy, Autonomous navigation, Robotics (cs.RO), Algorithms
FOS: Computer and information sciences, Artificial intelligence, Computer Science - Machine Learning, Catheters, Computer Science - Artificial Intelligence, 610, Machine Learning (cs.LG), Computer Science - Robotics, Machine learning, Humans, Computer Simulation, Thrombectomy, Endovascular intervention, Algorithms [MeSH] ; Endovascular Procedures/instrumentation [MeSH] ; Machine learning ; Endovascular Procedures/methods [MeSH] ; Thrombectomy/methods [MeSH] ; Inverse reinforcement learning ; Humans [MeSH] ; Autonomous navigation ; Catheters [MeSH] ; Artificial intelligence ; Thrombectomy/instrumentation [MeSH] ; Computer Simulation [MeSH] ; Mechanical thrombectomy ; Surgery, Computer-Assisted/methods [MeSH] ; Feasibility Studies [MeSH] ; Original Article ; Endovascular intervention, Endovascular Procedures, Inverse reinforcement learning, Artificial Intelligence (cs.AI), Surgery, Computer-Assisted, Feasibility Studies, Original Article, Mechanical thrombectomy, Autonomous navigation, Robotics (cs.RO), Algorithms
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 6 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 10% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Top 10% |
