Powered by OpenAIRE graph
Found an issue? Give us feedback
addClaim

A Multi-agent Reinforcement Learning Approach to the Schedule and Aircraft Recovery Problem in Airline Disruption Management

Authors: Lin, Hong;

A Multi-agent Reinforcement Learning Approach to the Schedule and Aircraft Recovery Problem in Airline Disruption Management

Abstract

Unexpected events such as airport closures pose significant operational and financial challenges to the operations control centre, requiring decision-makers to make rapid and cost-efficient decisions that minimise the impacts and recover the schedule from these disruptions as soon as possible. This thesis addresses the Schedule and Aircraft Recovery Problem (SARP) by developing three reinforcement learning approaches: a two-stage Single-Agent Reinforcement Learning (SARL) approach with Q-Learning, a Deep-Q-Network Multi-Agent Reinforcement Learning (DQN-MARL) model, and a Communication Networks Multi-Agent Reinforcement Learning (CommNet-MARL) model. Results were benchmarked with an optimisation-based model. These models were tested in a case-study network with 456 flights and 81 aircraft for a two-hour airport closure scenario. The MARL training results achieved objective values within 1% of the CPLEX exact solution, while the two-stage SARL achieved a 9.14% gap, highlighting the importance of agents' collaboration in SARP. A key difference between CPLEX and MARL is that CPLEX cancelled early flights to minimise cascade disruption effects, while MARL models showed a strong preference for delays over cancellations to preserve network connectivity. This thesis further evaluated the generalisation performance of models across varying disruption scenarios, including closure windows of one to six hours from 05:00 - 24:00. The evaluation results demonstrated the CommNet-MARL model with action-masking achieving a smaller generalisation gap between 0% to 11.97%, outperforming DQN-MARL (1.52% - 31.56%) and CommNet-MARL without action masking (0% - 30.44%), with all solutions generated within seconds. Scenarios with fewer than 100 total affected flights achieved near-optimal results (gaps < 5%), whereas larger disruptions with extensive downstream delays exhibited larger gaps. This research contributes to the field by first introducing a collaborative MARL framework for SARP that addresses the limitations of existing SARL and sequential optimisation approaches. The proposed MARL framework introduces a communication mechanism that enables iterative negotiation between agents, reflecting the distributed nature of real-world recovery decision-making in an airline's AOCC. Second, this research provided a systematic generalisation analysis across 192 disruption scenarios. Results show that RL approaches excel in small-scale disruptions but struggle in scenarios with extensive cascade delays due to sequential decision-making limitations. Third, this research demonstrated that collaborative MARL frameworks outperform the sequential approach, underscoring the importance of agent collaboration in airline operations and offering valuable insights into future airline disruption management.

Keywords

350901 Air transportation and freight services, 460210 Satisfiability and optimisation, Reinforcement Learning, Integrated Schedule and Aircraft Recovery, Multi-Agent Reinforcement Learning

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
Upload OA version
Are you the author of this publication? Upload your Open Access version to Zenodo!
It’s fast and easy, just two clicks!