Downloads provided by UsageCounts
Abstract In control theory, to solve a finite-horizon sequential decision problem (SDP) commonly means to find a list of decision rules that result in an optimal expected total reward (or cost) when taking a given number of decision steps. SDPs are routinely solved using Bellman’s backward induction. Textbook authors (e.g. Bertsekas or Puterman) typically give more or less formal proofs to show that the backward induction algorithm is correct as solution method for deterministic and stochastic SDPs. Botta, Jansson and Ionescu propose a generic framework for finite horizon, monadic SDPs together with a monadic version of backward induction for solving such SDPs. In monadic SDPs, the monad captures a generic notion of uncertainty, while a generic measure function aggregates rewards. In the present paper, we define a notion of correctness for monadic SDPs and identify three conditions that allow us to prove a correctness result for monadic backward induction that is comparable to textbook correctness proofs for ordinary backward induction. The conditions that we impose are fairly general and can be cast in category-theoretical terms using the notion of Eilenberg–Moore algebra. They hold in familiar settings like those of deterministic or stochastic SDPs, but we also give examples in which they fail. Our results show that backward induction can safely be employed for a broader class of SDPs than usually treated in textbooks. However, they also rule out certain instances that were considered admissible in the context of Botta et al. ’s generic framework. Our development is formalised in Idris as an extension of the Botta et al. framework and the sources are available as supplementary material.
ddc:004, FOS: Computer and information sciences, Computer Science - Logic in Computer Science, Monadic Dynamical Systems, 330, Sequential Decision Problems, Type Theory, Institut für Informatik und Computational Science, Verified generic programming, Eilenberg-Moore and Kleisli constructions for monads, 004, Logic in Computer Science (cs.LO), Dependent Type Theory, Verified Decision Making, Verified programming, Backward Induction, Idris, Control Theory, Functional programming and lambda calculus
ddc:004, FOS: Computer and information sciences, Computer Science - Logic in Computer Science, Monadic Dynamical Systems, 330, Sequential Decision Problems, Type Theory, Institut für Informatik und Computational Science, Verified generic programming, Eilenberg-Moore and Kleisli constructions for monads, 004, Logic in Computer Science (cs.LO), Dependent Type Theory, Verified Decision Making, Verified programming, Backward Induction, Idris, Control Theory, Functional programming and lambda calculus
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 3 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 10% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
| views | 2 | |
| downloads | 4 |

Views provided by UsageCounts
Downloads provided by UsageCounts