Speeding up planning in Markov decision processes via automatically constructed abstractions

Isaza, Alejandro and Szepesvári, Csaba and Bulitko, Vadim and Greiner, Russel (2008) Speeding up planning in Markov decision processes via automatically constructed abstractions. In: UAI 2008. Proceedings of the 24th conference in uncertainty in artificial intelligence. Helsinki, 2008..

[img] Text
prmdp.pdf - Published Version
Restricted to Registered users only

Download (357kB)

Abstract

In this paper, we consider planning in stochastic shortest path problems, a subclass of Markov Decision Problems (MDP). We focus on medium-size problems whose state space can be fully enumerated. This problem has numerous important applications, such as navigation and planning under uncertainty. We propose a new approach for constructing a multi-level hierarchy of progressively simpler abstractions of the original problem. Once computed, the hierarchy can be used to speed up planning by first finding a policy for the most abstract level and then recursively refining it into a solution to the original problem. This approach is fully automated and delivers a speed-up of two orders of magnitude over a state-of-the-art MDP solver on sample problems while returning near-optimal solutions.

Item Type: Conference or Workshop Item (Paper)
Subjects: Q Science > QA Mathematics and Computer Science > QA75 Electronic computers. Computer science / számítástechnika, számítógéptudomány
Depositing User: Eszter Nagy
Date Deposited: 11 Dec 2012 15:32
Last Modified: 11 Dec 2012 15:32
URI: http://eprints.sztaki.hu/id/eprint/5602

Update Item Update Item