• Login
    View Item 
    •   DSpace Home
    • ASME
    • Journal of Dynamic Systems, Measurement, and Control
    • View Item
    •   DSpace Home
    • ASME
    • Journal of Dynamic Systems, Measurement, and Control
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Adiabatic Markov Decision Process: Convergence of Value Iteration Algorithm

    Thumbnail
    View/Open
    ds_138_06_061009.pdf (764.9Kb)
    Date
    2016
    Author
    Duong, Thai
    Nguyen
    Nguyen, Thinh
    Metadata
    Show full item record
    Abstract
    Markov decision process (MDP) is a wellknown framework for devising the optimal decisionmaking strategies under uncertainty. Typically, the decision maker assumes a stationary environment which is characterized by a timeinvariant transition probability matrix. However, in many realworld scenarios, this assumption is not justified, thus the optimal strategy might not provide the expected performance. In this paper, we study the performance of the classic value iteration algorithm for solving an MDP problem under nonstationary environments. Specifically, the nonstationary environment is modeled as a sequence of timevariant transition probability matrices governed by an adiabatic evolution inspired from quantum mechanics. We characterize the performance of the value iteration algorithm subject to the rate of change of the underlying environment. The performance is measured in terms of the convergence rate to the optimal average reward. We show two examples of queuing systems that make use of our analysis framework.
    URI
    http://yetl.yabesh.ir/yetl/handle/yetl/160690
    Collections
    • Journal of Dynamic Systems, Measurement, and Control

    DSpace software copyright © 2002-2015  DuraSpace
    Contact Us | Send Feedback
    Theme by 
    @mire NV
     

     

    Browse

    All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    LoginRegister

    Statistics

    View Usage Statistics

    DSpace software copyright © 2002-2015  DuraSpace
    Contact Us | Send Feedback
    Theme by 
    @mire NV