Publications

• Sorted by Date • Classified by Publication Type • Classified by Research Category •

Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version)

Philipp Robbel, Frans A. Oliehoek, and Mykel J. Kochenderfer. Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version). ArXiv e-prints, arXiv:1511.09080, February 2016.

Download

pdf [392.8kB]  

Abstract

Many exact and approximate solution methods for Markov Decision Processes (MDPs) attempt to exploit structure in the problem and are based on factorization of the value function. Especially multiagent settings, however, are known to suffer from an exponential increase in value component sizes as interactions become denser, meaning that approximation architectures are restricted in the problem sizes and types they can handle. We present an approach to mitigate this limitation for certain types of multiagent systems, exploiting a property that can be thought of as "anonymous influence" in the factored MDP. Anonymous influence summarizes joint variable effects efficiently whenever the explicit representation of variable identity in the problem can be avoided. We show how representational benefits from anonymity translate into computational efficiencies, both for general variable elimination in a factor graph but in particular also for the approximate linear programming solution to factored MDPs. The latter allows to scale linear programming to factored MDPs that were previously unsolvable. Our results are shown for the control of a stochastic disease process over a densely connected graph with 50 nodes and 25 agents.

BibTeX Entry

@article{Robbel16arxiv,
    title =     {Exploiting Anonymity in Approximate Linear Programming:
                 Scaling to Large Multiagent {MDPs} (Extended Version)},
    author =    {Philipp Robbel and
                 Frans A. Oliehoek and
                 Mykel J. Kochenderfer},
    journal =   {ArXiv e-prints},
    volume =    {arXiv:1511.09080},
    year =      2016,
    month =     feb,
    keywords =   {nonrefereed, arxiv},
    abstract = {
    Many exact and approximate solution methods for Markov Decision Processes
    (MDPs) attempt to exploit structure in the problem and are based on
    factorization of the value function. Especially multiagent settings,
    however, are known to suffer from an exponential increase in value component
    sizes as interactions become denser, meaning that approximation
    architectures are restricted in the problem sizes and types they can
    handle. We present an approach to mitigate this limitation for certain
    types of multiagent systems, exploiting a property that can be thought of
    as "anonymous influence" in the factored MDP. Anonymous influence
    summarizes joint variable effects efficiently whenever the explicit
    representation of variable identity in the problem can be avoided. We show
    how representational benefits from anonymity translate into computational
    efficiencies, both for general variable elimination in a factor graph but
    in particular also for the approximate linear programming solution to
    factored MDPs. The latter allows to scale linear programming to factored
    MDPs that were previously unsolvable. Our results are shown for the control
    of a stochastic disease process over a densely connected graph with 50
    nodes and 25 agents. 
    }
}

Generated by bib2html.pl (written by Patrick Riley) on Mon Apr 08, 2024 20:28:07 UTC