We address the relative paucity of empirical testing of learning algorithms (of any type) by introducing a new public-domain, Modular, Optimal Learning Testing Environment (MOLTE) for Bayesian ranking and selection problem, stochastic bandits or sequential experimental design problems. The Matlab-based simulator allows the comparison of a number of learning policies (represented as a series of .m modules) in the context of a wide range of problems (each represented in its own .m module) which makes it easy to add new algorithms and new test problems. State-of-the-art policies and various problem classes are provided in the package. The choice of problems and policies is guided through a spreadsheet-based interface. Different graphical metrics are included. MOLTE is designed to be compatible with parallel computing to scale up from local desktop to clusters and clouds. We offer MOLTE as an easy-to-use tool for the research community that will make it possible to perform much more comprehensive testing, spanning a broader selection of algorithms and test problems. We demonstrate the capabilities of MOLTE through a series of comparisons of policies on a starter library of test problems. We also address the problem of tuning and constructing priors that have been largely overlooked in optimal learning literature. We envision MOLTE as a modest spur to provide researchers an easy environment to study interesting questions involved in optimal learning.

**Click to Read Paper*** in ISMB Bio-Ontologies, 2012

**Click to Read Paper**

Optimal Learning for Stochastic Optimization with Nonlinear Parametric Belief Models

Nov 22, 2016

Xinyu He, Warren B. Powell

We consider the problem of estimating the expected value of information (the knowledge gradient) for Bayesian learning problems where the belief model is nonlinear in the parameters. Our goal is to maximize some metric, while simultaneously learning the unknown parameters of the nonlinear belief model, by guiding a sequential experimentation process which is expensive. We overcome the problem of computing the expected value of an experiment, which is computationally intractable, by using a sampled approximation, which helps to guide experiments but does not provide an accurate estimate of the unknown parameters. We then introduce a resampling process which allows the sampled model to adapt to new information, exploiting past experiments. We show theoretically that the method converges asymptotically to the true parameters, while simultaneously maximizing our metric. We show empirically that the process exhibits rapid convergence, yielding good results with a very small number of experiments.
Nov 22, 2016

Xinyu He, Warren B. Powell

**Click to Read Paper**

An optimal learning method for developing personalized treatment regimes

Jul 06, 2016

Yingfei Wang, Warren Powell

Jul 06, 2016

Yingfei Wang, Warren Powell

**Click to Read Paper**

**Click to Read Paper**

XNMR is a system designed to explore the results of combining the well-founded semantics system XSB with the stable-models evaluator SMODELS. Its main goal is to work as a tool for fast and interactive exploration of knowledge bases.

* 2 pages; no figures; NMR2000 Systems Description

* 2 pages; no figures; NMR2000 Systems Description

**Click to Read Paper**
ABox Abduction via Forgetting in ALC (Long Version)

Nov 13, 2018

Warren Del-Pinto, Renate A. Schmidt

Nov 13, 2018

Warren Del-Pinto, Renate A. Schmidt

* Long version of a paper accepted for publication in the proceedings of AAAI 2019

**Click to Read Paper**

* Accepted to Asilomar 2018 - special session on "Machine Learning for Wireless Systems"

**Click to Read Paper**

Optimal Learning for Sequential Decision Making for Expensive Cost Functions with Stochastic Binary Feedbacks

Sep 13, 2017

Yingfei Wang, Chu Wang, Warren Powell

We consider the problem of sequentially making decisions that are rewarded by "successes" and "failures" which can be predicted through an unknown relationship that depends on a partially controllable vector of attributes for each instance. The learner takes an active role in selecting samples from the instance pool. The goal is to maximize the probability of success in either offline (training) or online (testing) phases. Our problem is motivated by real-world applications where observations are time-consuming and/or expensive. We develop a knowledge gradient policy using an online Bayesian linear classifier to guide the experiment by maximizing the expected value of information of labeling each alternative. We provide a finite-time analysis of the estimated error and show that the maximum likelihood estimator based produced by the KG policy is consistent and asymptotically normal. We also show that the knowledge gradient policy is asymptotically optimal in an offline setting. This work further extends the knowledge gradient to the setting of contextual bandits. We report the results of a series of experiments that demonstrate its efficiency.
Sep 13, 2017

Yingfei Wang, Chu Wang, Warren Powell

* arXiv admin note: text overlap with arXiv:1510.02354

**Click to Read Paper**

* Published as a conference paper at the 2017 International Symposium on Information Theory (ISIT)

**Click to Read Paper**

* David S. Warren and Yanhong A. Liu (Editors). 33 pages. Including summaries by Christopher Kane and abstracts or position papers by M. Aref, J. Rosenwald, I. Cervesato, E.S.L. Lam, M. Balduccini, J. Lobo, A. Russo, E. Lupu, N. Leone, F. Ricca, G. Gupta, K. Marple, E. Salazar, Z. Chen, A. Sobhi, S. Srirangapalli, C.R. Ramakrishnan, N. Bj{\o}rner, N.P. Lopes, A. Rybalchenko, and P. Tarau

**Click to Read Paper**

Descriptor transition tables for object retrieval using unconstrained cluttered video acquired using a consumer level handheld mobile device

Mar 21, 2016

Warren Rieutort-Louis, Ognjen Arandjelovic

Mar 21, 2016

Warren Rieutort-Louis, Ognjen Arandjelovic

* 2016

**Click to Read Paper**

The Knowledge Gradient with Logistic Belief Models for Binary Classification

Oct 08, 2015

Yingfei Wang, Chu Wang, Warren Powell

We consider sequential decision making problems for binary classification scenario in which the learner takes an active role in repeatedly selecting samples from the action pool and receives the binary label of the selected alternatives. Our problem is motivated by applications where observations are time consuming and/or expensive, resulting in small samples. The goal is to identify the best alternative with the highest response. We use Bayesian logistic regression to predict the response of each alternative. By formulating the problem as a Markov decision process, we develop a knowledge-gradient type policy to guide the experiment by maximizing the expected value of information of labeling each alternative and provide a finite-time analysis on the estimated error. Experiments on benchmark UCI datasets demonstrate the effectiveness of the proposed method.
Oct 08, 2015

Yingfei Wang, Chu Wang, Warren Powell

**Click to Read Paper**

Recursive Optimization of Convex Risk Measures: Mean-Semideviation Models

Oct 29, 2018

Dionysios S. Kalogerias, Warren B. Powell

We develop recursive, data-driven, stochastic subgradient methods for optimizing a new, versatile, and application-driven class of convex risk measures, termed here as mean-semideviations, strictly generalizing the well-known and popular mean-upper-semideviation. We introduce the MESSAGEp algorithm, which is an efficient compositional subgradient procedure for iteratively solving convex mean-semideviation risk-averse problems to optimality. We analyze the asymptotic behavior of the MESSAGEp algorithm under a flexible and structure-exploiting set of problem assumptions. In particular: 1) Under appropriate stepsize rules, we establish pathwise convergence of the MESSAGEp algorithm in a strong technical sense, confirming its asymptotic consistency. 2) Assuming a strongly convex cost, we show that, for fixed semideviation order $p>1$ and for $\epsilon\in\left[0,1\right)$, the MESSAGEp algorithm achieves a squared-${\cal L}_{2}$ solution suboptimality rate of the order of ${\cal O}(n^{-\left(1-\epsilon\right)/2})$ iterations, where, for $\epsilon>0$, pathwise convergence is simultaneously guaranteed. This result establishes a rate of order arbitrarily close to ${\cal O}(n^{-1/2})$, while ensuring strongly stable pathwise operation. For $p\equiv1$, the rate order improves to ${\cal O}(n^{-2/3})$, which also suffices for pathwise convergence, and matches previous results. 3) Likewise, in the general case of a convex cost, we show that, for any $\epsilon\in\left[0,1\right)$, the MESSAGEp algorithm with iterate smoothing achieves an ${\cal L}_{1}$ objective suboptimality rate of the order of ${\cal O}(n^{-\left(1-\epsilon\right)/\left(4\bf{1}_{\left\{ p>1\right\} }+4\right)})$ iterations. This result provides maximal rates of ${\cal O}(n^{-1/4})$, if $p\equiv1$, and ${\cal O}(n^{-1/8})$, if $p>1$, matching the state of the art, as well.
Oct 29, 2018

Dionysios S. Kalogerias, Warren B. Powell

* 90 pages, 3 figures. Update: Substantial revision of the technical content, with an additional fully detailed analysis in regard to the rate of convergence of the MESSAGEp algorithm. NOTE: Please open in browser to see the math in the abstract!

**Click to Read Paper**

Consider a sample of $n$ points taken i.i.d from a submanifold $\Sigma$ of Euclidean space. We show that there is a way to estimate the Ricci curvature of $\Sigma$ with respect to the induced metric from the sample. Our method is grounded in the notions of Carr\'e du Champ for diffusion semi-groups, the theory of Empirical processes and local Principal Component Analysis.

* 47 pages

* 47 pages

**Click to Read Paper**
Clusters of Driving Behavior from Observational Smartphone Data

Jan 11, 2018

Josh Warren, Jeff Lipkowitz, Vadim Sokolov

Jan 11, 2018

Josh Warren, Jeff Lipkowitz, Vadim Sokolov

**Click to Read Paper**

Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures

May 09, 2017

Daniel R. Jiang, Warren B. Powell

May 09, 2017

Daniel R. Jiang, Warren B. Powell

* 39 pages, 7 figures

**Click to Read Paper**

Multiple-Path Selection for new Highway Alignments using Discrete Algorithms

Jul 30, 2015

Yasha Pushak, Warren Hare, Yves Lucet

Jul 30, 2015

Yasha Pushak, Warren Hare, Yves Lucet

* to be published in European Journal of Operational Research

**Click to Read Paper**

The Knowledge Gradient Policy Using A Sparse Additive Belief Model

Mar 18, 2015

Yan Li, Han Liu, Warren Powell

Mar 18, 2015

Yan Li, Han Liu, Warren Powell

**Click to Read Paper**

Understanding partition comparison indices based on counting object pairs

Jan 07, 2019

Matthijs J. Warrens, Hanneke van der Hoef

Jan 07, 2019

Matthijs J. Warrens, Hanneke van der Hoef

* 29 pages, 7 tables

**Click to Read Paper**