Learning Robust Rewards with Adversarial Inverse Reinforcement Learning

Aug 13, 2018

Justin Fu, Katie Luo, Sergey Levine

Aug 13, 2018

Justin Fu, Katie Luo, Sergey Levine

**Click to Read Paper**

One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors

Aug 11, 2016

Justin Fu, Sergey Levine, Pieter Abbeel

Aug 11, 2016

Justin Fu, Sergey Levine, Pieter Abbeel

**Click to Read Paper**

From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following

Feb 20, 2019

Justin Fu, Anoop Korattikara, Sergey Levine, Sergio Guadarrama

Feb 20, 2019

Justin Fu, Anoop Korattikara, Sergey Levine, Sergio Guadarrama

**Click to Read Paper**

EX2: Exploration with Exemplar Models for Deep Reinforcement Learning

May 27, 2017

Justin Fu, John D. Co-Reyes, Sergey Levine

May 27, 2017

Justin Fu, John D. Co-Reyes, Sergey Levine

**Click to Read Paper**

Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition

May 31, 2018

Justin Fu, Avi Singh, Dibya Ghosh, Larry Yang, Sergey Levine

May 31, 2018

Justin Fu, Avi Singh, Dibya Ghosh, Larry Yang, Sergey Levine

* First two authors contributed equally. Website: https://sites.google.com/view/inverse-event

**Click to Read Paper**

Generalizing Skills with Semi-Supervised Reinforcement Learning

Mar 09, 2017

Chelsea Finn, Tianhe Yu, Justin Fu, Pieter Abbeel, Sergey Levine

Mar 09, 2017

Chelsea Finn, Tianhe Yu, Justin Fu, Pieter Abbeel, Sergey Levine

* ICLR 2017

**Click to Read Paper**

**Click to Read Paper**

**Click to Read Paper**

Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets

Oct 30, 2015

Justin Domke

Oct 30, 2015

Justin Domke

* Advances in Neural Information Processing Systems 2015

**Click to Read Paper**

* Advances in Neural Information Processing Systems 2013

**Click to Read Paper**

Graphical models trained using maximum likelihood are a common tool for probabilistic inference of marginal distributions. However, this approach suffers difficulties when either the inference process or the model is approximate. In this paper, the inference process is first defined to be the minimization of a convex function, inspired by free energy approximations. Learning is then done directly in terms of the performance of the inference process at univariate marginal prediction. The main novelty is that this is a direct minimization of emperical risk, where the risk measures the accuracy of predicted marginals.

* Appears in Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI2008)

* Appears in Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI2008)

**Click to Read Paper**
DGM: A deep learning algorithm for solving partial differential equations

Sep 05, 2018

Justin Sirignano, Konstantinos Spiliopoulos

High-dimensional PDEs have been a longstanding computational challenge. We propose to solve high-dimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to $200$ dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE and Burgers' equation. The deep learning algorithm approximates the general solution to the Burgers' equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs.
Sep 05, 2018

Justin Sirignano, Konstantinos Spiliopoulos

* Deep learning, machine learning, partial differential equations

**Click to Read Paper**

Solving Equations of Random Convex Functions via Anchored Regression

Aug 13, 2018

Sohail Bahmani, Justin Romberg

Aug 13, 2018

Sohail Bahmani, Justin Romberg

**Click to Read Paper**

**Click to Read Paper**

Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem

Nov 02, 2017

Justin Sirignano, Konstantinos Spiliopoulos

Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An L$^p$ convergence rate is also proven for the algorithm in the strongly convex case.
Nov 02, 2017

Justin Sirignano, Konstantinos Spiliopoulos

**Click to Read Paper**

Stochastic Gradient Descent in Continuous Time

Oct 29, 2017

Justin Sirignano, Konstantinos Spiliopoulos

Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. SGDCT performs an online parameter update in continuous time, with the parameter updates $\theta_t$ satisfying a stochastic differential equation. We prove that $\lim_{t \rightarrow \infty} \nabla \bar g(\theta_t) = 0$ where $\bar g$ is a natural objective function for the estimation of the continuous-time dynamics. The convergence proof leverages ergodicity by using an appropriate Poisson equation to help describe the evolution of the parameters for large times. SGDCT can also be used to solve continuous-time optimization problems, such as American options. For certain continuous-time problems, SGDCT has some promising advantages compared to a traditional stochastic gradient descent algorithm. As an example application, SGDCT is combined with a deep neural network to price high-dimensional American options (up to 100 dimensions).
Oct 29, 2017

Justin Sirignano, Konstantinos Spiliopoulos

**Click to Read Paper**

AI Programmer: Autonomously Creating Software Programs Using Genetic Algorithms

Sep 17, 2017

Kory Becker, Justin Gottschlich

Sep 17, 2017

Kory Becker, Justin Gottschlich

**Click to Read Paper**

Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation

Mar 16, 2017

Sohail Bahmani, Justin Romberg

Mar 16, 2017

Sohail Bahmani, Justin Romberg

* Accepted in AISTATS 2017. Extended the discussion of related work and added a few more references. Clarified some of the statements and notations

**Click to Read Paper**

We examine the effect of clamping variables for approximate inference in undirected graphical models with pairwise relationships and discrete variables. For any number of variable labels, we demonstrate that clamping and summing approximate sub-partition functions can lead only to a decrease in the partition function estimate for TRW, and an increase for the naive mean field method, in each case guaranteeing an improvement in the approximation and bound. We next focus on binary variables, add the Bethe approximation to consideration and examine ways to choose good variables to clamp, introducing new methods. We show the importance of identifying highly frustrated cycles, and of checking the singleton entropy of a variable. We explore the value of our methods by empirical analysis and draw lessons to guide practitioners.

**Click to Read Paper**
bartMachine: Machine Learning with Bayesian Additive Regression Trees

Nov 24, 2014

Adam Kapelner, Justin Bleich

Nov 24, 2014

Adam Kapelner, Justin Bleich

* 39 pages, 13 figures, 4 tables, 2 appendices

**Click to Read Paper**