Wasserstein regularization for sparse multi-task regression

Oct 11, 2018

Hicham Janati, Marco Cuturi, Alexandre Gramfort

Oct 11, 2018

Hicham Janati, Marco Cuturi, Alexandre Gramfort

**Click to Read Paper**

Celer: a Fast Solver for the Lasso with Dual Extrapolation

Jun 06, 2018

Mathurin Massias, Alexandre Gramfort, Joseph Salmon

Convex sparsity-inducing regularizations are ubiquitous in high-dimensional machine learning, but solving the resulting optimization problems can be slow. To accelerate solvers, state-of-the-art approaches consist in reducing the size of the optimization problem at hand. In the context of regression, this can be achieved either by discarding irrelevant features (screening techniques) or by prioritizing features likely to be included in the support of the solution (working set techniques). Duality comes into play at several steps in these techniques. Here, we propose an extrapolation technique starting from a sequence of iterates in the dual that leads to the construction of improved dual points. This enables a tighter control of optimality as used in stopping criterion, as well as better screening performance of Gap Safe rules. Finally, we propose a working set strategy based on an aggressive use of Gap Safe screening rules. Thanks to our new dual point construction, we show significant computational speedups on multiple real-world problems.
Jun 06, 2018

Mathurin Massias, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

On the Consistency of Ordinal Regression Methods

Jul 21, 2017

Fabian Pedregosa, Francis Bach, Alexandre Gramfort

Jul 21, 2017

Fabian Pedregosa, Francis Bach, Alexandre Gramfort

**Click to Read Paper**

From safe screening rules to working sets for faster Lasso-type solvers

May 01, 2017

Mathurin Massias, Alexandre Gramfort, Joseph Salmon

May 01, 2017

Mathurin Massias, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

Anomaly Detection and Localisation using Mixed Graphical Models

Jul 20, 2016

Romain Laby, François Roueff, Alexandre Gramfort

We propose a method that performs anomaly detection and localisation within heterogeneous data using a pairwise undirected mixed graphical model. The data are a mixture of categorical and quantitative variables, and the model is learned over a dataset that is supposed not to contain any anomaly. We then use the model over temporal data, potentially a data stream, using a version of the two-sided CUSUM algorithm. The proposed decision statistic is based on a conditional likelihood ratio computed for each variable given the others. Our results show that this function allows to detect anomalies variable by variable, and thus to localise the variables involved in the anomalies more precisely than univariate methods based on simple marginals.
Jul 20, 2016

Romain Laby, François Roueff, Alexandre Gramfort

**Click to Read Paper**

Mind the duality gap: safer rules for the Lasso

Dec 03, 2015

Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

Screening rules allow to early discard irrelevant variables from the optimization in Lasso problems, or its derivatives, making solvers faster. In this paper, we propose new versions of the so-called $\textit{safe rules}$ for the Lasso. Based on duality gap considerations, our new rules create safe test regions whose diameters converge to zero, provided that one relies on a converging solver. This property helps screening out more variables, for a wider range of regularization parameter values. In addition to faster convergence, we prove that we correctly identify the active sets (supports) of the solutions in finite time. While our proposed strategy can cope with any solver, its performance is demonstrated using a coordinate descent algorithm particularly adapted to machine learning use cases. Significant computing time reductions are obtained with respect to previous safe rules.
Dec 03, 2015

Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

Calibration of One-Class SVM for MV set estimation

Aug 30, 2015

Albert Thomas, Vincent Feuillard, Alexandre Gramfort

A general approach for anomaly detection or novelty detection consists in estimating high density regions or Minimum Volume (MV) sets. The One-Class Support Vector Machine (OCSVM) is a state-of-the-art algorithm for estimating such regions from high dimensional data. Yet it suffers from practical limitations. When applied to a limited number of samples it can lead to poor performance even when picking the best hyperparameters. Moreover the solution of OCSVM is very sensitive to the selection of hyperparameters which makes it hard to optimize in an unsupervised setting. We present a new approach to estimate MV sets using the OCSVM with a different choice of the parameter controlling the proportion of outliers. The solution function of the OCSVM is learnt on a training set and the desired probability mass is obtained by adjusting the offset on a test set to prevent overfitting. Models learnt on different train/test splits are then aggregated to reduce the variance induced by such random splits. Our approach makes it possible to tune the hyperparameters automatically and obtain nested set estimates. Experimental results show that our approach outperforms the standard OCSVM formulation while suffering less from the curse of dimensionality than kernel density estimates. Results on actual data sets are also presented.
Aug 30, 2015

Albert Thomas, Vincent Feuillard, Alexandre Gramfort

**Click to Read Paper**

Fast Optimal Transport Averaging of Neuroimaging Data

Apr 10, 2015

Alexandre Gramfort, Gabriel Peyré, Marco Cuturi

Apr 10, 2015

Alexandre Gramfort, Gabriel Peyré, Marco Cuturi

**Click to Read Paper**

Small-sample Brain Mapping: Sparse Recovery on Spatially Correlated Designs with Randomization and Clustering

Jun 27, 2012

Gael Varoquaux, Alexandre Gramfort, Bertrand Thirion

Jun 27, 2012

Gael Varoquaux, Alexandre Gramfort, Bertrand Thirion

**Click to Read Paper**

Beyond Pham's algorithm for joint diagonalization

Nov 28, 2018

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

Nov 28, 2018

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

**Click to Read Paper**

Accelerating likelihood optimization for ICA on real signals

Jun 25, 2018

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

Jun 25, 2018

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

**Click to Read Paper**

Faster ICA under orthogonal constraint

Nov 29, 2017

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data widely used in observational sciences. In its classical form, ICA relies on modeling the data as a linear mixture of non-Gaussian independent sources. The problem can be seen as a likelihood maximization problem. We introduce Picard-O, a preconditioned L-BFGS strategy over the set of orthogonal matrices, which can quickly separate both super- and sub-Gaussian signals. It returns the same set of sources as the widely used FastICA algorithm. Through numerical experiments, we show that our method is faster and more robust than FastICA on real data.
Nov 29, 2017

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

**Click to Read Paper**

Faster independent component analysis by preconditioning with Hessian approximations

Sep 08, 2017

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data that is widely used in observational sciences. In its classic form, ICA relies on modeling the data as linear mixtures of non-Gaussian independent sources. The maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data. We introduce the Preconditioned ICA for Real Data (Picard) algorithm, which is a relative L-BFGS algorithm preconditioned with sparse Hessian approximations. Extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique, especially on real data, for which the ICA model does not necessarily hold.
Sep 08, 2017

Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort

**Click to Read Paper**

Gap Safe screening rules for sparsity enforcing penalties

Dec 27, 2017

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

Dec 27, 2017

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

Generalized Concomitant Multi-Task Lasso for sparse multimodal regression

Oct 18, 2017

Mathurin Massias, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

In high dimension, it is customary to consider Lasso-type estimators to enforce sparsity. For standard Lasso theory to hold, the regularization parameter should be proportional to the noise level, yet the latter is generally unknown in practice. A possible remedy is to consider estimators, such as the Concomitant/Scaled Lasso, which jointly optimize over the regression coefficients as well as over the noise level, making the choice of the regularization independent of the noise level. However, when data from different sources are pooled to increase sample size, or when dealing with multimodal datasets, noise levels typically differ and new dedicated estimators are needed. In this work we provide new statistical and computational solutions to deal with such heteroscedastic regression models, with an emphasis on functional brain imaging with combined magneto- and electroencephalographic (M/EEG) signals. Adopting the formulation of Concomitant Lasso-type estimators, we propose a jointly convex formulation to estimate both the regression coefficients and the (square root of the) noise covariance. When our framework is instantiated to de-correlated noise, it leads to an efficient algorithm whose computational cost is not higher than for the Lasso and Concomitant Lasso, while addressing more complex noise structures. Numerical experiments demonstrate that our estimator yields improved prediction and support identification while correctly estimating the noise (square root) covariance. Results on multimodal neuroimaging problems with M/EEG data are also reported.
Oct 18, 2017

Mathurin Massias, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

The iterative reweighted Mixed-Norm Estimate for spatio-temporal MEG/EEG source reconstruction

Jul 28, 2016

Daniel Strohmeier, Yousra Bekhti, Jens Haueisen, Alexandre Gramfort

Jul 28, 2016

Daniel Strohmeier, Yousra Bekhti, Jens Haueisen, Alexandre Gramfort

**Click to Read Paper**

GAP Safe Screening Rules for Sparse-Group-Lasso

Feb 19, 2016

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

Feb 19, 2016

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

GAP Safe screening rules for sparse multi-task and multi-class models

Nov 18, 2015

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

Nov 18, 2015

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon

**Click to Read Paper**

HRF estimation improves sensitivity of fMRI encoding and decoding models

May 13, 2013

Fabian Pedregosa, Michael Eickenberg, Bertrand Thirion, Alexandre Gramfort

May 13, 2013

Fabian Pedregosa, Michael Eickenberg, Bertrand Thirion, Alexandre Gramfort

**Click to Read Paper**

EM algorithms for ICA

May 25, 2018

Pierre Ablin, Alexandre Gramfort, Jean-François Cardoso, Francis Bach

May 25, 2018

Pierre Ablin, Alexandre Gramfort, Jean-François Cardoso, Francis Bach

**Click to Read Paper**