Add to Chrome
Add to Firefox
Add to EdgeYilang Zhang
Meta-Learning with Versatile Loss Geometries for Fast Adaptation Using Mirror Descent
Dec 20, 2023
Utilizing task-invariant prior knowledge extracted from related tasks, meta-learning is a principled framework that empowers learning a new task especially when data records are limited. A fundamental challenge in meta-learning is how to quickly "adapt" the extracted prior in order to train a task-specific model within a few optimization steps. Existing approaches deal with this challenge using a preconditioner that enhances convergence of the per-task training process. Though effective in representing locally a quadratic training loss, these simple linear preconditioners can hardly capture complex loss geometries. The present contribution addresses this limitation by learning a nonlinear mirror map, which induces a versatile distance metric to enable capturing and optimizing a wide range of loss geometries, hence facilitating the per-task training. Numerical tests on few-shot learning datasets demonstrate the superior expressiveness and convergence of the advocated approach.
Scalable Bayesian Meta-Learning through Generalized Implicit Gradients
Mar 31, 2023
Meta-learning owns unique effectiveness and swiftness in tackling emerging tasks with limited data. Its broad applicability is revealed by viewing it as a bi-level optimization problem. The resultant algorithmic viewpoint however, faces scalability issues when the inner-level optimization relies on gradient-based iterations. Implicit differentiation has been considered to alleviate this challenge, but it is restricted to an isotropic Gaussian prior, and only favors deterministic meta-learning approaches. This work markedly mitigates the scalability bottleneck by cross-fertilizing the benefits of implicit differentiation to probabilistic Bayesian meta-learning. The novel implicit Bayesian meta-learning (iBaML) method not only broadens the scope of learnable priors, but also quantifies the associated uncertainty. Furthermore, the ultimate complexity is well controlled regardless of the inner-level optimization trajectory. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one. Extensive numerical tests are also carried out to empirically validate the performance of the proposed method.
Dense Color Constancy with Effective Edge Augmentation
Nov 17, 2019
Recently, computational color constancy via convolutional neural networks (CNNs) has received much attention. In this paper, we propose a color constancy algorithm called the Dense Color Constancy (DCC), which employs a self-attention DenseNet to estimate the illuminant based on the $2$D $\log$-chrominance histograms of input images and their augmented edges. The augmented edges help to tell apart the edge and non-edge pixels in the $\log$-histogram, which largely contribute to the feature extraction and color ambiguity elimination, thereby improving the accuracy of illuminant estimation. Experiments on benchmark datasets show that the DCC algorithm is very effective for illuminant estimation compared to the state-of-the-art methods.