Models, code, and papers for "Qiang Sun":

Bayesian high-dimensional linear regression with generic spike-and-slab priors

Feb 12, 2020
Bai Jiang, Qiang Sun

Spike-and-slab priors are popular Bayesian solutions for high-dimensional linear regression problems. Previous theoretical studies on spike-and-slab methods focus on specific prior formulations and use prior-dependent conditions and analyses, and thus can not be generalized directly. In this paper, we propose a class of generic spike-and-slab priors and develop a unified framework to rigorously assess their theoretical properties. Technically, we provide general conditions under which generic spike-and-slab priors can achieve the nearly-optimal posterior contraction rate and the model selection consistency. Our results include those of Narisetty and He (2014) and Castillo et al. (2015) as special cases.

* 17 pages for main file, 13 pages for appendix 

  Access Model/Code and Paper
Modified Multidimensional Scaling and High Dimensional Clustering

Oct 24, 2018
Xiucai Ding, Qiang Sun

Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a unified framework that includes low, moderate and high dimensions, we study multidimensional scaling in the setting of clustering noisy data. Our results suggest that, in order to achieve consistent estimation of the embedding scheme, the classical multidimensional scaling needs to be modified, especially when the noise level increases. To this end, we propose {\it modified multidimensional scaling} which applies a nonlinear transformation to the sample eigenvalues. The nonlinear transformation depends on the dimensionality, sample size and unknown moment. We show that modified multidimensional scaling followed by various clustering algorithms can achieve exact recovery, i.e., all the cluster labels can be recovered correctly with probability tending to one. Numerical simulations and two real data applications lend strong support to our proposed methodology. As a byproduct, we unify and improve existing results on the $\ell_{\infty}$ bound for eigenvectors under only low bounded moment conditions. This can be of independent interest.

* 31 pages, 4 figures 

  Access Model/Code and Paper
Detection and Classification of Astronomical Targets with Deep Neural Networks in Wide Field Small Aperture Telescopes

Mar 14, 2020
Peng Jia, Qiang Liu, Yongyang Sun

Wide field small aperture telescopes are widely used for optical transient observations. Detection and classification of astronomical targets in observed images are the most important and basic step. In this paper, we propose an astronomical targets detection and classification framework based on deep neural networks. Our framework adopts the concept of the Faster R-CNN and uses a modified Resnet-50 as backbone network and a Feature Pyramid Network to extract features from images of different astronomical targets. To increase the generalization ability of our framework, we use both simulated and real observation images to train the neural network. After training, the neural network could detect and classify astronomical targets automatically. We test the performance of our framework with simulated data and find that our framework has almost the same detection ability as that of the traditional method for bright and isolated sources and our framework has 2 times better detection ability for dim targets, albeit all celestial objects detected by the traditional method can be classified correctly. We also use our framework to process real observation data and find that our framework can improve 25 % detection ability than that of the traditional method when the threshold of our framework is 0.6. Rapid discovery of transient targets is quite important and we further propose to install our framework in embedded devices such as the Nvidia Jetson Xavier to achieve real-time astronomical targets detection and classification abilities.

* Accepted by Astronomical Journal. The complete code can be downloaded from This code can be directly used to process images obtained by WFSATs. Images obtained by ordinary sky survey telescopes can also be processed with this code, however more annotated images are required to train the neural network 

  Access Model/Code and Paper
A cooperative game for automated learning of elasto-plasticity knowledge graphs and models with AI-guided experimentation

Mar 08, 2019
Kun Wang, WaiChing Sun, Qiang Du

We introduce a multi-agent meta-modeling game to generate data, knowledge, and models that make predictions on constitutive responses of elasto-plastic materials. We introduce a new concept from graph theory where a modeler agent is tasked with evaluating all the modeling options recast as a directed multigraph and find the optimal path that links the source of the directed graph (e.g. strain history) to the target (e.g. stress) measured by an objective function. Meanwhile, the data agent, which is tasked with generating data from real or virtual experiments (e.g. molecular dynamics, discrete element simulations), interacts with the modeling agent sequentially and uses reinforcement learning to design new experiments to optimize the prediction capacity. Consequently, this treatment enables us to emulate an idealized scientific collaboration as selections of the optimal choices in a decision tree search done automatically via deep reinforcement learning.

  Access Model/Code and Paper
An Analysis of Classical Multidimensional Scaling

Jan 15, 2019
Anna Little, Yuying Xie, Qiang Sun

Classical multidimensional scaling is an important tool for dimension reduction in many applications. Yet few theoretical results characterizing its statistical performance exist. In this paper, we provide a theoretical framework for analyzing the quality of embedded samples produced by classical multidimensional scaling. This lays down the foundation for various downstream statistical analysis. As an application, we study its performance in the setting of clustering noisy data. Our results provide scaling conditions on the sample size, ambient dimensionality, between-class distance and noise level under which classical multidimensional scaling followed by a clustering algorithm can recover the cluster labels of all samples with high probability. Numerical simulations confirm these scaling conditions are sharp in low, moderate, and high dimensional regimes. Applications to both human RNAseq data and natural language data lend strong support to the methodology and theory.

* 37 pages including supplementary material 

  Access Model/Code and Paper
Stochastic Training of Residual Networks: a Differential Equation Viewpoint

Dec 01, 2018
Qi Sun, Yunzhe Tao, Qiang Du

During the last few years, significant attention has been paid to the stochastic training of artificial neural networks, which is known as an effective regularization approach that helps improve the generalization capability of trained models. In this work, the method of modified equations is applied to show that the residual network and its variants with noise injection can be regarded as weak approximations of stochastic differential equations. Such observations enable us to bridge the stochastic training processes with the optimal control of backward Kolmogorov's equations. This not only offers a novel perspective on the effects of regularization from the loss landscape viewpoint but also sheds light on the design of more reliable and efficient stochastic training strategies. As an example, we propose a new way to utilize Bernoulli dropout within the plain residual network architecture and conduct experiments on a real-world image classification task to substantiate our theoretical findings.

* 20 pages, 8 figures, and 1 table 

  Access Model/Code and Paper
An Anchor-Free Region Proposal Network for Faster R-CNN based Text Detection Approaches

Apr 24, 2018
Zhuoyao Zhong, Lei Sun, Qiang Huo

The anchor mechanism of Faster R-CNN and SSD framework is considered not effective enough to scene text detection, which can be attributed to its IoU based matching criterion between anchors and ground-truth boxes. In order to better enclose scene text instances of various shapes, it requires to design anchors of various scales, aspect ratios and even orientations manually, which makes anchor-based methods sophisticated and inefficient. In this paper, we propose a novel anchor-free region proposal network (AF-RPN) to replace the original anchor-based RPN in the Faster R-CNN framework to address the above problem. Compared with a vanilla RPN and FPN-RPN, AF-RPN can get rid of complicated anchor design and achieve higher recall rate on large-scale COCO-Text dataset. Owing to the high-quality text proposals, our Faster R-CNN based two-stage text detection approach achieves state-of-the-art results on ICDAR-2017 MLT, ICDAR-2015 and ICDAR-2013 text detection benchmarks when using single-scale and single-model (ResNet50) testing only.

* Technical report 

  Access Model/Code and Paper
Nonconvex Regularized Robust Regression with Oracle Properties in Polynomial Time

Jul 09, 2019
Xiaoou Pan, Qiang Sun, Wen-Xin Zhou

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed random errors for high-dimensional adaptive Huber regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, our results suggest that adaptive Huber regression with nonconvex regularization yields statistically optimal estimators that satisfy oracle properties as if the true underlying support set were known beforehand. Computationally, we need as many as O(log s + log log d) convex relaxations to reach such oracle estimators, where s and d denote the sparsity and ambient dimension, respectively. Numerical studies lend strong support to our methodology and theory.

* 55 pages 

  Access Model/Code and Paper
Distributionally Robust Reduced Rank Regression and Principal Component Analysis in High Dimensions

Oct 18, 2018
Kean Ming Tan, Qiang Sun, Daniela Witten

We propose robust sparse reduced rank regression and robust sparse principal component analysis for analyzing large and complex high-dimensional data with heavy-tailed random noise. The proposed methods are based on convex relaxations of rank-and sparsity-constrained non-convex optimization problems, which are solved using the alternating direction method of multipliers (ADMM) algorithm. For robust sparse reduced rank regression, we establish non-asymptotic estimation error bounds under both Frobenius and nuclear norms, while existing results focus mostly on rank-selection and prediction consistency. Our theoretical results quantify the tradeoff between heavy-tailedness of the random noise and statistical bias. For random noise with bounded $(1+\delta)$th moment with $\delta \in (0,1)$, the rate of convergence is a function of $\delta$, and is slower than the sub-Gaussian-type deviation bounds; for random noise with bounded second moment, we recover the results obtained under sub-Gaussian noise. Furthermore, the transition between the two regimes is smooth. For robust sparse principal component analysis, we propose to truncate the observed data, and show that this truncation will lead to consistent estimation of the eigenvectors. We then establish theoretical results similar to those of robust sparse reduced rank regression. We illustrate the performance of these methods via extensive numerical studies and two real data applications.

  Access Model/Code and Paper
ReLaText: Exploiting Visual Relationships for Arbitrary-Shaped Scene Text Detection with Graph Convolutional Networks

Mar 16, 2020
Chixiang Ma, Lei Sun, Zhuoyao Zhong, Qiang Huo

We introduce a new arbitrary-shaped text detection approach named ReLaText by formulating text detection as a visual relationship detection problem. To demonstrate the effectiveness of this new formulation, we start from using a "link" relationship to address the challenging text-line grouping problem firstly. The key idea is to decompose text detection into two subproblems, namely detection of text primitives and prediction of link relationships between nearby text primitive pairs. Specifically, an anchor-free region proposal network based text detector is first used to detect text primitives of different scales from different feature maps of a feature pyramid network, from which a text primitive graph is constructed by linking each pair of nearby text primitives detected from a same feature map with an edge. Then, a Graph Convolutional Network (GCN) based link relationship prediction module is used to prune wrongly-linked edges in the text primitive graph to generate a number of disjoint subgraphs, each representing a detected text instance. As GCN can effectively leverage context information to improve link prediction accuracy, our GCN based text-line grouping approach can achieve better text detection accuracy than previous text-line grouping methods, especially when dealing with text instances with large inter-character or very small inter-line spacings. Consequently, the proposed ReLaText achieves state-of-the-art performance on five public text detection benchmarks, namely RCTW-17, MSRA-TD500, Total-Text, CTW1500 and DAST1500.

  Access Model/Code and Paper
Mask R-CNN with Pyramid Attention Network for Scene Text Detection

Nov 22, 2018
Zhida Huang, Zhuoyao Zhong, Lei Sun, Qiang Huo

In this paper, we present a new Mask R-CNN based text detection approach which can robustly detect multi-oriented and curved text from natural scene images in a unified manner. To enhance the feature representation ability of Mask R-CNN for text detection tasks, we propose to use the Pyramid Attention Network (PAN) as a new backbone network of Mask R-CNN. Experiments demonstrate that PAN can suppress false alarms caused by text-like backgrounds more effectively. Our proposed approach has achieved superior performance on both multi-oriented (ICDAR-2015, ICDAR-2017 MLT) and curved (SCUT-CTW1500) text detection benchmark tasks by only using single-scale and single-model testing.

* Accepted by WACV 2019 

  Access Model/Code and Paper
Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling

Oct 25, 2018
Yunzhe Tao, Qi Sun, Qiang Du, Wei Liu

Nonlocal neural networks have been proposed and shown to be effective in several computer vision tasks, where the nonlocal operations can directly capture long-range dependencies in the feature space. In this paper, we study the nature of diffusion and damping effect of nonlocal networks by doing spectrum analysis on the weight matrices of the well-trained networks, and then propose a new formulation of the nonlocal block. The new block not only learns the nonlocal interactions but also has stable dynamics, thus allowing deeper nonlocal structures. Moreover, we interpret our formulation from the general nonlocal modeling perspective, where we make connections between the proposed nonlocal network and other nonlocal models, such as nonlocal diffusion process and Markov jump process.

* Accepted by NIPS 2018 

  Access Model/Code and Paper
Graphical Nonconvex Optimization for Optimal Estimation in Gaussian Graphical Models

Jun 04, 2017
Qiang Sun, Kean Ming Tan, Han Liu, Tong Zhang

We consider the problem of learning high-dimensional Gaussian graphical models. The graphical lasso is one of the most popular methods for estimating Gaussian graphical models. However, it does not achieve the oracle rate of convergence. In this paper, we propose the graphical nonconvex optimization for optimal estimation in Gaussian graphical models, which is then approximated by a sequence of convex programs. Our proposal is computationally tractable and produces an estimator that achieves the oracle rate of convergence. The statistical error introduced by the sequential approximation using the convex programs are clearly demonstrated via a contraction property. The rate of convergence can be further improved using the notion of sparsity pattern. The proposed methodology is then extended to semiparametric graphical models. We show through numerical studies that the proposed estimator outperforms other popular methods for estimating Gaussian graphical models.

* 3 figures 

  Access Model/Code and Paper
Selective Image Super-Resolution

Oct 27, 2010
Ju Sun, Qiang Chen, Shuicheng Yan, Loong-Fah Cheong

In this paper we propose a vision system that performs image Super Resolution (SR) with selectivity. Conventional SR techniques, either by multi-image fusion or example-based construction, have failed to capitalize on the intrinsic structural and semantic context in the image, and performed "blind" resolution recovery to the entire image area. By comparison, we advocate example-based selective SR whereby selectivity is exemplified in three aspects: region selectivity (SR only at object regions), source selectivity (object SR with trained object dictionaries), and refinement selectivity (object boundaries refinement using matting). The proposed system takes over-segmented low-resolution images as inputs, assimilates recent learning techniques of sparse coding (SC) and grouped multi-task lasso (GMTL), and leads eventually to a framework for joint figure-ground separation and interest object SR. The efficiency of our framework is manifested in our experiments with subsets of the VOC2009 and MSRC datasets. We also demonstrate several interesting vision applications that can build on our system.

* 20 pages, 5 figures. Submitted to Computer Vision and Image Understanding in March 2010. Keywords: image super resolution, semantic image segmentation, vision system, vision application 

  Access Model/Code and Paper
A unified framework of predicting binary interestingness of images based on discriminant correlation analysis and multiple kernel learning

Oct 14, 2019
Qiang Sun, Liting Wang, Maohui Li, Longtao Zhang, Yuxiang Yang

In the modern content-based image retrieval systems, there is an increasingly interest in constructing a computationally effective model to predict the interestingness of images since the measure of image interestingness could improve the human-centered search satisfaction and the user experience in different applications. In this paper, we propose a unified framework to predict the binary interestingness of images based on discriminant correlation analysis (DCA) and multiple kernel learning (MKL) techniques. More specially, on the one hand, to reduce feature redundancy in describing the interestingness cues of images, the DCA or multi-set discriminant correlation analysis (MDCA) technique is adopted to fuse multiple feature sets of the same type for individual cues by taking into account the class structure among the samples involved to describe the three classical interestingness cues, unusualness,aesthetics as well as general preferences, with three sets of compact and representative features; on the other hand, to make good use of the heterogeneity from the three sets of high-level features for describing the interestingness cues, the SimpleMKL method is employed to enhance the generalization ability of the built model for the task of the binary interestingness classification. Experimental results on the publicly-released interestingness prediction data set have demonstrated the rationality and effectiveness of the proposed framework in the binary prediction of image interestingness where we have conducted several groups of comparative studies across different interestingness feature combinations, different interestingness cues, as well as different feature types for the three interestingness cues.

* 30 pages, 9 figures, 6 tables 

  Access Model/Code and Paper
Communication-efficient sparse regression: a one-shot approach

Aug 11, 2015
Jason D. Lee, Yuekai Sun, Qiang Liu, Jonathan E. Taylor

We devise a one-shot approach to distributed sparse regression in the high-dimensional setting. The key idea is to average "debiased" or "desparsified" lasso estimators. We show the approach converges at the same rate as the lasso as long as the dataset is not split across too many machines. We also extend the approach to generalized linear models.

* 29 pages, 3 figures 

  Access Model/Code and Paper
OpenHowNet: An Open Sememe-based Lexical Knowledge Base

Jan 28, 2019
Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Qiang Dong, Maosong Sun, Zhendong Dong

In this paper, we present an open sememe-based lexical knowledge base OpenHowNet. Based on well-known HowNet, OpenHowNet comprises three components: core data which is composed of more than 100 thousand senses annotated with sememes, OpenHowNet Web which gives a brief introduction to OpenHowNet as well as provides online exhibition of OpenHowNet information, and OpenHowNet API which includes several useful APIs such as accessing OpenHowNet core data and drawing sememe tree structures of senses. In the main text, we first give some backgrounds including definition of sememe and details of HowNet. And then we introduce some previous HowNet and sememe-based research works. Last but not least, we detail the constituents of OpenHowNet and their basic features and functionalities. Additionally, we briefly make a summary and list some future works.

* 4 pages, 3 figures 

  Access Model/Code and Paper
Efficient Delivery Policy to Minimize User Traffic Consumption in Guaranteed Advertising

Nov 23, 2016
Jia Zhang, Zheng Wang, Qian Li, Jialin Zhang, Yanyan Lan, Qiang Li, Xiaoming Sun

In this work, we study the guaranteed delivery model which is widely used in online display advertising. In the guaranteed delivery scenario, ad exposures (which are also called impressions in some works) to users are guaranteed by contracts signed in advance between advertisers and publishers. A crucial problem for the advertising platform is how to fully utilize the valuable user traffic to generate as much as possible revenue. Different from previous works which usually minimize the penalty of unsatisfied contracts and some other cost (e.g. representativeness), we propose the novel consumption minimization model, in which the primary objective is to minimize the user traffic consumed to satisfy all contracts. Under this model, we develop a near optimal method to deliver ads for users. The main advantage of our method lies in that it consumes nearly as least as possible user traffic to satisfy all contracts, therefore more contracts can be accepted to produce more revenue. It also enables the publishers to estimate how much user traffic is redundant or short so that they can sell or buy this part of traffic in bulk in the exchange market. Furthermore, it is robust with regard to priori knowledge of user type distribution. Finally, the simulation shows that our method outperforms the traditional state-of-the-art methods.

* Already accepted by AAAI'17 

  Access Model/Code and Paper