ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies

Add code
Oct 26, 2021
Yu Shen, Yang Li, Jian Zheng, Wentao Zhang, Peng Yao, Jixiang Li, Sen Yang, Ji Liu, Bin Cui

Figure 1 for ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies
Figure 2 for ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies
Figure 3 for ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies
Figure 4 for ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

Designing neural architectures requires immense manual efforts. This has promoted the development of neural architecture search (NAS) to automate this design. While previous NAS methods achieve promising results but run slowly and zero-cost proxies run extremely fast but are less promising, recent work considers utilizing zero-cost proxies via a simple warm-up. The existing method has two limitations, which are unforeseeable reliability and one-shot usage. To address the limitations, we present ProxyBO, an efficient Bayesian optimization framework that utilizes the zero-cost proxies to accelerate neural architecture search. We propose the generalization ability measurement to estimate the fitness of proxies on the task during each iteration and then combine BO with zero-cost proxies via dynamic influence combination. Extensive empirical studies show that ProxyBO consistently outperforms competitive baselines on five tasks from three public benchmarks. Concretely, ProxyBO achieves up to 5.41x and 3.83x speedups over the state-of-the-art approach REA and BRP-NAS, respectively.

View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon