Proceedings of the ACM on Programming Languages | VOL. 6
Read

Neural architecture search using property guided synthesis

Publication Date Oct 31, 2022

Abstract

Neural architecture search (NAS) has become an increasingly important tool within the deep learning community in recent years, yielding many practical advancements in the design of deep neural network architectures. However, most existing approaches operate within highly structured design spaces, and hence (1) explore only a small fraction of the full search space of neural architectures while also (2) requiring significant manual effort from domain experts. In this work, we develop techniques that enable efficient NAS in a significantly larger design space. In particular, we propose to perform NAS in an abstract search space of program properties. Our key insights are as follows: (1) an abstract search space can be significantly smaller than the original search space, and (2) architectures with similar program properties should also have similar performance; thus, we can search more efficiently in the abstract search space. To enable this approach, we also introduce a novel efficient synthesis procedure, which performs the role of concretizing a set of promising program properties into a satisfying neural architecture. We implement our approach, αNAS, within an evolutionary framework, where the mutations are guided by the program properties. Starting with a ResNet-34 model, αNAS produces a model with slightly improved accuracy on CIFAR-10 but 96% fewer parameters. On ImageNet, αNAS is able to improve over Vision Transformer (30% fewer FLOPS and parameters), ResNet-50 (23% fewer FLOPS, 14% fewer parameters), and EfficientNet...

Concepts

Neural Architecture Neural Architecture Search Original Search Space Search Space Deep Learning Community Design Of Neural Network Architectures Degradation In Accuracy Evolutionary Framework Design Of Network Architectures Domain Experts

Round-ups are the summaries of handpicked papers around trending topics published every week. These would enable you to scan through a collection of papers and decide if the paper is relevant to you before actually investing time into reading it.

Climate change Research Articles published between Jan 16, 2023 to Jan 22, 2023

R DiscoveryJan 23, 2023
R DiscoveryArticles Included:  5

Carbon capture and utilization (CCU) is an emerging technology with commercial potential to convert atmospheric carbon dioxide (CO2) into net zero or ...

Read More

Quality Of Education Research Articles published between Jan 16, 2023 to Jan 22, 2023

R DiscoveryJan 23, 2023
R DiscoveryArticles Included:  2

Introduction: The Educational Scholar Program (ESP) is a creative method to focus on the quality of education and the scholarship of education. The ed...

Read More

Gender Equality Research Articles published between Jan 16, 2023 to Jan 22, 2023

R DiscoveryJan 23, 2023
R DiscoveryArticles Included:  5

This study deviates from the predominantly feminist/critical school of thought associated with existing gender studies to apply an interpretivist appr...

Read More

Coronavirus Pandemic

You can also read COVID related content on R COVID-19

R ProductsCOVID-19

ONE PROBLEM . ONE PURPOSE . ONE PLACE

Creating the world’s largest AI-driven & human-curated collection of research, news, expert recommendations and educational resources on COVID-19

COVID-19 Dashboard

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on “as is” basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The Copyright Law.