MTLP-JR: Multi-task learning-based prediction for joint ranking in neural architecture search

Abstract

At present, great attentions have been paid to multi-objective neural architecture search (NAS) and resource-aware NAS for their comprehensive consideration of the overall evaluation of architectures, including inference latency, precision, and model scale. However NAS also exacerbates the ever-increasing cost (engineering, time complexity, computation resource). Aiming to alleviate this, the reproducible NAS research releases the benchmark, which includes the metrics (e.g. Accuracy, Latency, and Parameters) of representative models from the typical search space on specific tasks. Motivated by the multi-objective NAS, resource-aware NAS, and reproducible NAS, this paper dedicates to binary-relation prediction (Latency, Accuracy), which is a more reasonable and effective way to satisfy the general NAS scenarios with less cost. We conduct a reproducible NAS study on the MobileNet-based search space and release the dataset. Further, we first propose the modeling of common features among prediction tasks (Latency, Accuracy, Parameters, and FLOPs), which will facilitate the prediction of individual tasks, and creatively formulate the architecture ranking prediction with a multi-task learning framework. Eventually, the proposed multi-task learning based binary-relation prediction model reaches the performance of 94.3% on Latency and 85.02% on Top1 Accuracy even with only 100 training points, which outperforms the single-task learning based model.