Designing high-performance adders and multiplier components for diverse specifications and constraints is of practical concern. However, selecting the best architecture for adder or multiplier, which largely affects the performance of synthesized circuits, is difficult. To tackle this difficulty, a machine-learning-driven approach is proposed for automatic architectural selection of adders and multipliers. It trains a machine learning model for classification through learning a number of existing design schemes and their performance data. Experimental results show that the proposed approach based on a multi-perception neural network achieves as high as 94% prediction accuracy with negligible inference time. On a CPU server, the proposed approach runs about 4× faster than a brute-force approach trying four candidate architectures and consumes 10%~20% less runtime than the DesignWare datapath generator for obtaining the optimal adder/multiplier circuit. The adder (multiplier) generated with the proposed approach achieves performance metrics close to the optimal and has 1.6% (5.2%) less area and 2.2% (7.1%) more worst negative slack averagely than that generated with the DesignWare datapath generator. Our experiment also shows that the proposed approach is not sensitive to the size of training subset.