Demand for software-based applications has grown drastically in various real-time applications. However, software testing schemes have been developed which include manual and automatic testing. Manual testing requires human effort and chances of error may still affect the quality of software. To overcome this issue, automatic software testing techniques based on machine learning techniques have been developed. In this work, we focus on the machine learning scheme for early prediction of software defects using Levenberg-Marquardt algorithm (LM), Back Propagation (BP) and Bayesian Regularisation (BR) techniques. Bayesian regularisation achieves better performance in terms of bug prediction. However, this performance can be enhanced further. Hence, we developed a novel approach for attribute selection-based feature selection technique to improve the performance of BR classification. An extensive study is carried out with the PROMISE repository where we considered KC1 and JM1 datasets. Experimental study shows that the proposed approach achieves better performance in predicting the defects in software.