This research introduces the Boosted Ensemble deep Multiple Layer Perceptron (EdMLP) architecture with multiple output layers, a novel enhancement for the traditional Multi-Layer Perceptron (MLP). By adopting a layer-wise training approach, EdMLP enables the integration of boosting techniques within a single model, treating each layer as a weak learner, resulting in substantial performance gains. Additionally, the inclusion of layer-wise hyperparameter tuning allows us to optimize individual layers thereby reducing the tuning time. Furthermore, the ensemble deep architecture’s versatility can be extended to other neural network-based models, such as the Self Normalized Network (SNN) model where experiments demonstrate substantial performance enhancements yielded by the EdSNN compared to standard the original SNN model. This research underscores the potential of the EdMLP, and the Ed architecture in general as a powerful tool for improving the performance of various multilayer feedforward neural network models.