Fuzzy clustering-based neural network based on linear fitting residual-driven weighted fuzzy clustering and convolutional regularization strategy

Abstract

In this study, a reinforced Fuzzy Clustering-based Neural Network (FCNN) is introduced as an augmented FCNN architecture to address regression issues. It is widely recognized that regardless of the design method and rules employed by a fuzzy model, the determination of fuzzy sets remains a crucial aspect. FCNN and its improved variants utilize conventional fuzzy clustering algorithms to partition the feature space into fuzzy sets. However, this approach tends to disregard the distinctions inherent in data patterns. Although FCNN is a nonlinear model in relation to the input variables, it is a linear model with respect to the parameters that need to be estimated. Inspired by this, our method incorporates a pre-training phase where we utilize sample residuals from a linear regression algorithm to measure differences between data patterns. These differences are subsequently integrated into the fuzzy partition, yielding more refined fuzzy sets. To combat overfitting that can degrade the model’s predictive capability, we introduce a convolutional L2 regularization strategy that integrates the convolution operator from harmonic analysis into the construction of L2 regularization. Compared to conventional L2 regularization, this convolutional regularization strategy is more effective in improving the regularity of the design matrix, thereby reducing the variation between coefficients and enhancing the model’s generalization ability. The efficacy of the presented method is substantiated by experimental studies conducted on both synthetic and real-world datasets.