This article presents an extended Parameterized Fuzzy Semi-supervised learning (PFSL) method, in which the key innovation is the capability of separating a sample set into two independent subsets: outlier sample subset and regular sample subset. In our proposed PFSL, we first develop an improved parameterized Fuzzy Linear Discriminant Analysis (F-LDA) algorithm to classify regular samples, in which the distribution information of each sample in terms of fuzzy membership degree is incorporated with the redefined within-class and between-class scatter matrices. To achieve good parameter estimation for this improved F-LDA, we advocate the use of Hopfield Neural Networks (HNN) due to its efficiency. Second, a new semi-supervised Fuzzy C-Means (S-FCM) algorithm is designed using pre-computed cluster number and cluster centers in the supervised pattern discovery stage. It is applied to classify the remaining outlier samples and generate the final classification result. Third, since Kernel Fisher Discriminant (KFD) is an efficient way to extract nonlinear discriminant features, a kernel version of the proposed PFSL (K-PFSL) is discussed. Extensive experiments on the ORL, NUST603, FERET and Yale face datasets show the effectiveness and the superiority of the proposed algorithm.