Drop In Accuracy Research Articles

The Impact of Device Uniformity on Functionality of Analog Passively-Integrated Memristive Circuits

Passively-integrated memristors are the most prospective candidates for designing high-speed, energy-efficient, and compact neuromorphic circuits. Despite all the promising properties, experimental demonstrations of passive memristive crossbars have been limited to circuits with few thousands of devices until now, which stems from the strict uniformity requirements on the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IV</i> characteristics of memristors. This paper expands upon this vital challenge and investigates how uniformity impacts the computing accuracy of analog memristive circuits, focusing on neuromorphic applications. Specifically, the paper explores the tradeoffs between computing accuracy, crossbar size, switching threshold variations, and target precision. All-embracing simulations of matrix multipliers and deep neural networks on CIFAR-10 and ImageNet datasets have been carried out to evaluate the role of uniformity on the accuracy of computing systems. Further, we study three post-fabrication methods that increase the accuracy of nonuniform 0T1R neuromorphic circuits: hardware-aware training, improved tuning algorithm, and switching threshold modification. The application of these techniques allows us to implement advanced deep neural networks with almost no accuracy drop, using current state-of-the-art analog 0T1R technology.

EFFORT: A Comprehensive Technique to Tackle Timing Violations and Improve Energy Efficiency of Near-Threshold Tensor Processing Units

Modern deep neural network (DNN) applications demand a remarkable processing throughput usually unmet by traditional Von Neumann architectures. Consequently, hardware accelerators, comprising a sea of multiplier-and-accumulate (MAC) units, have recently gained prominence in accelerating DNN inference engine. For example, tensor processing units (TPUs) account for a lion’s share of Google’s datacenter inference operations. The proliferation of real-time DNN predictions is accompanied by a tremendous energy budget. In quest of trimming the energy footprint of DNN accelerators, we propose Energy eFFicient and errOr Resilient TPU (EFFORT)—an energy optimized, yet high-performance TPU architecture, operating at the near-threshold computing (NTC) region. EFFORT promotes a better-than-worst case design by operating the NTC TPU at a substantially high frequency while keeping the voltage at the NTC nominal value. In order to tackle the timing errors due to such aggressive operation, we employ an opportunistic error mitigation strategy. In addition, we implement an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in situ</i> clock gating architecture, drastically reducing the MACs’ dynamic power consumption. Compared to a cutting-edge error mitigation technique for TPUs, EFFORT enables up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.5\times $ </tex-math></inline-formula> better performance at NTC with only 4% average accuracy drop across six out of eight DNN benchmarks.

A computational approach for progressive architecture shrinkage in action recognition

AbstractEfficiency plays a key role in video understanding modeling, and developing more efficient spatiotemporal deep networks is a key ingredient for enabling their usage in production scenarios. In this work, we propose a methodology for reducing the computational complexity of a video understanding backbone while limiting the drop in accuracy caused by architectural changes. Our approach, named, Progressive Architecture Shrinkage, applies a sequence of reduction operators to the hyperparameters of a network to reduce its computational footprint. The choice of the sequence of operations is automatically optimized in a coordinate‐descent schema, and the approach transfers knowledge from both the initial network and previous stages of the shrinking process by employing a Knowledge Distillation and an adaptive fine‐tuning strategy. As each iteration of the shrinking algorithm requires to train a large‐scale video understanding network, we perform experiments on MARCONI 100—a supercomputer equipped with an IBM Power9 architecture and Volta NVIDIA GPUs. Experimental evaluations are conducted using two backbones and three different action recognition benchmarks. We show that, through our approach, high accuracy levels can be maintained while reducing the number of multiply–adds operations by four times with respect to the original architectures. Code will be made available.

Impact of HKMG and FDSOI FeFET drain current variation in processing-in-memory architectures

In this study, we analyze the impact of drain current ( $$ I_{{{\text{DS}}}} $$ ) variation in 28 nm high-K metal-gate and 22 nm fully-depleted silicon-on-insulator Ferroelectric FET devices on processing-in-memory (PIM) deep neural network (DNN) accelerators. When performing repeated read operations on several devices at various read frequencies and under various biasing and programming conditions, non-Normal variation in $$ I_{{{\text{DS}}}} $$ is observed. Device-circuit co-analysis is used to emulate PIM performance subject to noise when classifying images. Marginal degradation is observed in Fashion-MNIST classification accuracy using LeNet-5, and more significant degradation is observed in CIFAR-10 classification accuracy using MobileNetV2. Variation-aware training is shown to fully recover minor drops in LeNet-5 accuracy but becomes difficult for large workloads like MobileNetV2. We demonstrate that $$ I_{{{\text{DS}}}} $$ variation in individual FeFETs over many read cycles is not prohibitive to designing DNN accelerators with small workloads, but advanced design techniques are required to mitigate error for larger workloads.

Tolerating Defects in Low-Power Neural Network Accelerators Via Retraining-Free Weight Approximation

Hardware accelerators are essential to the accommodation of ever-increasing Deep Neural Network (DNN) workloads on the resource-constrained embedded devices. While accelerators facilitate fast and energy-efficient DNN operations, their accuracy is threatened by faults in their on-chip and off-chip memories, where millions of DNN weights are held. The use of emerging Non-Volatile Memories (NVM) further exposes DNN accelerators to a non-negligible rate of permanent defects due to immature fabrication, limited endurance, and aging. To tolerate defects in NVM-based DNN accelerators, previous work either requires extra redundancy in hardware or performs defect-aware retraining, imposing significant overhead. In comparison, this paper proposes a set of algorithms that exploit the flexibility in setting the fault-free bits in weight memory to effectively approximate weight values, so as to mitigate defect-induced accuracy drop. These algorithms can be applied as a one-step solution when loading the weights to embedded devices. They only require trivial hardware support and impose negligible run-time overhead. Experiments on popular DNN models show that the proposed techniques successfully boost inference accuracy even in the face of elevated defect rates in the weight memory.

Pipelined Training with Stale Weights in Deep Convolutional Neural Networks

The growth in size and complexity of convolutional neural networks (CNNs) is forcing the partitioning of a network across multiple accelerators during training and pipelining of backpropagation computations over these accelerators. Pipelining results in the use of stale weights. Existing approaches to pipelined training avoid or limit the use of stale weights with techniques that either underutilize accelerators or increase training memory footprint. This paper contributes a pipelined backpropagation scheme that uses stale weights to maximize accelerator utilization and keep memory overhead modest. It explores the impact of stale weights on the statistical efficiency and performance using 4 CNNs (LeNet-5, AlexNet, VGG, and ResNet) and shows that when pipelining is introduced in early layers, training with stale weights converges and results in models with comparable inference accuracies to those resulting from nonpipelined training (a drop in accuracy of 0.4%, 4%, 0.83%, and 1.45% for the 4 networks, respectively). However, when pipelining is deeper in the network, inference accuracies drop significantly (up to 12% for VGG and 8.5% for ResNet-20). The paper also contributes a hybrid training scheme that combines pipelined with nonpipelined training to address this drop. The potential for performance improvement of the proposed scheme is demonstrated with a proof-of-concept pipelined backpropagation implementation in PyTorch on 2 GPUs using ResNet-56/110/224/362, achieving speedups of up to 1.8X over a 1-GPU baseline.

Radial Basis Function Network with Differential Privacy

Differential privacy (DP) remains a potent solution to what is arguably the defining issue in machine learning: balancing user privacy with an ever-increasing need for data. Practitioners must respect privacy, especially in sensitive healthcare domains. DP strives towards this aim by adding noise to training data to occlude its origin and nature, and is ideal for multiple Neural Network (NN) types. This includes deep varieties that utilise multiple hidden layers, and shallow ones with single hidden layers such as the Radial Base Function Network (RBFN). The work herein explores DP within this context by devising a model that leverages Gaussian RBF parameters to add privacy during training. Our model’s efficacy is examined against two real and three synthetic datasets, with results showing reasonable trade-offs between accuracy and privacy. With high intra-class variation, we retained 100% accuracy for two synthetic datasets and a drop of only 1.72% for another. If privacy is prioritised with low intra-class variation, we achieved accuracy drops of 8%–23% with an inherited epsilon that never exceeds one, indicating a good privacy guarantee. We also show that timely training is achievable on a high-dimensional dataset consisting of 2M records and 170 features.

An unsupervised domain adaptation approach for change detection and its application to deforestation mapping in tropical biomes

Changes in environmental conditions, geographical variability and different sensor properties typically make it almost impossible to employ previously trained classifiers for new data without a significant drop in classification accuracy. Domain adaptation (DA) techniques been proven useful to alleviate that problem. In particular, appearance adaptation techniques may be used to adapt images from a specific dataset in such a way that the generated images have a style that is similar to the images from another dataset. Such techniques are, however, prone to creating artifacts that hinder proper classification of the adapted images. In this work we propose an unsupervised DA approach for change detection tasks, which is based on a particular appearance adaptation method: the Cycle-Consistent Generative Adversarial Network (CycleGAN). Specifically, we extend that method by introducing additional constraints in the training phase of the model components, which make it preserve the semantic structure and class transitions in the adapted images. We evaluate the proposed approach on a deforestation detection application, considering different sites in the Amazon rain-forest and in the Brazilian Cerrado (savanna) using Landsat-8 images. In the experiments, each site corresponds to a domain, and the accuracy of a classifier trained with images and references from one (source) domain is measured in the classification of another (target) domain. The results show that the proposed approach is successful in producing artifact-free adapted images, which can be satisfactory classified by the pre-trained source classifiers. On average, the accuracies achieved in the classification of the adapted images outperformed the baselines (when no adaptation was made) by 7.1% in terms of mean average precision, and 9.1% in terms of F1-Score. To the best of our knowledge, the proposed method is the first unsupervised domain adaptation approach devised for change detection.

REHASH

Battery-free sensing devices harvest energy from their surrounding environment to perform sensing, computation, and communication. This enables previously impossible applications in the Internet-of-Things. A core challenge for these devices is maintaining usefulness despite erratic, random or irregular energy availability; which causes inconsistent execution, loss of service and power failures. Adapting execution (degrading or upgrading) seems promising as a way to stave off power failures, meet deadlines, or increase throughput. However, because of constrained resources and limited local information, it is a challenge to decide when would be the best time to adapt, and how exactly to adapt execution. In this paper, we systematically explore the fundamental mechanisms of energy-aware adaptation, and propose heuristic adaptation as a method for modulating the performance of tasks to enable higher sensor coverage, completion rates, or throughput, depending on the application. We build a task based adaptive runtime system for intermittently powered sensors embodying this concept. We complement this runtime with a user facing simulator that enables programmers to conceptualize the tradeoffs they make when choosing what tasks to adapt, and how, relative to real world energy harvesting environment traces. While we target battery-free, intermittently powered sensors, we see general application to all energy harvesting devices. We explore heuristic adaptation with varied energy harvesting modalities and diverse applications: machine learning, activity recognition, and greenhouse monitoring, and find that the adaptive version of our ML app performs up to 46% more classifications with only a 5% drop in accuracy; the activity recognition app captures 76% more classifications with only nominal down-sampling; and find that heuristic adaptation leads to higher throughput versus non-adaptive in all cases.

Detection of COVID-19 from Chest X-ray Images Using Deep Convolutional Neural Networks.

The COVID-19 global pandemic has wreaked havoc on every aspect of our lives. More specifically, healthcare systems were greatly stretched to their limits and beyond. Advances in artificial intelligence have enabled the implementation of sophisticated applications that can meet clinical accuracy requirements. In this study, customized and pre-trained deep learning models based on convolutional neural networks were used to detect pneumonia caused by COVID-19 respiratory complications. Chest X-ray images from 368 confirmed COVID-19 patients were collected locally. In addition, data from three publicly available datasets were used. The performance was evaluated in four ways. First, the public dataset was used for training and testing. Second, data from the local and public sources were combined and used to train and test the models. Third, the public dataset was used to train the model and the local data were used for testing only. This approach adds greater credibility to the detection models and tests their ability to generalize to new data without overfitting the model to specific samples. Fourth, the combined data were used for training and the local dataset was used for testing. The results show a high detection accuracy of 98.7% with the combined dataset, and most models handled new data with an insignificant drop in accuracy.

Enhancing long tail item recommendation in collaborative filtering: An econophysics-inspired approach

• A novel viewpoint of econophysics principles is formulated to address long tail effect. • Rating injection strategy inspired from econophysics principles is proposed. • Performance of the proposed method is evaluated on long tail and non-long tail items. • Proposed approach immensely improved the number of long tail items in recommendations. • The proposed principles can be applied as an add-on to state-of-the-art recommendation technique. Recommender systems have been immensely successful in overcoming information overload problem through personalized suggestions to consumers. Traditional recommendation algorithms tend to recommend more popular items. A significant number of items in an enterprise are non-popular (long tail items) due to lack of visibility in recommendations. These long tail items are left unsold and result in a significant loss to the business. The consumers on the other end are deprived of receiving relevant item recommendations. In this paper, we propose two approaches inspired from econophysics to recommend long tail items. The proposed approaches selectively inject ratings to the long tail items to diminish the bias towards the popular items by utilizing the existing rating information. Subsequently, the injected rating datasets are used to provide recommendations. The results on real-world datasets show that the proposed approaches outperform the existing techniques in mitigating long tail effect with little or no drop in accuracy.

American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation.

Sign language is designed to assist the deaf and hard of hearing community to convey messages and connect with society. Sign language recognition has been an important domain of research for a long time. Previously, sensor-based approaches have obtained higher accuracy than vision-based approaches. Due to the cost-effectiveness of vision-based approaches, researchers have been conducted here also despite the accuracy drop. The purpose of this research is to recognize American sign characters using hand images obtained from a web camera. In this work, the media-pipe hands algorithm was used for estimating hand joints from RGB images of hands obtained from a web camera and two types of features were generated from the estimated coordinates of the joints obtained for classification: one is the distances between the joint points and the other one is the angles between vectors and 3D axes. The classifiers utilized to classify the characters were support vector machine (SVM) and light gradient boosting machine (GBM). Three character datasets were used for recognition: the ASL Alphabet dataset, the Massey dataset, and the finger spelling A dataset. The results obtained were 99.39% for the Massey dataset, 87.60% for the ASL Alphabet dataset, and 98.45% for Finger Spelling A dataset. The proposed design for automatic American sign language recognition is cost-effective, computationally inexpensive, does not require any special sensors or devices, and has outperformed previous studies.

Mental Workload Estimation Based on Physiological Features for Pilot-UAV Teaming Applications.

Manned-Unmanned Teaming (MUM-T) can be defined as the teaming of aerial robots (artificial agents) along with a human pilot (natural agent), in which the human agent is not an authoritative controller but rather a cooperative team player. To our knowledge, no study has yet evaluated the impact of MUM-T scenarios on operators' mental workload (MW) using a neuroergonomic approach (i.e., using physiological measures), nor provided a MW estimation through classification applied on those measures. Moreover, the impact of the non-stationarity of the physiological signal is seldom taken into account in classification pipelines, particularly regarding the validation design. Therefore this study was designed with two goals: (i) to characterize and estimate MW in a MUM-T setting based on physiological signals; (ii) to assess the impact of the validation procedure on classification accuracy. In this context, a search and rescue (S&R) scenario was developed in which 14 participants played the role of a pilot cooperating with three UAVs (Unmanned Aerial Vehicles). Missions were designed to induce high and low MW levels, which were evaluated using self-reported, behavioral and physiological measures (i.e., cerebral, cardiac, and oculomotor features). Supervised classification pipelines based on various combinations of these physiological features were benchmarked, and two validation procedures were compared (i.e., a traditional one that does not take time into account vs. an ecological one that does). The main results are: (i) a significant impact of MW on all measures, (ii) a higher intra-subject classification accuracy (75%) reached using ECG features alone or in combination with EEG and ET ones with the Adaboost, Linear Discriminant Analysis or the Support Vector Machine classifiers. However this was only true with the traditional validation. There was a significant drop in classification accuracy using the ecological one. Interestingly, inter-subject classification with ecological validation (59.8%) surpassed both intra-subject with ecological and inter-subject with traditional validation. These results highlight the need for further developments to perform MW monitoring in such operational contexts.

Oil palm fresh fruit bunch ripeness classification on mobile devices using deep learning approaches

The implementations of deep learning combined with other methods such as transfer learning and data augmentation in oil palm fresh fruit bunch (FFB) ripeness classification have been researched throughout the years. However, most of the methods require devices with high computational resources which could not be implemented in mobile applications. To overcome this problem, this research would focus on creating a mobile application to classify the ripeness levels of oil palm FFB using lightweight Convolutional Neural Network (CNN). In this research, we implemented ImageNet transfer learning on 4 lightweight CNN models with a novel data augmentation method named “9-angle crop”, which would be further optimized using post-training quantization. Transfer learning with 3 unfrozen convolution blocks and 9 angle crop successfully increased the classification accuracy on MobileNetV1, and when it was compared to other lightweight models, EfficientNetB0 performed best with 0.898 test accuracy on Keras. Float16 quantization also proved to be the most suitable post-training quantization method for this model, halving the size of EfficientNetB0 with the least increase in image classification time and an accuracy drop of only 0.005 after the model was converted to TensorFlow Lite interpreter. In conclusion, the best model created in this research is EfficientNetB0, trained with the combination of transfer learning, 9 angle crop, and float16 quantization, which enabled it to achieve an overall test accuracy of 0.893 on TensorFlow Lite with 96 ms classification time per image, far surpassing the other 3 compared models with the second-best model being MobileNetV1 with 0.811 accuracy. The model itself was able to achieve similar results when it was implemented on an Android application to classify the ripeness levels of oil palm FFB images obtained through live camera input.

A Slight Increase in Reverberation Time in the Classroom Affects Performance and Behavioral Listening Effort.

The purpose of this study was to investigate the effect of a small change in reverberation time (from 0.57 to 0.69 s) in a classroom on children's performance and listening effort. Aiming for ecological listening conditions, the change in reverberation time was combined with the presence or absence of classroom noise. In three academic tasks, the study examined whether the effect of reverberation was modulated by the presence of noise and depended on the children's age. A total of 302 children (aged 11-13 years, grades 6-8) with normal hearing participated in the study. Three typical tasks of daily classroom activities (speech perception, sentence comprehension, and mental calculation) were administered to groups of children in two listening conditions (quiet and classroom noise). The experiment was conducted inside real classrooms, where reverberation time was controlled. The outcomes considered were task accuracy and response times (RTs), the latter taken as a behavioral proxy for listening effort. Participants were also assessed on reading comprehension and math fluency. To investigate the impact of noise and/or reverberation, these two scores were entered in the statistical model to control for individual child's general academic abilities. While the longer reverberation time did not significantly affect accuracy or RTs under the quiet condition, it had several effects when in combination with classroom noise, depending on the task measured. A significant drop in accuracy with a longer reverberation time emerged for the speech perception task, but only for the grade 6 children. The effect on accuracy of a longer reverberation time was nonsignificant for sentence comprehension (always at ceiling), and depended on the children's age in the mental calculation task. RTs were longer for moderate than for short reverberation times in the speech perception and sentence comprehension tasks, while there was no significant effect of the different reverberation times on RTs in the mental calculation task. The results indicate small, but statistically significant, effects of a small change in reverberation time on listening effort as well as accuracy for children aged 11 to 13 performing typical tasks of daily classroom activities. Thus, the results extend previous findings in adults to children as well. The findings also contribute to a better understanding of the practical implications and importance of optimal ranges of reverberation time in classrooms. A comparison with previous studies underscored the importance of early reflections as well as reverberation times in classrooms.

Designing an encoder for StyleGAN image manipulation

Recently, there has been a surge of diverse methods for performing image editing by employing pre-trained unconditional generators. Applying these methods on real images, however, remains a challenge, as it necessarily requires the inversion of the images into their latent space. To successfully invert a real image, one needs to find a latent code that reconstructs the input image accurately, and more importantly, allows for its meaningful manipulation. In this paper, we carefully study the latent space of StyleGAN, the state-of-the-art unconditional generator. We identify and analyze the existence of a distortion-editability tradeoff and a distortion-perception tradeoff within the StyleGAN latent space. We then suggest two principles for designing encoders in a manner that allows one to control the proximity of the inversions to regions that StyleGAN was originally trained on. We present an encoder based on our two principles that is specifically designed for facilitating editing on real images by balancing these tradeoffs. By evaluating its performance qualitatively and quantitatively on numerous challenging domains, including cars and horses, we show that our inversion method, followed by common editing techniques, achieves superior real-image editing quality, with only a small reconstruction accuracy drop.

Improving motor imagery classification during induced motor perturbations

Objective. Motor imagery is the mental simulation of movements. It is a common paradigm to design brain-computer interfaces (BCIs) that elicits the modulation of brain oscillatory activity similar to real, passive and induced movements. In this study, we used peripheral stimulation to provoke movements of one limb during the performance of motor imagery tasks. Unlike other works, in which induced movements are used to support the BCI operation, our goal was to test and improve the robustness of motor imagery based BCI systems to perturbations caused by artificially generated movements. Approach. We performed a BCI session with ten participants who carried out motor imagery of three limbs. In some of the trials, one of the arms was moved by neuromuscular stimulation. We analysed 2-class motor imagery classifications with and without movement perturbations. We investigated the performance decrease produced by these disturbances and designed different computational strategies to attenuate the observed classification accuracy drop. Main results. When the movement was induced in a limb not coincident with the motor imagery classes, extracting oscillatory sources of the movement imagination tasks resulted in BCI performance being similar to the control (undisturbed) condition; when the movement was induced in a limb also involved in the motor imagery tasks, the performance drop was significantly alleviated by spatially filtering out the neural noise caused by the stimulation. We also show that the loss of BCI accuracy was accompanied by weaker power of the sensorimotor rhythm. Importantly, this residual power could be used to predict whether a BCI user will perform with sufficient accuracy under the movement disturbances. Significance. We provide methods to ameliorate and even eliminate motor related afferent disturbances during the performance of motor imagery tasks. This can help improving the reliability of current motor imagery based BCI systems.

Designing an encoder for StyleGAN image manipulation

Recently, there has been a surge of diverse methods for performing image editing by employing pre-trained unconditional generators. Applying these methods on real images, however, remains a challenge, as it necessarily requires the inversion of the images into their latent space. To successfully invert a real image, one needs to find a latent code that reconstructs the input image accurately, and more importantly, allows for its meaningful manipulation. In this paper, we carefully study the latent space of StyleGAN, the state-of-the-art unconditional generator. We identify and analyze the existence of a distortion-editability tradeoff and a distortion-perception tradeoff within the StyleGAN latent space. We then suggest two principles for designing encoders in a manner that allows one to control the proximity of the inversions to regions that StyleGAN was originally trained on. We present an encoder based on our two principles that is specifically designed for facilitating editing on real images by balancing these tradeoffs. By evaluating its performance qualitatively and quantitatively on numerous challenging domains, including cars and horses, we show that our inversion method, followed by common editing techniques, achieves superior real-image editing quality, with only a small reconstruction accuracy drop.

Individualized prediction of three- and six-year outcomes of psychosis in a longitudinal multicenter study: a machine learning approach

Schizophrenia and related disorders have heterogeneous outcomes. Individualized prediction of long-term outcomes may be helpful in improving treatment decisions. Utilizing extensive baseline data of 523 patients with a psychotic disorder and variable illness duration, we predicted symptomatic and global outcomes at 3-year and 6-year follow-ups. We classified outcomes as (1) symptomatic: in remission or not in remission, and (2) global outcome, using the Global Assessment of Functioning (GAF) scale, divided into good (GAF ≥ 65) and poor (GAF < 65). Aiming for a robust and interpretable prediction model, we employed a linear support vector machine and recursive feature elimination within a nested cross-validation design to obtain a lean set of predictors. Generalization to out-of-study samples was estimated using leave-one-site-out cross-validation. Prediction accuracies were above chance and ranged from 62.2% to 64.7% (symptomatic outcome), and 63.5–67.6% (global outcome). Leave-one-site-out cross-validation demonstrated the robustness of our models, with a minor drop in predictive accuracies of 2.3% on average. Important predictors included GAF scores, psychotic symptoms, quality of life, antipsychotics use, psychosocial needs, and depressive symptoms. These robust, albeit modestly accurate, long-term prognostic predictions based on lean predictor sets indicate the potential of machine learning models complementing clinical judgment and decision-making. Future model development may benefit from studies scoping patient’s and clinicians' needs in prognostication.

Reducing explicit word vectors dimensions using BPSO-based labeling algorithm and voting method

Interpretability of word vector components is very important for obtaining conceptual relations. Word vectors derived from counting models are interpretable but suffer from the high-dimensionality problem. Our goal in this study is to obtain interpretable low-dimensional word vectors in such a way that the least accuracy loss occurs. To achieve this goal, we propose an approach to reduce the dimensions of word vectors using a labeling method based on the BPSO algorithm and a voting method for selecting final context words. In this approach, we define several different base models to solve the labeling problem using different data and different objective functions. Then, we train each base model and select 3 of the best solutions for each model. We create the target word vectors of the dictionary based on the context words labeled 1. Next, we use the three best solutions of each base model to build the ensemble. After creating the ensemble, we use the voting method to assign the final label to the primary context words and select N final context words. In this study, we use the corpus ukWaC to construct word vectors. We evaluate the resulting word vectors on the MEN, RG-65, and SimLex-999 test sets. The evaluation results show that by reducing the word vectors dimensions from 5000 to 1507, the Spearman correlation coefficient of the proposed approach has been reduced to a lesser extent compared to each base model. Therefore, the accuracy drop of the proposed approach is justified after reducing the dimensions from 5000 to 1507. It is not a large penalty because the resulting word vectors are low-dimensional and interpretable.