Random Weights Research Articles

RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning.

PurposeTo demonstrate the value of pretraining with millions of radiologic images compared with ImageNet photographic images on downstream medical applications when using transfer learning.Materials and MethodsThis retrospective study included patients who underwent a radiologic study between 2005 and 2020 at an outpatient imaging facility. Key images and associated labels from the studies were retrospectively extracted from the original study interpretation. These images were used for RadImageNet model training with random weight initiation. The RadImageNet models were compared with ImageNet models using the area under the receiver operating characteristic curve (AUC) for eight classification tasks and using Dice scores for two segmentation problems.ResultsThe RadImageNet database consists of 1.35 million annotated medical images in 131 872 patients who underwent CT, MRI, and US for musculoskeletal, neurologic, oncologic, gastrointestinal, endocrine, abdominal, and pulmonary pathologic conditions. For transfer learning tasks on small datasets—thyroid nodules (US), breast masses (US), anterior cruciate ligament injuries (MRI), and meniscal tears (MRI)—the RadImageNet models demonstrated a significant advantage (P < .001) to ImageNet models (9.4%, 4.0%, 4.8%, and 4.5% AUC improvements, respectively). For larger datasets—pneumonia (chest radiography), COVID-19 (CT), SARS-CoV-2 (CT), and intracranial hemorrhage (CT)—the RadImageNet models also illustrated improved AUC (P < .001) by 1.9%, 6.1%, 1.7%, and 0.9%, respectively. Additionally, lesion localizations of the RadImageNet models were improved by 64.6% and 16.4% on thyroid and breast US datasets, respectively.ConclusionRadImageNet pretrained models demonstrated better interpretability compared with ImageNet models, especially for smaller radiologic datasets.Keywords: CT, MR Imaging, US, Head/Neck, Thorax, Brain/Brain Stem, Evidence-based Medicine, Computer Applications–General (Informatics) Supplemental material is available for this article. Published under a CC BY 4.0 license.See also the commentary by Cadrin-Chênevert in this issue.

Random weighting method for M-test in linear model with dependent errors

We study the random weighting method for M-test in a linear model with dependent errors. The asymptotic distribution of this test statistic is derived under some conditions. This article shows that the random weighting method can be used for deciding the critical value for the M-test regardless of whether the null hypothesis is true or not. Finally, the performance of these results is evaluated by a simulation study.

Recursive integral equations for random weights averages: Exponential functions and Cauchy distribution

In this article, firstly, we prove that functions of the form ϕ(x)=ecxI(−∞,0)(x)+ebxI[0,+∞)(x), c,b constants, are the only solutions to the integral equation ϕ(x)=∫01ϕ(ux)ϕ((1−u)x)du. This indeed gives the result of Van Asshe (1987) who used the Schwartz distribution theory to prove that for i.i.d X and Y, UX+(1−U)Y=dX if and only if X has a Cauchy distribution. Secondly, by looking into certain recursive integral equations involving characteristic functions, we prove that if for an n≥2, the random weight mean U(1)X1+(U(2)−U(1))X2+⋯+(1−U(n−1))Xn has a Cauchy distribution, then X1 has a Cauchy distribution; random variables X1,…,Xn are i.i.d, the random weights are the cuts of (0,1) by a uniform sample. The multivariate analogue of this result is also provided.

Myers-Briggs Type Indicator Personality Model Classification in English Text using Convolutional Neural Network Method

Myers-Briggs Type Indicator (MBTI) is a personality model developed by Katharine Cooks Briggs and Isabel Briggs Myers in 1940. It displays a combination of preferences from four domains. Generally, test takers need to answer about 50 to 70 questions, and it is relatively expensive to know MBTI personality. The researcher developed a personality classification system using the Convolutional Neural Network (CNN) method and GloVe (Global Vectors for Word Representation) word embedding to solve this problem. The dataset used in this research consists of 8,675 data from the Kaggle site. The steps in this research are downloading the dataset from Kaggle, text preprocessing, GloVe weighting, classification using the CNN method, and evaluation using accuracy from the Confusion Matrix. Based on the tests carried out, using GloVe weighting can improve the model accuracy rather than random weighting. The best GloVe word dimensions depend on the metrics used to measure the model performance and the data of the classes contained in the dataset. From the CNN hyperparameter tuning test, the Adamax optimizer performs better and produces higher accuracy than the Adam optimizer. In addition, the CNN hyperparameter tuning increased model accuracy more significantly compared with the best GloVe word embedding dimensions.

Multiclass Classification of Systolic and Diastolic Peaks on Open Benchmark PPG Signals: Performance Evaluation

Photoplethysmogram (PPG) signals contain valuable health information that is in the relation between the volumetric variations of blood circulation and the cardiovascular and respiratory systems. This study introduces the performance evaluation on open clinical benchmark PPG signals with a multiclass neural network with random weights (NNRW) classification method for systolic peak and diastolic point detection. The best performance of the peak and point detection is crucial to be achieved at the early stage for extracting further valuable information in addition to future predictions of cardiovascular-related illness. Various open clinical datasets of PPG signals have been introduced, however, there is a lack of information on peak annotations. Due to the lack of peak annotation information, it is time-consuming to be prepared. One suitable clinical benchmark dataset with peak annotation information for peak detection has been previously evaluated, however, it cannot be generalized and rely upon only one dataset. Therefore, for generalization, there is a new open clinical benchmark dataset that is found in the year 2018 and our own collected data from normal participants is utilized in this study. The findings exhibit more convincing overall accuracy and Gmean of testing results with 94.86 and 94.74 percent, respectively. The findings of the comparison with previous work indicate that the proposed methodology to predict PPG-based multi-class systolic and diastolic points is more generalizable.

A Deep Modality-Specific Ensemble for Improving Pneumonia Detection in Chest X-rays.

Pneumonia is an acute respiratory infectious disease caused by bacteria, fungi, or viruses. Fluid-filled lungs due to the disease result in painful breathing difficulties and reduced oxygen intake. Effective diagnosis is critical for appropriate and timely treatment and improving survival. Chest X-rays (CXRs) are routinely used to screen for the infection. Computer-aided detection methods using conventional deep learning (DL) models for identifying pneumonia-consistent manifestations in CXRs have demonstrated superiority over traditional machine learning approaches. However, their performance is still inadequate to aid in clinical decision-making. This study improves upon the state of the art as follows. Specifically, we train a DL classifier on large collections of CXR images to develop a CXR modality-specific model. Next, we use this model as the classifier backbone in the RetinaNet object detection network. We also initialize this backbone using random weights and ImageNet-pretrained weights. Finally, we construct an ensemble of the best-performing models resulting in improved detection of pneumonia-consistent findings. Experimental results demonstrate that an ensemble of the top-3 performing RetinaNet models outperformed individual models in terms of the mean average precision (mAP) metric (0.3272, 95% CI: (0.3006,0.3538)) toward this task, which is markedly higher than the state of the art (mAP: 0.2547). This performance improvement is attributed to the key modifications in initializing the weights of classifier backbones and constructing model ensembles to reduce prediction variance compared to individual constituent models.

Deep stochastic configuration networks with different random sampling strategies

Stochastic configuration networks (SCNs) are a class of randomized learner models that ensure the universal approximation property, whereby random weights and biases are drawn from the uniform distribution and selected by a supervisory mechanism. This paper looks into the impact of the distribution of random weights on the performance of SCNs. In the light of a fundamental principle in machine learning, that is, a model with smaller parameters holds improved generalization, we recommend using symmetric zero-centered distributions in constructing SCNs to improve the generalization performance. Further, we introduce a scalar in the distributions to make the SCN model adaptively feasible to different datasets. Simulation results are reported for both regression and classification tasks over twenty-one benchmark datasets using SCN. Results are also presented on ten regression datasets using a deep implementation of SCN, known as deep stochastic configuration networks (DeepSCN).

Design of Machine Learning Algorithm for Tourism Demand Prediction.

Unused hotel rooms, unused event tickets, and unsold items are all examples of wasted expenses and earnings. Governments require accurate tourism demand forecasting in order to make informed decisions on topics such as infrastructure development and lodging site planning; therefore, accurate tourism demand forecasting becomes vital. Artificial intelligence (AI) models such as neural networks and security violation report (SVR) have been used effectively in tourist demand forecasting as a result of the fast advancement of AI. This paper constructs a tourism demand forecasting model based on machine learning on the basis of the existing forecasting model research. The completed work is as follows: (1) It introduces a large number of domestic and foreign literatures on tourism volume forecasting and proposes the research content of this paper. (2) It is proposed to stack the long short-term memory- (LSTM-) based autoencoders deeply, by adopting a hierarchical greedy pretraining method to replace the random weight initialization method used in the deep network and combining this pretraining stage and fine-tuning network together to form the SAE-LSTM prediction model for improving the performance of deep learning models. (3) This paper uses the monthly search engine strength data of city A's monthly tourist volume and its related influencing factors as the data set; processes the data set to make the model adapt to the data input; uses mean absolute error (MAE), root mean square error (RMSE), MAPE, and other model evaluation indicators; and uses LSTM and the constructed SAE-LSTM model to conduct comparative experiments to predict the number of tourist arrivals in four years. The prediction results of the models proposed in this paper are better than those of the LSTM model. According to the experimental results, the superiority of the proposed LSTM-based unsupervised pretraining method is demonstrated.

Intelligent quantification of natural gas pipeline defects using improved sparrow search algorithm and deep extreme learning machine

Accurate damage degree prediction is important to ensure the safety, stability, and economic operation of the natural gas pipeline in service, especially those in long-term service. In order to maintain the efficiency and safety of natural gas pipeline transportation, the intelligent diagnosis and evaluation technology of pipeline is very important. In this paper, The opposition-based learning strategy and adaptive T-distribution mutation operator are introduced to optimize the sparrow search algorithm (SSA),improve the search ability,convergence speed and accuracy of SSA algorithm. 12 classical benchmark functions are used to evaluate the performance of the improved algorithm. Experimental results demonstrate the feasibility and validity of ISSA compared with the original Sparrow search algorithm. Based on its excellent performance, ISSA is used to optimize the input layer weight and bias parameters of deep extreme learning machine (DELM). Therefore, a comprehensive ISSA-DELM network model is constructed for intelligent quantitative evaluation of natural gas pipeline defects. The results show that the model can effectively overcome the problem that the DELM effect is affected by the random input weight and random bias of each ELM-AE, and improve the quantitative prediction performance. This will facilitate the assessment of the integrity and safety status of natural gas pipelines.

Image Classification and Recognition Based on Deep Learning and Random Forest Algorithm

Convolutional neural network (CNN) is an important way to solve the problems of image classification and recognition. It can realize effective feature representation and make continuous breakthroughs in the field of image recognition, but it needs a lot of time in the training process. At the same time, random forest (RF) has the advantages of fast training speed and high classification accuracy. Aiming at the problem of image classification and recognition, this paper proposes a hybrid model based on CNN, which inputs the features extracted by CNN into RF for classification. Since the random weight network can also obtain valid results, the gradient algorithm is not used to adjust the network parameters to avoid consuming a lot of time. Finally, experiments are conducted on MNIST dataset and rotated MNIST dataset, and the results show that the classification accuracy of the hybrid model has improved more than RF, and also, the generalization ability has been improved.

Eigenvalue distribution of large random matrices arising in deep neural networks: Orthogonal case

This paper deals with the distribution of singular values of the input–output Jacobian of deep untrained neural networks in the limit of their infinite width. The Jacobian is the product of random matrices where the independent weight matrices alternate with diagonal matrices whose entries depend on the corresponding column of the nearest neighbor weight matrix. The problem has been considered in the several recent studies of the field for the Gaussian weights and biases and also for the weights that are Haar distributed orthogonal matrices and Gaussian biases. Based on a free probability argument, it was claimed in those papers that, in the limit of infinite width (matrix size), the singular value distribution of the Jacobian coincides with that of the analog of the Jacobian with special random but weight independent diagonal matrices, the case well known in random matrix theory. In this paper, we justify the claim for random Haar distributed weight matrices and Gaussian biases. This, in particular, justifies the validity of the mean field approximation in the infinite width limit for the deep untrained neural networks and extends the macroscopic universality of random matrix theory to this new class of random matrices.

Discriminative and regularized echo state network for time series classification

An echo State Network (ESN) is a special structure of a recurrent neural network (RNN) in which the recurrent neurons are randomly connected. ESN models which have achieved a high accuracy on time series prediction tasks can be used as time series prediction models in many domains. Nevertheless, in most ESN models, the input weights are randomly generated and the output weights calculated by the least square method are susceptible to outliers, which cannot guarantee that the ESN models will always be optimal for a given task. In this paper, a novel discriminative and regularized ESN (DR-ESN) combines discriminative feature aggregation (DFA) and outlier-robust weights (ORW) algorithms are proposed for time series classification. DFA is firstly proposed to replace the random input weights of ESN with the constrained weights generated from sample information. In DFA, weight vectors are selected from the vector space spanned by initial input sequence vectors, then the new generated input weights can adequately represent the data features. Secondly, ORW is employed to enhance the robustness of output weights by constraining the weights assigned to samples with large training errors. The weights evaluation and experiments on a massive set of the synthetic time series data, real-world bearing fault data and UCR benchmarks indicate that the proposed DR-ESN can not only considerably improve the original ESN classifier but also effectively suppress the effect of outliers on classification performance.

An efficient robust optimized functional link broad learning system for solar irradiance prediction

A new machine learning approach for forecasting short-termsolar irradiance is presented in this paper considering different weather conditions and time horizon in a microgrid environment. Using historical solar irradiance, a short-term forecasting model known as Robust optimized functional link broad learning system (FLBLS) is developed. Here the original input comprising delayed historical solar irradiance samples is initially expanded by exponential trigonometric functions and the expanded features are then used to create a large number of feature nodes using random weights and biases. Further the newly created features are fed as inputs to the enhancement nodes forming a broad learning system (BLS). This newly formed flat FLBLS facilitates incremental learning in fast remodelling of the network if needed for efficient performance without a retraining process unlike deep neural networks using several hidden layers and filters, etc. The proposed approach is designed on the basis of the standard random vector functional link network (RVFLN), however unlike the traditional RVFLN that applies inputs directly and set up the enhancement nodes, in our proposed approach, initially the inputs are expanded and map the expanded inputs to generate a set of mapped features. It reduces the lengthy process unlike deep neural networks that involves the back propagation algorithm for error minimization. The proposed FLBLS approach simplifies the learning process with the proper choice of feature nodes as it can be implemented to a flat network or a simpler network such as ELM that requires only the simple calculation of the output weight and obtain the final output where retraining is not required for adding extra feature nodes or enhancement nodes like deep neural architectures. Also, the minimum variance strategy with the training samples is useful in rejecting the effects of noise and outliers in the system and make the network robust which gets reflected in computing output weights using generalized least square. The minimum variance strategy further enhances the performance of the proposed system. To improve the prediction accuracy, the FLBLS parameters are optimized using an efficient metaheuristic chaotic firefly-sine cosine levy flight algorithm. Also, to reject the effects of noise and outliers and make the network robust a minimum variance strategy is used with the training samples that gets reflected in computing output weights using generalised least squares. The suggested method is experimented rigorously and analysed with different time frames designed for 30 min and 60 min ahead solar irradiance prediction. The obtained results from experiments confirm that the proposed FLBLS approach outperforms other randomized neural network forecasting methods as well many state-of-the-art methods using solar irradiance data of Hyderabad, India collected from National Renewable Energy Laboratory (NREL). This approach exhibits with the lowest errors in different case studies among all the compared methods. This FLBLS designed approach is an efficient approach for the purpose of solar irradiance prediction with highly nonlinear data.

Capacity configuration of distributed photovoltaic and battery system for office buildings considering uncertainties

In practice, the actual operation conditions are generally different from those assumed in the design stage, which causes uncertainties of actual building load and photovoltaic (PV) power generation. This study conducted an optimization design for PV-battery (PVB) system with consideration of these uncertainties. Three groups of factors with uncertainty, relating to outdoor conditions, building construction and indoor conditions, were discussed. An uncertainty-based multi-objective optimization design method was proposed for the PVB system based on the random simulations and weight sum method. Compared with conventional deterministic optimization method, the uncertainty-based optimization method is more likely to achieve the optimal configuration of PV and batteries in actual conditions. When the weight factors are the same, the impacts of self-consumption rate (SCR) on the PVB system configuration are larger than that of the self-sufficiency rate (SSR). The performance of the PVB system is much more sensitive to PV capacity than to battery capacity. In practice, on the premise of achieving the required SCR, the PV capacity should be maximized while the battery capacity be minimized. Representative values of technical indicators (SCR and SSR), of which a relative deviation of 5% covers more than 84% of all the results, were determined. Based on the representative values, the configuration indices of PV and battery capacities for office buildings in the hot-summer and cold-winter, and the IV solar resource zone of China, were proposed, and summarized in the form of table. The proposed configuration indices can be used directly as guide for PVB system design.

Instantaneous Brain Strain Estimation for Automotive Head Impacts via Deep Learning.

Efficient brain strain estimation is critical for routine application of a head injury model. Lately, a convolutional neural network (CNN) has been successfully developed to estimate spatially detailed brain strains instantly and accurately in contact sports. Here, we extend its application to automotive head impacts, where impact profiles are typically more complex with longer durations. Head impact kinematics (N=458) from two public databases were used to generate augmented impacts (N=2694). They were simulated using the anisotropic Worcester Head Injury Model (WHIM) V1.0, which provided baseline elementwise peak maximum principal strain (MPS). For each augmented impact, rotational velocity (vrot) and the corresponding rotational acceleration (arot) profiles were concatenated as static images to serve as CNN input. Three training strategies were evaluated: 1) "baseline", using random initial weights; 2) "transfer learning", using weight transfer from a previous CNN model trained on head impacts drawn from contact sports; and 3) "combined training", combining previous training data from contact sports (N=5661) for training. The combined training achieved the best performances. For peak MPS, the CNN achieved a coefficient of determination (R2) of 0.932 and root mean squared error (RMSE) of 0.031 for the real-world testing dataset. It also achieved a success rate of 60.5% and 94.8% for elementwise MPS, where the linear regression slope, k, and correlation coefficient, r, between estimated and simulated MPS did not deviate from 1.0 (when identical) by more than 0.1 and 0.2, respectively. Cumulative strain damage measure (CSDM) from the CNN estimation was also highly accurate compared to those from direct simulation across a range of thresholds (R2 of 0.899-0.943 with RMSE of 0.054-0.069). Finally, the CNN achieved an average k and r of 0.98±0.12 and 0.90±0.07, respectively, for six reconstructed car crash impacts drawn from two other sources independent of the training dataset. Importantly, the CNN is able to efficiently estimate elementwise MPS with sufficient accuracy while conventional kinematic injury metrics cannot. Therefore, the CNN has the potential to supersede current kinematic injury metrics that can only approximate a global peak MPS or CSDM. The CNN technique developed here may offer enhanced utility in the design and development of head protective countermeasures, including in the automotive industry. This is the first study aimed at instantly estimating spatially detailed brain strains for automotive head impacts, which employs >8.8 thousand impact simulations generated from ~1.5 years of nonstop computations on a high-performance computing platform.

Older adult driving performance assessed under simulated and on-road conditions

Simulated driving offers a convenient test of driving ability for older drivers, although the viability of using simulated driving with this population is mixed. The relative weighting of the relevant perceptual, cognitive, and physical factors may vary between simulated and on-road driving. The current study was designed to assess this possibility. We conducted simulated and on-road driving tests of 61 older adults aged 66–92 years. To ensure that the driving performance was measured similarly between the two driving modalities, we employed the Record of Driving Errors (RODE) driving assessment system during both driving tests. Correlation and random weights analysis (RWA) results indicated only modest evidence of correspondence between the simulated and on-road driving performances. The primary factors operative in both simulated and on-road driving was Useful Field of View and a measure of basic cognition. Unique factors for simulated driving included a measure of physical mobility (Time-Up-and-Go) and spatial reasoning (Line), and for on-road driving included chronological age and sensorimotor processing (Trail-Making Task A). Chronological age was correlated primarily the on-road rather than simulated test, was greatly reduced with the inclusion of additional explanatory factors, and likely reflects driving efficiency rather than driving safety. We conclude that simulated driving in healthy older drivers can be beneficial for research purposes to assess cognitive and perceptual factors that underly driving effectiveness, although it cannot serve as a clear proxy for on-road driving.

Stochastic Graphon Games: II. The Linear-Quadratic Case

In this paper, we analyze linear-quadratic stochastic differential games with a continuum of players interacting through graphon aggregates, each state being subject to idiosyncratic Brownian shocks. The major technical issue is the joint measurability of the player state trajectories with respect to samples and player labels, which is required to compute for example costs involving the graphon aggregate. To resolve this issue we set the game in a Fubini extension of a product probability space. We provide conditions under which the graphon aggregates are deterministic and the linear state equation is uniquely solvable for all players in the continuum. The Pontryagin maximum principle yields equilibrium conditions for the graphon game in the form of a forward-backward stochastic differential equation, for which we establish existence and uniqueness. We then study how graphon games approximate games with finitely many players over graphs with random weights. We illustrate some of the results with a numerical example.

Spectroscopic profiling-based geographic herb identification by neural network with random weights

Daodi medicinal material plays an important role in traditional Chinese medicine (TCM). This study researches and validates the NNRW (neural network with random weights) model on spectroscopic profiling data for geographical origin identification. NNRW is a special neural network model that does not require an iterative training process. It has been proved effective in various resource-limited data-driven applications. However, whether NNRW works for spectroscopic profiling data remains to be explored. In this study, the Raman and UV (ultraviolet) profiling data of 160 radix astragali samples from four geographic regions are trained and evaluated by four classification models, i.e., NNRW, MLP (multi-layer perceptron), SVM (support vector machine), and DTC (decision tree classifier). Their validation accuracies are 96.3%, 98.0%, 98.4%, and 92.8% respectively. The training/fitting times are 0.372 ms (milli-seconds), 57.9 ms, 2.033 ms, and 3.351 ms, respectively. This study shows that NNRW has a significant training time cut while keeping a high prediction accuracy, and it is a promising solution to resource-limited edge computing applications.

RESERVOIR COMPUTING WITH TRUNCATED NORMAL DISTRIBUTION FOR SPEECH EMOTION RECOGNITION

Speech is an effective, quick, and important way for communicating and exchanging complex information between humans. Emotions have always been a part of normal human conversation which makes the speech more attractive. Because of this major role of both speech and emotion, many researchers are inspired by studying Speech Emotion Recognition (SER) which still has plenty of challenges. In this study, we proposed a novel reservoir computing approach with the initialization of random connection weights for the input weight by the truncated normal distribution. Furthermore, Population-Based Training (PBT) is adopted to optimize the hyperparameters of the whole Echo State Network (ESN) model which have a significant impact on the model performance. The proposed model has adopted bidirectional reservoir input to increase the memorization capability, and Sparse Random Projection (SRP) was applied for dimensional reduction as a simple, unsupervised, and low complexity approach. The speaker-independent strategy was employed on EMODB and SAVEE datasets as an acted speech emotion dataset and Aibo as a non-acted dataset. The model achieved 84.8%, 65.95%, and 45.99% unweighted average recalls on the EMODB, SAVEE, and Aibo datasets respectively. The results show that the proposed model outperforms the recent state-of-the-art studies with a cheaper computational cost.

Performance Evaluation of Chi-Square and Normal Distributions of Mesh Clients for WMNs Considering Five Router Replacement Methods

In our previous work, we implemented a simulation system to solve the node placement problem in WMNs considering Particle Swarm Optimization (PSO) and Distributed Genetic Algorithm (DGA), called WMN-PSODGA. In this paper, we compare Chi-square and Normal distributions of mesh clients for different router replacement methods. The router replacement methods considered are Constriction Method (CM), Random Inertia Weight Method (RIWM), Linearly Decreasing Inertia Weight Method (LDIWM), Linearly Decreasing Vmax Method (LDVM) and Rational Decrement of Vmax Method (RDVM). The simulation results show that for both distributions, the mesh routers cover all mesh clients for all router replacement methods. In terms of load balancing, Normal distribution shows better results than Chi-square. The best router replacement method for this distribution is LDIWM. Thus, the best scenario is the Normal distribution of mesh clients with LDIWM as a router replacement method.