Random Weights Research Articles

Genetic Algorithm-Based Ensemble Hybrid Sparse ELM for Grasp Stability Recognition With Multimodal Tactile Signals

Grasp stability recognition based on tactile perception has attracted increasing attention in the robotics community. In this article, we extract tactile features from multimodal tactile signals and propose a novel ensemble approach named genetic algorithm-based ensemble hybrid sparse extreme learning machine (GA-EHSELM) for the grasp stability recognition task. In contrast to traditional ensemble extreme learning machines (ELMs), the proposed approach takes the sparsity of the random weights assigned to the connection between the input layer and the hidden layer for each base learner into consideration, in order to avoid the problem of overfitting. In addition, The diversity of base learners is increased by constructing two types of sparse ELM (SELM). The random weights of the base SELMs are sampled from two different distributions with a certain probability. Furthermore, we utilize the genetic algorithm (GA) for the optimization of tactile features and the base sparse ELMs. To be specific, GA is employed for feature selection as well as the optimization of the sparsity of the random weights associated with different types of base learners and the number of each type of base learner. The effectiveness of the proposed approach has been demonstrated on a public tactile grasp stability dataset.

Weighted bootstrap for two-sample [formula omitted]-statistics

In this paper, we introduce weighted bootstrap algorithms for both non-degenerate and degenerate two-sample U-statistics with arbitrary degrees. For the non-degenerate case, weighted bootstrap with dependent weights is introduced as a generalization of Efron’s conventional bootstrap. In addition, two weighted bootstrap procedures with independent productive weights and independent additive weights are proposed under non-degeneracy. More importantly, we extend the weighted bootstrap method to two-sample U-statistics under the degeneracy of order 2 with a novel construction of random weights. Theoretical supports of the proposed weighted bootstrap procedures under non-degeneracy and degeneracy of order 2 are established. Numerical studies illustrate that the proposed approaches are feasible and effective for statistical inferences.

Weighted error-output recurrent echo kernel state network for multi-step water level prediction

With development of information techniques in navigation and shipping, machine learning algorithms are applied in enhancing navigation safety. One of critical areas, which attracts lots of attention from scientists and researchers, is water level prediction. Although randomization-based algorithms obtain good performance in water level prediction, these algorithms still have their own limitations. For example, unstable prediction problem caused by random weights selection, and the error accumulation problem caused by the conventional recurrent algorithm. In this study, we combine three proposed approach with the conventional echo state network. Firstly, the Gaussian kernel method transforms the input features into a high-dimensional features, which in some extent improves the forecasting accuracy. Secondly, kernel reservoir states are proposed. It not only abandons the random selected weights, but it also makes hidden neurons to connect closely. Lastly, a novel weighted error-output recurrent multi-step algorithm is proposed. It uses previous forecasting errors to update the current output weights in order to overcome the error accumulation problem. Based on above three approaches, a multi-step water level prediction model is proposed called Weighted Error-output Recurrent Echo Kernel State Network (WER-EKSN). The experimental results and statistical analysis represent that our proposed model has better forecasting performance than other compared models. It not only has the superior ability in water level prediction, but it also provides the evidences for the management of transportation in water-land, such as flood protection, and management of ship route.

Using VGG Models with Intermediate Layer Feature Maps for Static Hand Gesture Recognition

A hand gesture recognition system provides a robust and innovative solution to nonverbal communication through human–computer interaction. Deep learning models have excellent potential for usage in recognition applications. To overcome related issues, most previous studies have proposed new model architectures or have fine-tuned pre-trained models. Furthermore, these studies relied on one standard dataset for both training and testing. Thus, the accuracy of these studies is reasonable. Unlike these works, the current study investigates two deep learning models with intermediate layers to recognize static hand gesture images. Both models were tested on different datasets, adjusted to suit the dataset, and then trained under different methods. First, the models were initialized with random weights and trained from scratch. Afterward, the pre-trained models were examined as feature extractors. Finally, the pre-trained models were fine-tuned with intermediate layers. Fine-tuning was conducted on three levels: the fifth, fourth, and third blocks, respectively. The models were evaluated through recognition experiments using hand gesture images in the Arabic sign language acquired under different conditions. This study also provides a new hand gesture image dataset used in these experiments, plus two other datasets. The experimental results indicated that the proposed models can be used with intermediate layers to recognize hand gesture images. Furthermore, the analysis of the results showed that fine-tuning the fifth and fourth blocks of these two models achieved the best accuracy results. In particular, the testing accuracies on the three datasets were 96.51%, 72.65%, and 55.62% when fine-tuning the fourth block and 96.50%, 67.03%, and 61.09% when fine-tuning the fifth block for the first model. The testing accuracy for the second model showed approximately similar results.

Dynamic Parameter Identification of Collaborative Robot Based on WLS-RWPSO Algorithm

Parameter identification of the dynamic model of collaborative robots is the basis of the development of collaborative robot motion state control, path tracking, state monitoring, fault diagnosis, and fault tolerance systems, and is one of the core contents of collaborative robot research. Aiming at the identification of dynamic parameters of the collaborative robot, this paper proposes an identification algorithm based on weighted least squares and random weighted particle swarm optimization (WLS-RWPSO). Firstly, the dynamics mathematical model of the robot is established using the Lagrangian method, the dynamic parameters of the robot to be identified are determined, and the linear form of the dynamics model of the robot is derived taking into account the joint friction characteristics. Secondly, the weighted least squares method is used to obtain the initial solution of the parameters to be identified. Based on the traditional particle swarm optimization algorithm, a random weight particle swarm optimization algorithm is proposed for the local optimal problem to identify the dynamic parameters of the robot. Thirdly, the fifth-order Fourier series is designed as the excitation trajectory, and the original data collected by the sensor are denoised and smoothed by the Kalman filter algorithm. Finally, the experimental verification on a six-degree-of-freedom collaborative robot proves that the predicted torque obtained by the identification algorithm in this paper has a high degree of matching with the measured torque, and the established model can reflect the dynamic characteristics of the robot, effectively improving the identification accuracy.

Broad learning solution for rapid diagnosis of COVID-19

COVID-19 has put all of humanity in a health dilemma as it spreads rapidly. For many infectious diseases, the delay of detection results leads to the spread of infection and an increase in healthcare costs. COVID-19 diagnostic methods rely on a large number of redundant labeled data and time-consuming data training processes to obtain satisfactory results. However, as a new epidemic, obtaining large clinical datasets is still challenging, which will inhibit the training of deep models. And a model that can really rapidly diagnose COVID-19 at all stages of the model has still not been proposed. To address these limitations, we combine feature attention and broad learning to propose a diagnostic system (FA-BLS) for COVID-19 pulmonary infection, which introduces a broad learning structure to address the slow diagnosis speed of existing deep learning methods. In our network, transfer learning is performed with ResNet50 convolutional modules with fixed weights to extract image features, and the attention mechanism is used to enhance feature representation. After that, feature nodes and enhancement nodes are generated by broad learning with random weights to adaptly select features for diagnosis. Finally, three publicly accessible datasets were used to evaluate our optimization model. It was determined that the FA-BLS model had a 26–130 times faster training speed than deep learning with a similar level of accuracy, which can achieve a fast and accurate diagnosis, achieve effective isolation from COVID-19 and the proposed method also opens up a new method for other types of chest CT image recognition problems.

Structure and performance of fully connected neural networks: Emerging complex network properties

Understanding the behavior of Artificial Neural Networks is one of the main topics in the field recently, as black-box approaches have become usual since the widespread of deep learning. Such high-dimensional models may manifest instabilities and weird properties that resemble complex systems. Therefore, we propose Complex Network (CN) techniques to analyze the structure and performance of fully connected neural networks. For that, we build a dataset with 4 thousand models (varying the initialization seed) and their respective CN properties. This is the first work to explore the CN properties of an ample number of fully connected networks accounting for the variance caused by random weight initialization. The networks are trained in a supervised classification setup considering four vision benchmarks and then approached as a weighted and undirected graph of neurons and synapses (learned weights). Results show that neuronal centrality is highly correlated to network classification performance. We also propose the concept of Bag-Of-Neurons (BoN), a CN-based approach for finding topological signatures linking similar neurons. Results suggest that six neuronal types emerge in such networks, independently of the target domain, and are distributed differently according to classification accuracy. We also tackle specific CN properties related to performance, such as higher subgraph centrality on lower-performing models. Our findings suggest that CN properties play a critical role in the performance of fully connected neural networks, with topological patterns emerging independently on a wide range of models.

Intermediate Learning-Based Attention Regulated Densenet for Diagnosis of Covid-19

Objectives: The AI-based Computer-Aided-Diagnosis of Chest X-Rays related to COVID-19 is very essential. Here we present an Attention regulated Pre-trained DenseNet-121with intermediate transfer learning as a Chest X-Ray image classifier to classify images according to three labels: COVID-19, pneumonia, and normal. Methods: We are proposing a new Attention regulated ImageNet pre-trained DenseNet-121 neural network architecture, which is retrained on NIH ChestX-ray14 data as an intermediate database before the actual COVID- 19 database. We also fine-tuned the last layer of this neural network with a suggested novelty called the output neuron-keeping technique. Before feeding the Covid-19 data we removed all other neurons corresponding to Chest X-ray14 pathology classes except the “No finding” and “Pneumonia” classes. A new third neuron with random weights and bias is created in the final layer to detect Covid-19 pathology. A DenseNet-121 is supported by a GRAD-CAM-based attention mechanism in achieving detection accuracy and localization of pathologies. The used Covid-19 dataset is a combination of 370 Pneumonia, 1255 Normal, and 439 COVID-19 Chest X-Ray images. We randomly took 50 pictures from each class for testing purposes, the remaining images are augmented more to improve DenseNet performance on a small COVID- 19 dataset. Findings: Our state-of-art model achieved 98.6% test accuracy since it misclassified one out of 50 Covid-19 images, and one out of 50 Pneumonia images, but all 50 normal Chest X-Ray images are classified with 100% accuracy. We compared our model with the other three state-of-theart models, particularly under three-class classification problems (Pneumonia,Covid-19, and Normal). The experimental result shows that the proposed model outperforms the existing three models by obtaining an accuracy of 98.6%. The GRAD-CAM generated heatmaps improved the interpretation of the infections. Novelty: The intermediate transfer learning, neuron keeping, and attention-regulated DenseNet are the highlight of the proposed work in achieving detection accuracy. The GRAD-CAM generated heatmaps indicate future research on COVID-19 diagnosis to analyze the severity of the infection. We believe this model may bridge the medical community and Artificial Intelligence gap. Keywords: Chest XRay14; Covid19 diagnosis; DenseNet121; GradCam; Neuron keeping

MaxEnt brings comparable results when the input data are being completed; Model parameterization of four species distribution models.

Species distribution models (SDMs) are practical tools to assess the habitat suitability of species with numerous applications in environmental management and conservation planning. The manipulation of the input data to deal with their spatial bias is one of the advantageous methods to enhance the performance of SDMs. However, the development of a model parameterization approach covering different SDMs to achieve well-performing models has rarely been implemented. We integrated input data manipulation and model tuning for four commonly-used SDMs: generalized linear model (GLM), gradient boosted model (GBM), random forest (RF), and maximum entropy (MaxEnt), and compared their predictive performance to model geographically imbalanced-biased data of a rare species complex of mountain vipers. Models were tuned up based on a range of model-specific parameters considering two background selection methods: random and background weighting schemes. The performance of the fine-tuned models was assessed based on recently identified localities of the species. The results indicated that although the fine-tuned version of all models shows great performance in predicting training data (AUC > 0.9 and TSS > 0.5), they produce different results in classifying out-of-bag data. The GBM and RF with higher sensitivity of training data showed more different performances. The GLM, despite having high predictive performance for test data, showed lower specificity. It was only the MaxEnt model that showed high predictive performance and comparable results for identifying test data in both random and background weighting procedures. Our results highlight that while GBM and RF are prone to overfitting training data and GLM over-predict nonsampled areas MaxEnt is capable of producing results that are both predictable (extrapolative) and complex (interpolative). We discuss the assumptions of each model and conclude that MaxEnt could be considered as a practical method to cope with imbalanced-biased data in species distribution modeling approaches.

Echo state graph neural networks with analogue random resistive memory arrays

Recent years have witnessed a surge of interest in learning representations of graph-structured data, with applications from social networks to drug discovery. However, graph neural networks, the machine learning models for handling graph-structured data, face significant challenges when running on conventional digital hardware, including the slowdown of Moore’s law due to transistor scaling limits and the von Neumann bottleneck incurred by physically separated memory and processing units, as well as a high training cost. Here we present a hardware–software co-design to address these challenges, by designing an echo state graph neural network based on random resistive memory arrays, which are built from low-cost, nanoscale and stackable resistors for efficient in-memory computing. This approach leverages the intrinsic stochasticity of dielectric breakdown in resistive switching to implement random projections in hardware for an echo state network that effectively minimizes the training complexity thanks to its fixed and random weights. The system demonstrates state-of-the-art performance on both graph classification using the MUTAG and COLLAB datasets and node classification using the CORA dataset, achieving 2.16×, 35.42× and 40.37× improvements in energy efficiency for a projected random resistive memory-based hybrid analogue–digital system over a state-of-the-art graphics processing unit and 99.35%, 99.99% and 91.40% reductions of backward pass complexity compared with conventional graph learning. The results point to a promising direction for next-generation artificial intelligence systems for graph learning.

Feature-Selection-Based Attentional-Deconvolution Detector for German Traffic Sign Detection Benchmark

In this study, we propose a novel traffic sign detection algorithm based on the deeplearning approach. The proposed algorithm, which we termed the feature-selection-based attentionaldeconvolution detector (FSADD), is used along with the “you look only once” (YOLO) v5 structure for feature selection. When applying feature selection inside a detection algorithm, the network divides the extracted feature maps after the convolution layer into similar and non similar feature maps. Generally, the feature maps obtained after the convolution layers are the outputs of filters with random weights. Owing to the randomness of the filter, the network obtains various kinds of feature maps with unnecessary components, which degrades the detection performance. However, grouping feature maps with high similarities can increase the relativeness of each feature map, thereby improving the network detection of specific targets from images. Furthermore, the proposed FSADD model has modified sizes of the receptive fields for improved traffic sign detection performance. Many of the available general detection algorithms are unsuitable for the German traffic sign detection benchmark (GTSDB) because of the small sizes of these signs in the images. Experimental comparisons were performed with respect to the GTSDB to show that the proposed FSADD is comparable to the state-of-the-art while detecting 29 kinds of traffic signs with 73.9% accuracy of classification performances.

Stratified incomplete local simplex tests for curvature of nonparametric multiple regression

Principled nonparametric tests for regression curvature in $\mathbb{R}^{d}$ are often statistically and computationally challenging. This paper introduces the stratified incomplete local simplex (SILS) tests for joint concavity of nonparametric multiple regression. The SILS tests with suitable bootstrap calibration are shown to achieve simultaneous guarantees on dimension-free computational complexity, polynomial decay of the uniform error-in-size, and power consistency for general (global and local) alternatives. To establish these results, a general theory for incomplete $U$-processes with stratified random sparse weights is developed. Novel technical ingredients include maximal inequalities for the supremum of multiple incomplete $U$-processes.

Infinite-color randomly reinforced urns with dominant colors

We define and prove limit results for a class of dominant P\'olya sequences, which are randomly reinforced urn processes with color-specific random weights and unbounded number of possible colors. Under fairly mild assumptions on the expected reinforcement, we show that the predictive and the empirical distributions converge almost surely (a.s.) in total variation to the same random probability measure $\tilde{P}$; moreover, $\tilde{P}(\mathcal{D})=1$ a.s., where $\mathcal{D}$ denotes the set of dominant colors for which the expected reinforcement is maximum. In the general case, the predictive probabilities and the empirical frequencies of any $\delta$-neighborhood of $\mathcal{D}$ converge a.s. to one. That is, although non-dominant colors continue to be regularly observed, their distance to $\mathcal{D}$ converges in probability to zero. We refine the above results with rates of convergence. We further hint potential applications of dominant P\'olya sequences in randomized clinical trials and species sampling, and use our central limit results for Bayesian inference.

Fine asymptotics for the maximum degree in weighted recursive trees with bounded random weights

A weighted recursive tree is an evolving tree in which vertices are assigned random vertex-weights and new vertices connect to a predecessor with a probability proportional to its weight. Here, we study the maximum degree and near-maximum degrees in weighted recursive trees when the vertex-weights are almost surely bounded and their distribution function satisfies a mild regularity condition near zero. We are able to specify higher-order corrections to the first order growth of the maximum degree established in prior work. The accuracy of the results depends on the behaviour of the weight distribution near the largest possible value and in certain cases we manage to find the corrections up to random order. Additionally, we describe the tail distribution of the maximum degree, the distribution of the number of vertices attaining the maximum degree, and establish asymptotic normality of the number of vertices with near-maximum degree. Our analysis extends the results proved for random recursive trees (where the weights are constant) to the case of random weights. The main technical result shows that the degrees of several uniformly chosen vertices are asymptotically independent with explicit error corrections.

Less Is More: Adaptive Trainable Gradient Dropout for Deep Neural Networks.

The undeniable computational power of artificial neural networks has granted the scientific community the ability to exploit the available data in ways previously inconceivable. However, deep neural networks require an overwhelming quantity of data in order to interpret the underlying connections between them, and therefore, be able to complete the specific task that they have been assigned to. Feeding a deep neural network with vast amounts of data usually ensures efficiency, but may, however, harm the network's ability to generalize. To tackle this, numerous regularization techniques have been proposed, with dropout being one of the most dominant. This paper proposes a selective gradient dropout method, which, instead of relying on dropping random weights, learns to freeze the training process of specific connections, thereby increasing the overall network's sparsity in an adaptive manner, by driving it to utilize more salient weights. The experimental results show that the produced sparse network outperforms the baseline on numerous image classification datasets, and additionally, the yielded results occurred after significantly less training epochs.

AI Enabled Ensemble Deep Learning Method for Automated Sensing and Quantification of DNA Damage in Comet Assay

Comet assay is a widely used technique to assess and quantify DNA damage in individual cells. Recently, researchers have applied various deep learning techniques to automate the analysis of comet assay. Image analysis using deep learning allows combining multiple parameters of images and performing computation at a pixel level to provide quantifiable information about the comets. The current deep learning analysis algorithms use a single neural network as a standard method, which relies on many comet images and prone to high variance in predictions. Here, we propose a new ensemble model consisting of a collection of deep learning networks with different configurations and different initial random weights trained on the same dataset to calculate one weighted prediction for DNA damage quantification. To develop this model, we curated a trainable comet assay image dataset consisting of1309 images with 9204 extracted features of cell head and tail length, area, etc With the proposed method we could achieve significantly higher accuracy (R2 = 89.3%, compared to 74% with the standard single neural network as reported in data published by M. D. Zeiler and R Fergus (European conference on computer vision, pp. 818–833 2014). Furthermore, deep regression with the proposed architecture produced much more reliable and accurate results than conventional method.

Random-Enabled Hidden Moving Target Defense against False Data Injection Alert Attackers

Hidden moving target defense (HMTD) is a proactive defense strategy that is kept hidden from attackers by changing the reactance of transmission lines to thwart false data injection (FDI) attacks. However, alert attackers with strong capabilities pose additional risks to the HMTD and thus, it is much-needed to evaluate the hiddenness of the HMTD. This paper first summarizes two existing alert attacker models, i.e., bad-data-detection-based alert attackers and data-driven alert attackers. Furthermore, this paper proposes a novel model-based alert attacker model that uses the MTD operation models to estimate the dispatched line reactance. The proposed attacker model can use the estimated line reactance to construct stealthy FDI attacks against HMTD methods that lack randomness. We propose a novel random-enabled HMTD (RHMTD) operation method, which utilizes random weights to introduce randomness and uses the derived hiddenness operation conditions as constraints. RHMTD is theoretically proven to be kept hidden from three alert attacker models. In addition, we analyze the detection effectiveness of the RHMTD against three alert attacker models. Simulation results on the IEEE 14-bus systems show that traditional HMTD methods fail to detect attacks by the model-based alert attacker, and RHMTD is kept hidden from three alert attackers and is effective in detecting attacks by three alert attackers.

ANN Prediction Model of Concrete Fatigue Life Based on GRW-DBA Data Augmentation

In order to improve the prediction accuracy of the machine learning model for concrete fatigue life using small datasets, a group calculation and random weight dynamic time warping barycentric averaging (GRW-DBA) data augmentation method is proposed. First, 27 sets of real experimental data were augmented by 10 times, 20 times, 50 times, 100 times, 200 times, 500 times, and 1000 times, respectively, using the GRW-DBA method, and the optimal factor was determined by comparing the model’s training time and prediction accuracy under different augmentation multiples. Then, a concrete fatigue life prediction model was established based on artificial neural network (ANN), and the hyperparameters of the model were determined through experiments. Finally, comparisons were made with data augmentation methods such as generative adversarial network (GAN) and regression prediction models such as support vector machine (SVM), and the generalization of the method was verified using another fatigue life dataset collected on the Internet. The result shows that the GRW-DBA algorithm can significantly improve the prediction accuracy of the ANN model when using small datasets (the R2 index increased by 20.1% compared with the blank control, reaching 98.6%), and this accuracy improvement is also verified in different data distributions. Finally, a graphical user interface is created based on the developed model to facilitate application in engineering.

Lithium-Ion Battery Life Prediction Method under Thermal Gradient Conditions

Thermal gradient is inevitable in a lithium-ion battery pack because of uneven heat generation and dissipation, which will affect battery aging. In this paper, an experimental platform for a battery cycle aging test is built that can simulate practical thermal gradient conditions. Experimental results indicate a high nonlinear degree of battery degradation. Considering the nonlinearity of Li-ion batteries aging, the extreme learning machine (ELM), which has good learning and fitting ability for highly nonlinear, highly nonstationary, and time-varying data, is adopted for prediction. A battery life prediction model based on the sparrow search algorithm (SSA) is proposed in this paper to optimize the random weights and bias of the ELM network and verified by experimental data. The results show that compared with traditional ELM and back-propagation neural networks, the prediction results of ELM optimized by SSA have lower mean absolute error percentages and root mean square errors, indicating that the SSA-ELM model has higher prediction accuracy and better stability and has obvious advantages in processing data with a high nonlinear degree.

Time-frequency fusion features-based incremental network for smartphone measured structural seismic response classification

Smartphone as a portable device can be used for public participatory structural seismic response acquisition. This data acquisition approach not only reduces the acquisition cost but also facilitates the possibility to obtain city-scale building seismic responses. In this paper, we propose a novel structural seismic response classification method using time–frequency fusion features-based incremental network, namely SeismicNet, that can automatically classify the signals acquired through smartphones into structural response under normal operation conditions (structural normal response) or structural response under strong ground motion (structural seismic response). In this method, the time-domain signals of structural response are converted into time–frequency maps, which will be fed into a lightweight convolutional network for the extraction of the initial fusion features. Subsequently, these features are mapped by random weights to dynamic nodes. This not only improves the capability of feature expression, but also allows incremental updating of the model. Finally, a simulated structural response dataset and some real structural seismic responses are used to verify the accuracy and practicability of the proposed method. Furthermore, the results showed that the proposed method is more than 84 times in training speed and 2 times in inference speed than some well-known deep learning networks (VGG16 and ResNet50).