Learning Accuracy Research Articles

Integration of machine learning and simulation for dynamic rescheduling in truck appointment systems

Traditionally, port terminals implement Truck Appointment System to control expected arrivals of trucks in port terminals. However, the occurrence of disruptions during transport causes unexpected circumstances, generating congestion and unbalanced arrivals at port hinterland. The use of smart technologies, in the context of Industry 4.0, is capable of collecting real-time geolocation and improve the visibility in the port logistics system, offering the opportunity to design a flexible TAS allowing dynamic rescheduling of truck appointments. In this sense, a new method combining a Machine Learning algorithm with discrete event simulation to predict truck conditions during the transport, and dynamically reschedule appointments of trucks classified as early and late trucks using real-time data acquired by smart technologies is proposed in this paper. The approach was evaluated in a Brazilian port terminal, and we use the average and the maximum waiting time of trucks in queue at the entrance gate, the average and the maximum number of trucks in queue at the entrance gate, and the percentage of trucks attended off-schedule as Key Performance Indicators. The Machine Learning accuracy was identified as 95.37% and the flexible method achieved a 90.4% improvement in the waiting time of trucks at port hinterland and a reduction of queue sizes compared to the current scenario, by the synchronization of arrivals in balanced time windows. Furthermore, this research contributes to increase the visibility in port logistics systems and reduce the vulnerability of port terminals.

Privacy-preserving distributed deep learning via LWE-based Certificateless Additively Homomorphic Encryption (CAHE)

Current privacy-preserving schemes proposed for distributed deep learning (DDL) are based on public key infrastructure (PKI) which has management of keys problem and also key escrow problem for those that may use identity-based public key cryptography (ID-PKC). This presents the possibility for the privacy guarantee of the distributed/collaborative deep learning system to be flawed due to the presence of a Trusted Third Party (TTP) (i.e. a certificate authority (CA) for public key cryptography (PKC) and a key generation center (KGC) for an ID-PKC), that must be trusted. Additionally, the presence of high communication cost associated with the transmission of homomorphically encrypted gradients from participants to the cloud server in distributed deep learning needs more attention. Furthermore, few schemes have been proposed for DDL that have post-quantum robustness. We, therefore, propose and design a new privacy-preserving distributed deep learning solution. First, a new LWE-based certificateless additively homomorphic encryption scheme called CAHE is proposed and designed. Secondly, partial sharing algorithms based on random selection and selection of the largest (or topmost) gradients in terms of magnitude are proposed. Thirdly, we put forward a new privacy-preserving distributed deep learning framework that combines the certificateless additively homomorphic encryption and partial sharing in the DDL. Our approach provides better privacy (i.e. not relying solely on the TTP) and also reduces communication cost for distributed deep learning. We note emphatically that this is the first time certificateless HE is being proposed, designed, and implemented in DDL. Results from our work indicate that the accuracy of the CAHE-based deep learning does not deteriorate (97.20%) while greatly preserving the privacy of participant’s dataset. With partial sharing, our solution records 97.17% and 97.12% accuracy for the selection of topmost gradients and random selection of gradients scenarios respectively, and with a great reduction in communication cost for transmitting encrypted gradients.

Spider-inspired tunable mechanosensor for biomedical applications

The recent advances of wearable sensors are remarkable but there are still limitations that they need to be refabricated to tune the sensor for target signal. However, biological sensory systems have the inherent potential to adjust their sensitivity according to the external environment, allowing for a broad and enhanced detection. Here, we developed a Tunable, Ultrasensitive, Nature-inspired, Epidermal Sensor (TUNES) that the strain sensitivity was dramatically increased (GF ~30k) and the pressure sensitivity could be tuned (10–254 kPa−1) by preset membrane tension. The sensor adjusts the sensitivity to the pressure regime by preset tension, so it can measure a wide range (0.05 Pa–25 kPa) with the best performance: from very small signals such as minute pulse to relatively large signals such as muscle contraction and respiration. We verified its capabilities as a wearable health monitoring system by clinical trial comparing with pressure wire which is considered the current gold standard of blood pressure (r = 0.96) and home health care system by binary classification of Old’s/Young’s pulse waves via machine learning (accuracy 95%).

Fast inspection and accurate recognition of target objects for astronaut robots through deep learning

Target object detection based on deep learning and image processing technology is widely used in robot astronauts. However, existing detection methods have limitations in detection speed and accuracy due to the complex space environment (such as uneven illumination and particle radiation) and inadequate training samples. Inspired by the neural network structure and transfer learning, a deep learning detection method for small samples in complex environments is proposed. A depthwise separable convolution is added to a feature fusion network to reduce the number of parameters in image output feature mapping, and a linear bottleneck inverted residual structure is introduced into a backbone feature extraction network to reduce the computation and memory requirements during feature extraction. As a result, a backbone feature extraction–fusion network structure is established to solve the problem in detection speed. A squeeze-and-excitation (SE) attention module is introduced in front of the head, and an SE detector is constructed to improve the detection accuracy in a spatially complex environment by dynamically assigning image channel weights to highlight the target object features in blurred images. The learning efficiency and accuracy of the network model in the small sample case in this paper are addressed by incorporating the transfer learning idea and establishing the evaluation function of learning samples. Experimental results show that the proposed algorithm enables astronaut robots to detect object rapidly and accurately in complex environments. The average speed (frames per second) and accuracy of detection under 2200 training samples are 45.19 and 93.14%, respectively.

Forecasting accuracy of machine learning and linear regression: evidence from the secondary CAT bond market

AbstractThe main challenge in empirical asset pricing is forecasting the future value of assets traded in financial markets with a high level of accuracy. Because machine learning methods can model relationships between explanatory and dependent variables based on complex, non-linear, and/or non-parametric structures, it is not surprising that machine learning approaches have shown promising forecasting results and significantly outperform traditional regression methods. Corresponding results were achieved for CAT bond premia forecasts in the primary market. However, since secondary market data sets have a panel data structure, it is unclear whether the results of primary market studies can be applied to the secondary market. Against this background, this study aims to build the first out-of-sample forecasting model for CAT bond premia in the secondary market, comparing different modeling approaches. We apply random forest and neural networks as representatives of machine learning methods and linear regression based on a comprehensive data set of CAT bond issues and across various forecasting settings and show that random forest forecasts are significantly more precise. Because the lack of transparency of machine learning methods may limit their applicability, especially for institutional investors, we show ways to identify important variables in the context of random forest price forecasting.

FMGNN: A Method to Predict Compound-Protein Interaction With Pharmacophore Features and Physicochemical Properties of Amino Acids.

Identifying interactions between compounds and proteins is an essential task in drug discovery. To recommend compounds as new drug candidates, applying the computational approaches has a lower cost than conducting the wet-lab experiments. Machine learning-based methods, especially deep learning-based methods, have advantages in learning complex feature interactions between compounds and proteins. However, deep learning models will over-generalize and lead to the problem of predicting less relevant compound-protein pairs when the compound-protein feature interactions are high-dimensional sparse. This problem can be overcome by learning both low-order and high-order feature interactions. In this paper, we propose a novel hybrid model with Factorization Machines and Graph Neural Network called FMGNN to extract the low-order and high-order features, respectively. Then, we design a compound-protein interactions (CPIs) prediction method with pharmacophore features of compound and physicochemical properties of amino acids. The pharmacophore features can ensure that the prediction results much more fit the expectation of biological experiment and the physicochemical properties of amino acids are loaded into the embedding layer to improve the convergence speed and accuracy of protein feature learning. The experimental results on several datasets, especially on an imbalanced large-scale dataset, showed that our proposed method outperforms other existing methods for CPI prediction. The western blot experiment results on wogonin and its candidate target proteins also showed that our proposed method is effective and accurate for finding target proteins. The computer program of implementing the model FMGNN is available at https://github.com/tcygxu2021/FMGNN.

AFARN: Domain Adaptation for Intelligent Cross-Domain Bearing Fault Diagnosis in Nuclear Circulating Water Pump

Domain adaptation can transfer cross-domain diagnosis knowledge by minimizing the divergence of labeled source and unlabeled target data. However, the model neglects to maximize physics prior knowledge during feature extraction and distribution alignment, resulting in a noninterpretable model even negative transfer. Hence, a physics-informed domain adaptation network, termed adaptive fault attention residual network (AFARN), is proposed. First, an adaptive fault attention mechanism is designed to refine features guided by bearing fault characteristics, suited to generating diagnosis-relevant features. Then, several metrics are applied to minimize the marginal and conditional distribution discrepancy of features, thus, generalizing the model from source to target domain. The AFARN utilizes the fault characteristics and label information simultaneously to train the model, which can enhance the distribution alignment of diagnosis-relevant features, thus, providing an interpretable knowledge transfer. Finally, experiments on public and circulating water pump datasets show that AFARN can enhance fault feature learning and diagnosis accuracy.

High stable and accurate vehicle selection scheme based on federated edge learning in vehicular networks

Federated edge learning (FEEL) technology for vehicular networks is considered as a promising technology to reduce the computation workload while keeping the privacy of users. In the FEEL system, vehicles upload data to the edge servers, which train the vehicles' data to update local models and then return the result to vehicles to avoid sharing the original data. However, the cache queue in the edge is limited and the channel between edge server and each vehicle is time-varying. Thus, it is challenging to select a suitable number of vehicles to ensure that the uploaded data can keep a stable cache queue in edge server while maximizing the learning accuracy. Moreover, selecting vehicles with different resource statuses to update data will affect the total amount of data involved in training, which further affects the model accuracy. In this paper, we propose a vehicle selection scheme, which maximizes the learning accuracy while ensuring the stability of the cache queue, where the statuses of all the vehicles in the coverage of edge server are taken into account. The performance of this scheme is evaluated through simulation experiments, which indicates that our proposed scheme can perform better than the known benchmark scheme.

FedTAR: Task and Resource-Aware Federated Learning for Wireless Computing Power Networks

In the 6G era, the proliferation of data and data-intensive applications poses unprecedented challenges on the current communication and computing networks. The collaboration among cloud computing, edge computing, and networking is imperative to process such massive data, eventually realizing ubiquitous computing and intelligence. In this article, we propose a wireless computing power network (WCPN) by orchestrating the computing and networking resources of heterogeneous nodes toward specific computing tasks. To enable intelligent service in WCPN, we design a task and resource-aware federated learning model, coined FedTAR, which minimizes the sum energy consumption of all computing nodes by the joint optimization of the computing strategies of individual computing nodes and their collaborative learning strategy. Based on the solution of the optimization problem, the neural network depth of computing nodes and the collaboration frequency among nodes are adjustable according to specific computing task requirements and resource constraints. To further adapt to heterogeneous computing nodes, we then propose an energy-efficient asynchronous aggregation algorithm for FedTAR, which accelerates the convergence speed of federated learning in WCPN. Numerical results show that the proposed scheme outperforms the existing studies in terms of learning accuracy, convergence rate, and energy saving.

Efficiency of Double-Barrier Magnetic Tunnel Junction-Based Digital eNVM Array for Neuro-Inspired Computing

This brief deals with the impact of spin-transfer torque magnetic random access memory (STT-MRAM) cell based on double-barrier magnetic tunnel junction (DMTJ) on the performance of a two-layer multilayer perceptron (MLP) neural network. The DMTJ-based cell is benchmarked against the conventional single-barrier MTJ (SMTJ) counterpart by means of a comprehensive evaluation carried out through a state-of-the-art device-to-algorithm simulation framework. The benchmark is based on the MNIST handwritten dataset, Verilog-A MTJ compact models developed by our group, and 0.8 V FinFET technology. Our results point out that the use of DMTJ-based STT-MRAM cells to implement digital embedded non-volatile memory (eNVM) synaptic core allows write/read energy and latency improvements of about 53%/61% and 66%/17%, respectively, as compared to the SMTJ-based equivalent design. This is achieved by ensuring a reduced area footprint and a learning accuracy of about 91%. Such results make the DMTJ-based STT-MRAM cell a good eNVM option for neuro-inspired computing.

One-Pass Online Learning Based on Gradient Descent for Multilayer Spiking Neural Networks

Online supervised learning algorithms update synaptic weights in real-time during the running process of spiking neural networks (SNNs), which are important for modeling the behavior and cognitive process of the brain. This article proposes an online supervised learning algorithm based on gradient descent for multilayer feedforward SNNs, where precisely timed spike trains are used to represent neural information. The online learning rule is derived from the real-time error function and backpropagation mechanism. The synaptic weights are adjusted when an output neuron fires a spike. Results of spike train learning demonstrate that the proposed online learning algorithm can achieve higher learning accuracy and requires fewer learning epochs than the corresponding offline learning method and other typical supervised learning algorithms. Furthermore, the proposed algorithm is used for solving pattern classification problems, where the one-pass learning approach is employed for training SNNs. Results show that the proposed algorithm can obtain comparable classification accuracy compared with other state-of-the-art algorithms even in the case of only one iteration. It indicates that the proposed algorithm is effective for solving spatio-temporal pattern recognition problems.

그래프 트랜스포머 기반 농가 사과 품질 이미지의 그래프 표현 학습 연구

Recently, a convolutional neural network (CNN) based system is being developed to overcome the limitations of human resources in the apple quality classification of farmhouse. However, since convolutional neural networks receive only images of the same size, preprocessing such as sampling may be required, and in the case of oversampling, information loss of the original image such as image quality degradation and blurring occurs. In this paper, in order to minimize the above problem, to generate a image patch based graph of an original image and propose a random walk-based positional encoding method to apply the graph transformer model. The above method continuously learns the position embedding information of patches which don`t have a positional information based on the random walk algorithm, and finds the optimal graph structure by aggregating useful node information through the self-attention technique of graph transformer model. Therefore, it is robust and shows good performance even in a new graph structure of random node order and an arbitrary graph structure according to the location of an object in an image. As a result, when experimented with 5 apple quality datasets, the learning accuracy was higher than other GNN models by a minimum of 1.3% to a maximum of 4.7%, and the number of parameters was 3.59M, which was about 15% less than the 23.52M of the ResNet18 model. Therefore, it shows fast reasoning speed according to the reduction of the amount of computation and proves the effect.

Influence of instructor's facial expressions in video lectures on motor learning in children with autism spectrum disorder.

During the COVID-19 pandemic, video materials have played crucial roles in supporting learning among children with autism spectrum disorder (ASD). This study aimed to explore the effects of the instructor's facial expressions in video lectures on attention and motor learning in children with ASD versus typically developing (TD) children. A total of 60 children were randomly assigned to four groups: (1) ASD-happy, (2) ASD-neutral, (3) TD-happy, and (4) TD-neutral. Both happy groups paid more attention to the video lectures. The ASD groups achieved higher accuracy and fidelity of motor learning when the instructor smiled. Results revealed that greater attention to video lectures predicted better performance in children with ASD. This study has practical implications for the design of learning materials for children with ASD: An instructor should be encouraged to show a happy expression to promote learning.

Notes on the improvement of concept-cognitive learning accuracy

The concept-cognitive learning (CCL) process is the specific implementation step of simulating the human brain to learn concepts, and the CCL model is its core carrier. Different CCL models constructed by different cognitive minds will produce different concept learning results. The existing CCL model based on sufficient and necessary granule approximations regards the human's approximation idea in the face of inconsistent information as a logical criterion, and aims at finding the closest concept pair of the clue as concept learning results with a certain learning accuracy. However, the existing CCL method based on sufficient and necessary granule approximations cannot guarantee that the clue must be between its lower approximation and upper approximation, which causes the fact that the learning accuracy may not effectively measure the consistency of concept learning results. What is more, although the computational process obeys logical cognitive condition, the concept learning results may not conform to the actual situation, such as the case of merely generating full concepts and empty concepts. For the first problem, we improve the learning accuracy of the existing CCL method, propose a new CCL method with learning accuracy under hybrid lattice structure, and develop CCL algorithms for the cases of objects and attributes as clues. Moreover, experiments show the effectiveness of the proposed CCL method with learning accuracy under hybrid lattice structure. For the second problem, we put forward a CCL method based on non-logical associative mechanism to handle the unreasonable situation where the concept learning results are full concepts and empty concepts. Finally, two associative CCL algorithms are explored, and experiments are conducted to show their effectiveness.

Assessing the accuracy of free automated plant identification applications

AbstractWidely available and inexpensive mobile phone applications offer users, whether professional ecologists or interested amateurs, the potential for simple and rapid automated identification of species, without the need to use field guides and identification keys. The increasing accuracy of machine learning is well established, but it is currently unclear if, and under what circumstances, free‐to‐use mobile phone applications are accurate for identifying plants to species level in real‐world field conditions.We test five popular and free identification applications for plants using 857 professionally identified images of 277 species from 204 genera. Across all applications, 85% of images were identified correctly in the top five suggestions, and 69% were correct with the first suggestion. Plant type (woody, forbs, grasses, rushes/sedges, ferns/horsetails) was a significant determinant of identification performance for each application. For some applications, image saliency was also important; exposure and focus were not significant.Applications performed well, with at least one of the three best‐performing applications identifying 96% of images correctly as their first suggestion. We conclude that, subject to some caveats, free phone‐based plant identification applications are valid and useful tools for those wanting rapid identification and for anyone wanting to engage with the natural world.Read the freePlain Language Summaryfor this article on the Journal blog.

AutoCRW: Learning based robust watermarking for smart city applications

AbstractDeep learning has become a promising model in the industry due to its superior learning accuracy and efficiency. In addition to conventional applications, such as fraud detection, natural language processing, image classification and reconstruction, object detection and segmentation this model can be widely used for data hiding, that is, watermarking. Existing transformed‐domain‐based watermarking provided better robustness toward attacks. In this article, an interesting autoencoder convolutional neural network (CNN)‐based watermarking technique, AutoCRW, is proposed, which can prevent intellectual property theft of digital images. First, the autoencoder functionality of CNN generates two versions of the same image, namely positive and negative version of the images, which decompose by a transformed domain scheme. Then, watermark information is embedded into the output images, which can be extracted to realize copyright protection and ownership verification. Finally, a denoising convolutional neural network (DnCNN) is employed over the extracted mark to ensure the robustness of the watermarking system. Extensive experiments demonstrate that the proposed algorithm has high invisibility and good robustness against several attacks.

Novel Inception V3 Deep Learning Model Designing for Alzheimer’s Disease Detection

Alzheimer's Disease (AD) is the extensive as well as untreatable neurodegenerative disease, as well as the important phase - Mild Cognitive Impairment (MCI) - is still a challenge to diagnose. This is called as a degenerative disease as well as it gets worse over time. AD is primarily a neurodegenerative disorder that is completely incurable. It not only harm mankind memory it also impacts responses to nature, movement and outer stimuli. In addition, Alzheimer's disease disruption the attachment of neurons and damages brain cells. This paper presents a Novel Inception V3 Deep learning model designing for Alzheimer’s disease observation. The goal to attain an automatic patient’s classification from the MRI Scanned Images of Alzheimer's disease and Mild Cognitive Impairment to assist medical doctor in Identification analysis. Several steps are included in this detection method. Comparative analysis of individual machine learning classifiers as K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Decision Tree (DT), RF (Random Forest) with Ensemble method (SVM & KNN) is provided in the result section. Obtained Accuracy of ensemble machine learning is 97.5% and similarly, Precision, Recall, F1-Score values are 96.4%, 96.6%, and 97.3% respectively. Results state that effectiveness of ensemble machine learning model detecting the AD.

Anhedonia in Relation to Reward and Effort Learning in Young People with Depression Symptoms.

Anhedonia, a central depression symptom, is associated with impairments in reward processing. However, it is not well understood which sub-components of reward processing (anticipation, motivation, consummation, and learning) are impaired in association with anhedonia in depression. In particular, it is unclear how learning about different rewards and the effort needed to obtain them might be associated with anhedonia and depression symptoms. Therefore, we examined learning in young people (N = 132, mean age 20, range 17-25 yrs.) with a range of depression and anhedonia symptoms using a probabilistic instrumental learning task. The task required participants to learn which options to choose to maximize their reward outcomes across three conditions (chocolate taste, puppy images, or money) and to minimize the physical effort required to obtain the rewards. Additionally, we collected questionnaire measures of anticipatory and consummatory anhedonia, as well as subjective reports of "liking", "wanting" and "willingness to exert effort" for the rewards used in the task. We found that as anticipatory anhedonia increased, subjective liking and wanting of rewards decreased. Moreover, higher anticipatory anhedonia was significantly associated with lower reward learning accuracy, and participants demonstrated significantly higher reward learning than effort learning accuracy. To our knowledge, this is the first study observing an association of anhedonia with reward liking, wanting, and learning when reward and effort learning are measured simultaneously. Our findings suggest an impaired ability to learn from rewarding outcomes could contribute to anhedonia in young people. Future longitudinal research is needed to confirm this and reveal the specific aspects of reward learning that predict anhedonia. These aspects could then be targeted by novel anhedonia interventions.

Use of machine learning tools and NIR spectra to estimate residual moisture in freeze-dried products

Residual Moisture (RM) in freeze-dried products is one of the most important critical quality attributes (CQAs) to monitor, since it affects the stability of the active pharmaceutical ingredient (API). The standard experimental method adopted for the measurements of RM is the Karl-Fischer (KF) titration, that is a destructive and time-consuming technique. Therefore, Near-Infrared (NIR) spectroscopy was widely investigated in the last decades as an alternative tool to quantify the RM. In the present paper, a novel method was developed based on NIR spectroscopy combined with machine learning tools for the prediction of RM in freeze-dried products. Two different types of models were used: a linear regression model and a neural network based one. The architecture of the neural network was chosen so as to optimize the prediction of the residual moisture, by minimizing the root mean square error with the dataset used in the learning step. Moreover, the parity plots and the absolute error plots were reported, allowing a visual evaluation of the results. Different factors were considered when developing the model, namely the range of wavelengths considered, the shape of the spectra and the type of model. The possibility of developing the model using a smaller dataset, obtained with just one product, that could be then applied to a wider range of products was investigated, as well as the performance of a model developed for a dataset encompassing several products. Different formulations were analyzed: the main part of the dataset was characterized by a different percentage of sucrose in solution (3%, 6% and 9% specifically); a smaller part was made up of sucrose-arginine mixtures at different percentages and only one formulation was characterized by another excipient, the trehalose. The product-specific model for the 6% sucrose mixture was found consistent for the prediction of RM in other sucrose containing mixtures and in the one containing trehalose, while failed for the dataset with higher percentage of arginine. Therefore, a global model was developed by including a certain percentage of all the available dataset in the calibration phase. Results presented and discussed in this paper demonstrate the higher accuracy and robustness of the machine learning based model with respect to the linear models.

Federated learning by employing knowledge distillation on edge devices with limited hardware resources

This paper presents a federated learning approach based on utilizing computational resources of the IoT edge devices for training deep neural networks. In this approach, the edge devices and the cloud server collaborate in the training phase while preserving the privacy of the edge device data. Owing to the limited computational power and resources available to the edge devices, instead of the original neural network (NN), we suggest to use a smaller NN generated using a proposed heuristic method. In the proposed approach, the smaller model, which is trained on the edge device, is generated from the main NN model. By the exploiting Knowledge Distillation (KD) approach, the learned knowledge in the server and the edge devices can be exchanged, leading to lower required computation on the server and preserving data privacy of the edge devices. Also, to reduce the knowledge transfer overhead on the communication links between the server and the edge devices, a method for selecting the most valuable data to transfer the knowledge is introduced. The effectiveness of this method is assessed by comparing it to state-of-the-art methods. The results show that the proposed method lowers the communication traffic by up to 250 × and increases the learning accuracy by an average of 8.9 % in the cloud compared to the prior KD-based distributed training approaches in CIFAR-10 dataset.