Output Of Neural Network Research Articles

Category-Aware Multimodal Attention Network for Fashion Compatibility Modeling

Fashion compatibility modeling, which is used to estimate the matching degree of a given set of fashion items, has received increasing attention in recent years. However, existing studies often fail to fully leverage multimodal information or ignore the semantic guidance of clothing categories in elevating the reliability of multimodal information. In this paper, we propose a fashion compatibility modeling approach with a category-aware multimodal attention network, termed as FCM-CMAN. In FCM-CMAN, we focus on enriching and aggregating multimodal representations of fashion items by means of the dynamic representations of categories and a contextual attention mechanism simultaneously. Specifically, considering that category correlations are always dynamic and varied for different fashion items, we design a categorical dynamic graph convolutional network to adaptively learn the semantic correlations between categories. When combined with the multi-layered visual outputs of a convolutional neural network and the surrounding contextual information, multiple content-aware category representations and context-aware attention weights are obtained to better characterize fashion items from different aspects. On this basis, two pieces of aware information are integrated by a multimodal factorized bilinear pooling strategy to generate visual-semantic embeddings, which are further improved by a multi-head self-attention mechanism to capture significant elements related to fashion compatibility. Extensive experiments conducted on the FashionVC and ExpFashion datasets demonstrate the superiority of FCM-CMAN over state-of-the-art methods.

A Hybridized Artificial Neural Network for Automated Software Test Oracle

Software testing is the methodology of analyzing the nature of software to test if it works as anticipated so as to boost its reliability and quality. These two characteristics are very critical in the software applications of present times. When testers want to perform scenario evaluations, test oracles are generally employed in the third phase. Upon test case execution and test outcome generation, it is essential to validate the results so as to establish the software behavior’s correctness. By choosing a feasible technique for the test case optimization and prioritization as along with an appropriate assessment of the application, leads to a reduction in the fault detection work with minimal loss of information and would also greatly reduce the cost for clearing up. A hybrid Particle Swarm Optimization (PSO) with Stochastic Diffusion Search (PSO-SDS) based Neural Network, and a hybrid Harmony Search with Stochastic Diffusion Search (HS-SDS) based Neural Network has been proposed in this work. Further to evaluate the performance, it is compared with PSO- SDS based artificial Neural Network (PSO-SDS ANN) and Artificial Neural Network (ANN). The Misclassification of correction output (MCO) of HS-SDS Neural Network is 6.37 for 5 iterations and is well suited for automated testing.

Generating Embedding Features Using Deep Learning for Ethnics Recognition

Although significant advances have been made recently in the field of ethnics recognition through face recognition, there is still a lack of studies of ethnics recognition through facial recognition. This study is concerned with ethnics recognition through facial representation using a few images used as samples for any selected group of ethnics using a deep neural network with a Variational Feature Learning (VFL) loss function that has been used to increase the performance accuracy during the evaluation process. The output of a deep neural network is an embedding of 128 bytes for each face image in each group of ethnics. After that, all embeddings of every face in each group of ethnics pass to a machine learning classification method like a Support Vector Machine (SVM). We achieved state-of-the-art ethnic recognition. The system achieved a classification accuracy of 97.3% on a collected group of image dataset collected from three different countries.

Passing Vehicle Road Occupancy Detection Using the Magnetic Sensor Array

The increasing presence of vehicles on roads necessitates intelligent traffic management solutions in areas where video cameras cannot be utilized. Currently, there are limited choices for depersonalized vehicle reidentification systems. This paper introduces a system that later will be used for vehicle reidentification. The system uses anisotropic magnetoresistive sensors and is based on the hypothesis that each vehicle leaves unique magnetic signatures which can be used for comparison and matching. Vehicle location on the road perpendicular to sensor array detection methodology is presented in this work. An array of magnetic sensors is installed in asphalt across the vehicle’s driving direction. The system continuously measures Earth’s natural magnetic field and detects distortions when vehicles pass an array of a sensors and then logs magnetic signatures. Useful parameters from raw sensor axes are calculated– modules and derivatives. Applying signal-to-noise ratio calculation for module derivatives between ambient noise and signal gives important features for neural network input. Different types of neural network architectures and output result interpretation techniques are investigated. Further, after evaluating network output it is possible to label sensor nodes that are directly beneath the vehicle. Experiment results show that implemented algorithm is highly sufficient for valid sensors under the vehicle selection. Correct sensor selection is important for further re-identification algorithms.

Gradient Descent-Barzilai Borwein-Based Neural Network Tracking Control for Nonlinear Systems With Unknown Dynamics.

In this article, a combined gradient descent-Barzilai Borwein (GD-BB) algorithm and radial basis function neural network (RBFNN) output tracking control strategy was proposed for a family of nonlinear systems with unknown drift function and control input gain function. In such a method, a neural network (NN) is used to approximate the controller directly. The main merits of the proposed strategy are given as follows: first, not only the NN parameters, such as weights, centers, and widths but also the learning rates of NN parameter updating laws are updated online via the proposed learning algorithm based on Barzilai-Borwein technique; and second, the controller design process can be further simplified, the controller parameters that should be tuned can be greatly reduced. Theoretical analysis about the stability of the closed-loop system is manifested. In addition, simulations were conducted on a numerical discrete time system and an inverted pendulum system to validate the presented control strategy.

Multi-output physics-informed neural network for one- and two-dimensional nonlinear time distributed-order models

<abstract><p>In this article, a physics-informed neural network based on the time difference method is developed to solve one-dimensional (1D) and two-dimensional (2D) nonlinear time distributed-order models. The FBN-$ \theta $, which is constructed by combining the fractional second order backward difference formula (BDF2) with the fractional Newton-Gregory formula, where a second-order composite numerical integral formula is used to approximate the distributed-order derivative, and the time direction at time $ t_{n+\frac{1}{2}} $ is approximated by making use of the Crank-Nicolson scheme. Selecting the hyperbolic tangent function as the activation function, we construct a multi-output neural network to obtain the numerical solution, which is constrained by the time discrete formula and boundary conditions. Automatic differentiation technology is developed to calculate the spatial partial derivatives. Numerical results are provided to confirm the effectiveness and feasibility of the proposed method and illustrate that compared with the single output neural network, using the multi-output neural network can effectively improve the accuracy of the predicted solution and save a lot of computing time.</p></abstract>

Error Dynamics Based Dual Heuristic Dynamic Programming for Self-Learning Flight Control

A data-driven nonlinear control approach, called error dynamics-based dual heuristic dynamic programming (ED-DHP), is proposed for air vehicle attitude control. To solve the optimal tracking control problem, the augmented system is defined by the derived error dynamics and reference trajectory so that the actor neural network can learn the feedforward and feedback control terms at the same time. During the online self-learning process, the actor neural network learns the control policy by minimizing the augmented system’s value function. The input dynamics identified by the recursive least square (RLS) and output of the critic neural network are used to update the actor neural network. In addition, the total uncertainty term of the error dynamics is also identified by RLS, which can compensate for the uncertainty caused by inaccurate modeling, parameter perturbation, and so on. The outputs of ED-DHP include the rough trim surface, feedforward and feedback terms from the actor neural network, and the compensation. Based on this control scheme, the complete knowledge of system dynamics and the reference trajectory dynamics are not needed, and offline learning is unnecessary. To verify the self-learning ability of ED-DHP, two numerical experiments are carried out based on the established morphing air vehicle model. One is sinusoidal signal tracking at a fixed operating point, and the other is guidance command tracking with a morphing process at variable operating points. The simulation results demonstrate the good performance of ED-DHP for online self-learning attitude control and validate the robustness of the proposed scheme

Effect of Resistance and Flexibility Exercise Intervention on Balance in Older Adults

The main objective of our study is to analyze the effect of resistance and flexibility exercise intervention on balance in older adults. Balance is the ability to maintain an upright posture during static and dynamic tasks. Maintaining balance involves a complex interaction between intrinsic factors, including peripheral, visual and vestibular sensation and muscle factors, and the interplay between the neural network and motor output, which are processed and mediated centrally. Programs to increase physical exercise among elderly persons have been found to be beneficial to health. Evidence suggests that training should combine both flexibility exercises and resistance exercises. Berg Balance Scale was the outcome measure. Twenty subjects were selected and divided into two groups. Group A (10 subjects) were given Resisted exercises for 40-45 minutes with two sets of 10 � 12 repetitions for ten weeks. Group B (10 subjects) were given flexibility exercises for 40-45 minutes and included 10�12 repetitions for ten weeks. On comparing the mean values of Group A & Group B on the Berg Balance Scale score, it shows a significant increase in the post-test mean values in both groups, but (Group A - Resistance Exercise) shows 51.10 which has the higher Mean value is effective than (Group B - Flexibility exercise) 46.40 at P = 0.001. Comparing Pretest and Posttest within Group A & B on the Berg Balance Scale score shows a highly significant difference in mean values at P = 0. 001. This study concludes that resisted exercise significantly improved balance in older adults than in flexibility exercise.

Vision-Based Activity Classification of Excavators by Bidirectional LSTM

Advancements in deep learning and vision-based activity recognition development have significantly improved the safety, continuous monitoring, productivity, and cost of the earthwork site. The construction industry has adopted the CNN and RNN models to classify the different activities of construction equipment and automate the construction operations. However, the currently available methods in the industry classify the activities based on the visual information of current frames. To date, the adjacent visual information of current frames has not been simultaneously examined to recognize the activity in the construction industry. This paper proposes a novel methodology to classify the activities of the excavator by processing the visual information of video frames adjacent to the current frame. This paper follows the CNN-BiLSTM standard deep learning pipeline for excavator activity recognition. First, the pre-trained CNN model extracted the sequential pattern of visual features from the video frames. Then BiLSTM classified the different activities of the excavator by analyzing the output of the pre-trained convolutional neural network. The forward and backward LSTM layers stacked on help the algorithm compute the output by considering previous and upcoming frames’ visual information. Experimental results have shown the average precision and recall to be 87.5% and 88.52%, respectively.

Evaluation of neural networks defenses and attacks using NDCG and reciprocal rank metrics

The problem of attacks on neural networks through input modification (i.e., adversarial examples) has attracted much attention recently. Being relatively easy to generate and hard to detect, these attacks pose a security breach that many suggested defenses try to mitigate. However, the evaluation of the effect of attacks and defenses commonly relies on traditional classification metrics, without adequate adaptation to adversarial scenarios. Most of these metrics are accuracy-based and therefore may have a limited scope and low distinctive power. Other metrics do not consider the unique characteristics of neural network functionality or measure the effectiveness of the attacks indirectly (e.g., through the complexity of their generation). In this paper, we present two metrics that are specifically designed to measure the effect of attacks, or the recovery effect of defenses, on the output of neural networks in multiclass classification tasks. Inspired by the normalized discounted cumulative gain and the reciprocal rank metrics used in information retrieval literature, we treat the neural network predictions as ranked lists of results. Using additional information about the probability of the rank enabled us to define novel metrics that are suited to the task at hand. We evaluate our metrics using various attacks and defenses on a pre-trained VGG19 model and the ImageNet dataset. Compared to the common classification metrics, our proposed metrics demonstrate superior informativeness and distinctiveness.

A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm.

Market making is a high-frequency trading problem for which solutions based on reinforcement learning (RL) are being explored increasingly. This paper presents an approach to market making using deep reinforcement learning, with the novelty that, rather than to set the bid and ask prices directly, the neural network output is used to tweak the risk aversion parameter and the output of the Avellaneda-Stoikov procedure to obtain bid and ask prices that minimise inventory risk. Two further contributions are, first, that the initial parameters for the Avellaneda-Stoikov equations are optimised with a genetic algorithm, which parameters are also used to create a baseline Avellaneda-Stoikov agent (Gen-AS); and second, that state-defining features forming the RL agent's neural network input are selected based on their relative importance by means of a random forest. Two variants of the deep RL model (Alpha-AS-1 and Alpha-AS-2) were backtested on real data (L2 tick data from 30 days of bitcoin-dollar pair trading) alongside the Gen-AS model and two other baselines. The performance of the five models was recorded through four indicators (the Sharpe, Sortino and P&L-to-MAP ratios, and the maximum drawdown). Gen-AS outperformed the two other baseline models on all indicators, and in turn the two Alpha-AS models substantially outperformed Gen-AS on Sharpe, Sortino and P&L-to-MAP. Localised excessive risk-taking by the Alpha-AS models, as reflected in a few heavy dropdowns, is a source of concern for which possible solutions are discussed.

The Experimental Study of Dust Effect on Solar Panel Efficiency

This study shows the dust effect on solar panel efficiency based on experimental measurements. Sieve analysis has been performed to investigate the effect on the efficiency of the panel. Dust particles have been divided into different sizes such as (-75), (+75/-105), (+105/-250), (+250/-450). In addition, each piece weighed 20 g, uniformly sprinkled onto a 40-Watt SUNNY polycrystalline panel. In order to obtain the output voltages of the PV panel used a data acquisition card. Additionally, a photodiode sensor has been used in order to obtain the light reflection data from the panel to the outside while there had covered different diameters of dust particles on the panel. The data set consisting of electrical parameters have been used to compered both experimental study and Artificial Neural Network (ANN) output. There were 5 different data sets with the clean PV panel in this study (randomly selected data from it used as %40 training - %60 test) and the clean panel data have been used twice in the training and test part of the ANN. In this study, it has been observed that the ANN method can be used to estimate panel efficiency due to the linearity (The R-value was gotten nearly equal to 1 and it shows that there is a linear relationship between outputs and targets.) if the appropriate transfer function is selected. Also, this study shows that as the particle diameter covering the panel gets smaller, the output voltage of the panel decreases linearly.

Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection

Deep learning techniques show success in detecting objects in medical images, but still suffer from false-positive predictions that may hinder accurate diagnosis. The estimated uncertainty of the neural network output has been used to flag incorrect predictions. We study the role played by features computed from neural network uncertainty estimates and shape-based features computed from binary predictions in reducing false positives in liver lesion detection by developing a classification-based post-processing step for different uncertainty estimation methods. We demonstrate an improvement in the lesion detection performance of the neural network (with respect to F1-score) for all uncertainty estimation methods on two datasets, comprising abdominal MR and CT images respectively. We show that features computed from neural network uncertainty estimates tend not to contribute much toward reducing false positives. Our results show that factors like class imbalance (true over false positive ratio) and shape-based features extracted from uncertainty maps play an important role in distinguishing false positive from true positive predictions.

Real-time pedestrian pose estimation, tracking and localization for social distancing.

The corona virus pandemic has introduced limitations which were previously not a cause for concern. Chief among them are wearing face masks in public and constraints on the physical distance between people as an effective measure to reduce the virus spread. Visual surveillance systems, which are common in urban environments and initially commissioned for security surveillance, can be re-purposed to help limit the spread of COVID-19 and prevent future pandemics. In this work, we propose a novel integration technique for real-time pose estimation and multiple human tracking in a pedestrian setting, primarily for social distancing, using CCTV camera footage. Our technique promises a sizeable increase in processing speed and improved detection in very low-resolution scenarios. Using existing surveillance systems, pedestrian pose estimation, tracking and localization for social distancing (PETL4SD) is proposed for measuring social distancing, which combines the output of multiple neural networks aided with fundamental 2D/3D vision techniques. We leverage state-of-the-art object and pose estimation algorithms, combining their strengths, for increase in speed and improvement in detections. These detections are then tracked using a bespoke version of the FASTMOT algorithm. Temporal and analogous estimation techniques are used to deal with occlusions when estimating posture. Projective geometry along with the aforementioned posture tracking is then used to localize the pedestrians. Inter-personal distances are calculated and locally inspected to detect possible violations of the social distancing rules. Furthermore, a "smart violations detector" is employed which estimates if people are together based on their current actions and eliminates false social distancing violations within groups. Finally, distances are intuitively visualized with the right perspective. All implementation is in real time and is performed on Python. Experimental results are provided to validate our proposed method quantitatively and qualitatively on public domain datasets using only a single CCTV camera feed as input. Our results show our technique to outperform the baseline in speed and accuracy in low-resolution scenarios. The code of this work will be made publicly available on GitHub at https://github.com/bilalze/PETL4SD.

WITHDRAWN: Semi-Supervised Learning with Ensemble Deep Learning Networks for Descriptor Generation

Deep neural networks and machine learning have made many real-world tasks easier. They can analyze large data sets that humans can't. Deep neural networks can be utilized for image, video, and sound matching, making them an intriguing research area. This research focuses on semi-supervised training and proposes a "enhanced semi-supervised" learning strategy. Robots can learn from labeled and unlabeled data with semi-supervised learning. Supervised learning employs labeled data. Semi-supervised learning is used to govern the Deep neural network's output. Assigning values to input groups and accessing the output area have no interaction. This method seeks to deliver a more efficient learning approach with an equitable distribution of output throughout the output field of space, and the authors developed an ensemble strategy based on three deep learning approaches (Convolution Deep neural network, Alexnet, and MobileNetv2). The result is a more effective learning approach. The handwritten digit experiment was 90.74 percent accurate, whereas Alzheimer's detection was 99.76 percent accurate. When the proposed strategy was used on two experimental data sets, the accuracy was better than when different applications used Siamese Deep neural networks.

TPE-based Short-Term Load Forecasting Neural Network Structure Design and Parameter Optimization Method

Short-term load forecasting is generally applied in power system real-time dispatching and day-ahead generation planning, which is of great significance for power system economic dispatching and safe operation of the system. Much research on short-term load forecasting using neural networks has been carried out at home and abroad. However, how to obtain the optimal network and structure evolution accurately and quickly has evolved into a key challenge for short-term load forecasting, because the prediction effect of the neural network method is more easily affected by the existing network structure and parameters, and the personality difference of the prediction object itself makes it difficult for the parameters to be reused. Aiming at this problem, this paper proposes a TPE-based neural network structure design and parameter optimization method for short-term load forecasting. This method realizes fast optimization of parameters of different types, structures, parameters, and input and output of deep neural networks, and is widely applied in Guangxi power grid. It is verified by an actual example that the method can effectively improve prediction accuracy.

Inverse-model-based iterative learning control for unknown MIMO nonlinear system with neural network

This paper provides an inverse-model-based iterative learning control (ILC) for the unknown multi-input multi-output (MIMO) nonlinear system with neural network (NN), where a novel gradient adaptive law is used to update the NN weights both hidden and output layers such a faster convergence can be achieved. First, a three-layer NN structure is introduced to observe the MIMO nonlinear system with input–output data, and a new gradient algorithm is proposed to update the unknown parameters of both hidden and output layers. Then, the input dynamic can be obtained with the NN observer, and the inversion-model-based control is designed. Moreover, the ideal inversion control can be obtained based on the reference signal, and the inverse ILC is designed. The stability of the NN observer and the convergence of the inverse-model-based control are analyzed. Finally, a SCARA manipulator MIMO model is simulated to illustrate the correctness of the proposed methods.

The Utilization and Modeling of Photo‐Fenton Process as a Single Unit in Textile Wastewater Treatment

AbstractStudies on the direct application of the photo‐Fenton process (PFOP) to disinfect and decontaminate textile wastewater are rare. The output of the artificial neural network (ANN) models applied to the wastewater of a textile factory producing woven fabrics, which is used to assess the efficiency of the PFOP process, are investigated and compared with each other in this study. The highest PFOP efficiency is obtained at a pH of 3. Chemical oxygen demand (COD), suspended solids (SS) and color removal rates are 94%, 90%, and 96%, respectively. The data are modeled with ANNs and nonlinear external input autoregressive ANNs (NARX‐ANN) using the MATLAB R2020a software program. Both Levenberg–Marquardt (trainlm) and scaled conjugate gradient (trainscg) algorithms are employed in the ANN and NARX‐ANN models, whereas hyperbolic tangent sigmoid (Tansig) and logistic sigmoid (Logsig) functions are superimposed on the hidden layer in the ANN model, and Tansig functions are superimposed on the NARX‐ANN model. It is determined that the developed ANN models are more effective in estimating the PFOP efficiency. The mean squared error is 0.000 953, and the coefficient of determination (R2) is 0.96 661.

Adaptive Neural Network Output Feedback Control of Uncertain Underactuated Systems With Actuated and Unactuated State Constraints

Underactuated systems are widely applied in industry, construction, manufacturing, etc., and the complex working environment puts forward higher demands for safety and transient performance. Hence, it is necessary to consider how to simultaneously ensure actuated and unactuated motion constraints by fewer control inputs, especially when systems suffer from model uncertainties, unavailable velocities, etc. Unfortunately, it is still a significant challenge to overcome in real applications and theoretical analysis. Additionally, most existing studies merely consider the specific control objects and few general methods are applicable to a class of underactuated systems. To this end, we design a new adaptive output-feedback controller for a class of uncertain underactuated systems. Compared with existing methods only handling actuated constraints, an important merit of this article is that by introducing the elaborately designed coupling term composed of actuated and unactuated constraints together, all state variables are kept within the preset time-variant ranges and converge to their desired values. Furthermore, a new Lyapunov function candidate is utilized to provide a theoretical guarantee. As far as we know, without the need of exact model knowledge and velocity feedback, this article provides the first solution to achieve accurate motion control and state constraints for both actuated and unactuated variables, which is meaningful both theoretically and practically. Meanwhile, the asymptotic stability of the equilibrium point for the closed-loop system is proven by utilizing Lyapunov techniques and Barbalat’s lemma. For verification, the presented controller is applied to underactuated overhead and rotary cranes, respectively, together with detailed theoretical analysis and experimental validations.

A Deep Learning Approach for Credit Scoring Using Feature Embedded Transformer

In this paper, we introduce a transformer into the field of credit scoring based on user online behavioral data and develop an end-to-end feature embedded transformer (FE-Transformer) credit scoring approach. The FE-Transformer neural network is composed of two parts: a wide part and a deep part. The deep part uses the transformer deep neural network. The output of the deep neural network and the feature data of the wide part are concentrated in a fusion layer. The experimental results show that the FE-Transformer deep learning model proposed in this paper outperforms the LR, XGBoost, LSTM, and AM-LSTM comparison methods in terms of area under the receiver operating characteristic curve (AUC) and the Kolmogorov–Smirnov (KS). This shows that the FE-Transformer deep learning model proposed in this paper can accurately predict user default risk.