Standard Backpropagation Research Articles

Capturing Relevant Context for Visual Tracking

Studies have shown that contextual information can promote the robustness of trackers. However, trackers based on convolutional neural networks (CNNs) only capture local features, which limits their performance. We propose a novel relevant context block (RCB), which employs graph convolutional networks to capture the relevant context. In particular, it selects the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> largest contributors as nodes for each query position (unit) that contain meaningful and discriminative contextual information and updates the nodes by aggregating the differences between the query position and its contributors. This operation can be easily incorporated into the existing networks and can be easily end-to-end trained using a standard backpropagation algorithm. To verify the effectiveness of RCB, we apply it to two trackers, SiamFC and GlobalTrack, respectively, and the two improved trackers are referred to as Siam-RCB and GlobalTrack-RCB. Extensive experiments on OTB, VOT, UAV123, LaSOT, TrackingNet, OxUvA, and VOT2018LT show the superiority of our method. For example, our Siam-RCB outperforms SiamFC by a very large margin (up to 11.2% in the success score and 7.8% in the precision score) on the OTB-100 benchmark.

Abstract 13186: Prevalence, Predictors, and Outcomes of Both True- and Pseudo-resistant Hypertension in the Action to Control Cardiovascular Risk in Diabetes Trial (ACCORD)

Introduction: Resistant hypertension (RH) has been poorly studied due to the difficulty distinguishing RH from medication non-adherence—exclusion of which is necessary to accurately diagnose RH. Therefore, little is known about prevalence, predictors, and outcomes of true RH, particularly for patients with diabetes. Objective: To characterize prevalence, predictors, and outcomes of true RH under a definition accounting for adherence in a study with strict adherence monitoring. Methods: We analyzed data from 1838 patients in the standard BP arm of ACCORD. Patients were classified into 3 mutually-exclusive groups: “true RH”, “pseudo-RH” (i.e., patients with BP treatment levels that would classify them as having RH, but who were non-adherent), and “Other” (i.e., those without “true RH” or “pseudo-RH” classifications). Predictors of true- and pseudo-RH were identified with logistic and boosted regression models. Relationship between true RH and primary composite outcome of nonfatal MI, nonfatal stroke, or cardiovascular death was evaluated with cox models. Results: Among 1838 participants with complete information, 489 (26.6%) met the definition of true RH, and 94 (16.1%) of RH patients had “pseudo-RH” on ≥1 visit over the first 12 months. Multivariable predictors of RH included: baseline SBP ≥160 mmHg (OR=8.79; 95CI:5.70-13.68), baseline SBP between 140-159 (OR=2.91; 95CI:2.13-4.00) compared to SBP &lt;140, additional baseline BP medication (OR=3.40; 95CI:2.83-4.11), macroalbuminuria (OR=2.35; 95CI:1.50-3.67), CKD (OR=1.53; 95CI:0.99-2.33), history of stroke (OR=1.73; 95CI:1.04-2.82), and black race (OR=1.39; 95CI:1.02-1.88); Overall, multivariable model had a cross-validated C-statistic of 0.80, indicating strong discriminative ability. “True RH” patients had a 65% increased hazard in primary outcome compared with other groups (HR=1.65; 95 CI:1.13-2.42). Conclusions: The majority of patients classified as RH had “true RH,” with only 16% being “pseudo-RH” due to non-adherence. True RH is more common among those who are black, have macroalbuminuria, CKD, history of stroke, higher baseline SBP, and are taking more baseline anti-hypertensives. “True RH” patients are at increased risk for cardiovascular and mortality events.

Abstract 14373: Efficacy and Safety of Antihypertensive Drug Classes: The Systolic Blood Pressure Intervention Trial (SPRINT)

Purpose: To assess the efficacy and safety of antihypertensive drug classes, including angiotensin converting enzyme inhibitors/angiotensin II receptor blockers (ACEi/ARB), beta blockers (BB), calcium channel blockers (CCB), and thiazide diuretics (TD), in subjects at high cardiovascular (CV) risk. Methods: SPRINT was a randomized, controlled, open-label trial in which individuals without diabetes aged ≥50 years, at high CV risk, and with a systolic blood pressure (SBP) 130-180 mmHg were randomized to intensive (SBP target &lt;120mmHg) or standard BP control (SBP target 135-139 mmHg). The primary outcome was the composite of acute coronary syndromes, stroke, heart failure, or CV death. Safety outcomes included serious adverse events, i.e., hypotension, syncope, electrolyte abnormalities, acute kidney injury or failure, and falls. Associations between baseline antihypertensive drug classes, efficacy, and safety outcomes, stratified by level of SBP control, were examined using Cox proportional-hazards regression. Results: Of 9361 participants, baseline use of antihypertensive agents was as follows: ACEi/ARB in 1317 (14%), BB in 911 (10%), CCB in 796 (9%), and TD in 979 (10%). A total of 1366 (15%) subjects did not have a record of being on an antihypertensive drug at baseline. In the intensive BP control group, use of a BB-based regimen at baseline was associated with a significantly higher risk of the primary outcome when compared with no medications ( Figure ). Similar patterns were observed for secondary efficacy endpoints. The risk of serious adverse events tended to be lower in patients receiving a treatment regimen containing either an ACEi/ARB or a TD compared with those receiving a regimen containing a BB or a CCB ( Figure ). Conclusions: In SPRINT, the risk of adverse events was lowest in patients who were not on an antihypertensive drug at baseline. ACEi/ARB-based and TD-based regimens appeared to have the best balance between efficacy and safety.

Brachial-cuff excess pressure is associated with carotid intima-media thickness among Australian children: a cross-sectional population study.

Reservoir pressure parameters (i.e., reservoir pressure [RP] and excess pressure [XSP]) independently predict cardiovascular events in adults, but this has not been investigated in children. This study aimed to determine (1) the association of reservoir pressure parameters with carotid intima-media thickness (carotid IMT), a preclinical vascular phenotype, and (2) whether a multivariable regression model with or without reservoir pressure parameters fits better for estimating carotid IMT in children. Study participants were 11-12-year-old children (n = 1231, 50% male) from the Child Health CheckPoint study, a cross-sectional substudy of the population-based Longitudinal Study of Australian Children. RP and XSP were obtained using brachial-cuff oscillometry (SphygmoCor XCEL, AtCor, Sydney). Carotid IMT was quantified by vascular ultrasonography. XSP was associated with carotid IMT after adjusting for confounders including age, sex, BMI z-score, heart rate, pubertal stage, moderate-to-vigorous physical activity, and mean arterial pressure (β = 0.93 µm, 95% CI 0.30-1.56 for XSP peak and β = 0.04 µm, 95% CI 0.01-0.08 for XSP integral). The results of the likelihood ratio test indicated a trend that the model with XSP and the above confounders fit better than a similar model without XSP for estimating carotid IMT. Our findings indicate that brachial-cuff device-measured XSP is associated with carotid IMT independent of conventional cardiovascular risk factors, including standard BP. This implies that a clinically convenient cuff approach could provide meaningful information for the early assessment of cardiovascular risk among children.

Artificial Neural Network based prediction of a direct injected diesel engine performance and emission characteristics powered with biodiesel

In this assessment, the ANN (Artificial Neural Network) offers the display of a diesel engine using biodiesel fuel to predict engine emissions and overall performance. In order to collect training and testing data for the planned ANN, a single-cylinder, 4-stroke diesel engine will be powered with biodiesel and diesel fuel varieties and will be run at variable load at stable engine rpm. Preliminary outcomes revealed that blends of biodiesel offer higher performance of an engine and enhanced emission qualities. An ANN model was progressed to be envisioning a relationship between brake thermal performance and exhaust emanations, such as carbon monoxide (CO), unburned hydrocarbon (HC), nitrogen oxides (NOx) and smoke intensity, the utilization of biodiesel-diesel blends and loads as input data. Approximately 70% of the overall experimental data was used for training, while 30% was used for testing. In this model, the standard Back-Propagation algorithm for the engine was used. It revealed that the ANN model can predict the engine output and exhaust emissions quite well with a regression coefficient lying closer to one, While the mean square error (MSE) was found to be very low.

Neural network model predicting forming limits for Bi-linear strain paths

The Forming Limit Curve (FLC) describes the maximum in-plane strains that can be applied to sheet metal before necking initiates. It is well known that the FLC depends on the yield locus, the hardening behavior, strain rate, temperature, superposed out-of-plane stresses and the shape of the loading path. There is an infinite number of possible paths for reaching a particular state of strain which makes the formulation of simple analytical models predicting the effect of non-linear loading paths on the FLC particularly challenging. Here, it is demonstrated that a neural network model is able to provide a computationally-efficient approach to describe the effect of non-proportional loading. A finite-element-based Marciniak-Kuczynski model is developed to characterize the forming limits of a DP780 steel through virtual experiments. Subsequently, a large data base is generated to serve as a basis for machine learning. Using a standard backpropagation algorithm with Bayesian regularization, a compact neural network model is trained and tested based on the results from virtual experiments. Even though the experimental data base includes only a finite number of random loading paths, the results show that the neural network model is able to predict the FLCs for a given proportional pre-straining history.

Research on BP model optimization of PSO algorithm based on Computational Mathematics

Construction of the quality and safety evaluation model for dairy products is significant to ensure food safety for people. In the case that there is a big error in the BP neural network quality and safety evaluation model for dairy products, this paper put forward the optimization method based on the PSO algorithm to reduce the output error of the BP neural network evaluation model through optimizing the weights and thresholds of BP neural network. By simulation of optimizing the BP evaluation model through specific data, experiments showed that compared with the standard BP evaluation model and standard BP evaluation, the optimized BP evaluation model is greatly improved in output error.

Modelling A Single-Rotor Wankel Engine Performance With Artificial Neural Network At Middle Speed Range

The researches on Wankel engines are very rare and considered new in modelling and prediction. Therefore this study deals with the artificial neural network (ANN) modelling of a Wankel engine to predict the power, volumetric efficiency and emissions, including nitrogen oxide, carbon dioxide, carbon monoxide and oxygen by using the change of mean effective pressure, intake manifold pressure, start of ignition angle and injection duration as inputs. The experiment results were taken from a research which is performed on a single-rotor, four stroke and port fuel injection 13B Wankel engine. The number of datas which are taken from experimental results were scarce and varied in six different data set (for example; mean effective pressure, from 1 to 6 bar) at 3000 rpm engine speed. The standard back-propagation (BPNN) Levenberg-Marquardt neural network algorithm is applied to evaluate the performance of middle speed range Wankel engine. The model performance was validated by comparing the prediction data sets with the measured experimental data. Results approved that the artificial neural network (ANN) model provided good agreement with the experimental data with good accuracy while the correlation coefficient R varies between 0.79 and 0.97.

Exclamativas e interrogativas com ‘ques’ em português brasileiro dialetal

Neste artigo, examinam-se exclamativas e interrogativas que contêm o determinante ‘ques’, em dialetos do português do Brasil. Diferentemente do português padrão, nestas estruturas, ‘que’ é o único constituinte do DP marcado com o morfema ‘-s’ de plural. Para explicar esses fatos, propõe-se uma análise não apenas da hierarquia do CP/DP na qual ‘ques’ se insere, mas também da distribuição do morfema de plural no DP. Para a derivação sintática, assume-se, seguindo propostas vigentes, que ‘ques’ é um determinante, e, portanto, um núcleo D. No que se refere a sentenças interrogativas, o DP ao qual ‘que’ pertence passa por movimento wh da posição de argumento interno do VP para o domínio do CP. No que se refere a sentenças exclamativas, assume-se que o DP se move da posição de predicado de uma small clause para o domínio do CP. Em ambos os casos, o movimento se dá por força ilocucionária. Para a distribuição do morfema de plural, assume-se que o cardinal divide o DP em dois domínios, sendo que sintagmas à sua esquerda são marcados com o morfema de plural, enquanto sintagmas à sua direita são não marcados. Como resultado, porque ‘ques’ é o único item mais alto que o cardinal nessas estruturas, ele é o único marcado com o morfema de plural. Portanto, desenvolve-se uma análise capaz de explicar: primeiramente, a estrutura oracional (CP) à qual o DP contendo ‘ques’ pertence; posteriormente, a hierarquia do DP; e finalmente a distribuição do morfema de plural na estrutura interna do DP.&#x0D; Palavras-chave: ‘Ques’. CP. Movimento wh. DP. NumP.

DATA: Differentiable ArchiTecture Approximation With Distribution Guided Sampling.

Neural architecture search (NAS) is inherently subject to the gap of architectures during searching and validating. To bridge this gap effectively, we develop Differentiable ArchiTecture Approximation (DATA) with Ensemble Gumbel-Softmax (EGS) estimator and Architecture Distribution Constraint (ADC) to automatically approximate architectures during searching and validating in a differentiable manner. Technically, the EGS estimator consists of a group of Gumbel-Softmax estimators, which is capable of converting probability vectors to binary codes and passing gradients reversely, reducing the estimation bias in a differentiable way. To narrow the distribution gap between sampled architectures and supernet, further, the ADC is introduced to reduce the variance of sampling during searching. Benefiting from such modeling, architecture probabilities and network weights in the NAS model can be jointly optimized with the standard back-propagation, yielding an end-to-end learning mechanism for searching deep neural architectures in an extended search space. Conclusively, in the validating process, a high-performance architecture that approaches to the learned one during searching is readily built. Extensive experiments on various tasks including image classification, few-shot learning, unsupervised clustering, semantic segmentation and language modeling strongly demonstrate that DATA is capable of discovering high-performance architectures while guaranteeing the required efficiency. Code is available at https://github.com/XinbangZhang/DATA-NAS.

A multiple multilayer perceptron neural network with an adaptive learning algorithm for thyroid disease diagnosis in the internet of medical things

Medical information systems such as Internet of Medical Things (IoMT) are gained special attention over recent years. X-ray and MRI images are important sources of information to be examined for a particular type of anomalies. Reports based on the images and laboratory examination results could be mined with machine learning techniques as well. Thyroid disease diagnosis is an important capability of medical information systems. The main objective of this study is to improve the diagnosis accuracy of thyroid diseases from semantic reports and examination results using artificial neural network (ANN) in IoMT systems. In order to improve generalization and avoid over-fitting of ANN during the training process, a set of multiple multilayer perceptron (MMLP) neural network with the back-propagation error ability is proposed in this paper. Moreover, an adaptive learning rate algorithm is used to deal with the slow convergence and the local minima problem of the back-propagation error algorithm. The proposed MMLP significantly increased the overall accuracy of thyroid disease classification. With MMLP with a set of 6 networks, an improvement of 0.7% accuracy is achieved compared to a single network. In addition, comparing to the standard back-propagation, by using an adaptive learning rate algorithm in the proposed MMLP, an improvement of 4.6% accuracy and the final accuracy of 99% have been obtained in IoMT systems. The proposed MMLP is compared to recent researches reported for thyroid disease diagnosis, and its superiority is shown.

Prognostics in Aeronautics with Deep Recurrent Neural Networks

Recurrent neural networks (RNNs) such as LSTM and GRU are not new to the field of prognostics. However, the performance of neural networks strongly depends on their architectural structure. In this work, we investigate a hybrid network architecture that is a combination of recurrent and feed-forward (conditional) layers. Two networks, one recurrent and another feed-forward, are chained together, with inference and weight gradients being learned using the standard back-propagation learning procedure. To better tune the network, instead of using raw sensor data, we do some preprocessing on the data, using mostly simple but effective statistics (researched in previous work). This helps the feature extraction phase and eases the problem of finding a suitable network configuration among the immense set of possible ones. This is not the first proposal of a hybrid network in prognostics but our work is novel in the sense that it performs a more comprehensive comparison of this type of architecture for different RNN layers and number of layers. Also, we compare our work with other classical machine learning methods. Evaluation is performed on two real-world case studies from the aero-engine industry: one involving a critical valve subsystem of the jet engine and another the whole reliability of the jet engine. Our goal here is to compare two cases contrasting micro (valve) and macro (whole engine) prognostics. Our results indicate that the performance of the LSTM and GRU deep networks are significantly better than that of other models.

Associations Of Fitness, Physical Activity, And Fatness With A New Index Of Endothelial Function

Current methods for assessing endothelial function, an early marker of cardiovascular disease (CVD), are either invasive or use noninvasive methods that are highly operator dependent or require nonreusable expensive probes, thereby limiting their use for CVD risk prediction. PURPOSE: To assess the relationship of fitness, physical activity (PA), and fatness, each a CVD risk factor, with a novel measure of endothelial function. METHODS: This was a cross-sectional data analysis in adults recruited for a study examining the clinical utility of a new device that measures endothelial vasodilator function using a standard BP cuff. The resulting EnDys score is derived from direct calibrated measurements of brachial arterial compliance throughout the entire transmural pressure curve during a staged cuff release after 5 minutes of upper arm occlusion. A higher score means better endothelial function. Fitness was assessed by a 6-minute walk test (6MWT). Self-report of PA was assessed by the Rapid Assessment of Physical Activity tool. Body mass index (BMI) was used as a marker of general fatness and waist circumference (WC) as a marker of abdominal fatness. RESULTS: In all 153 subjects, 51% were female, 7% were smokers, 7% had CVD, and 10% had type 2 diabetes. The mean ± SD for age was 49.3 ± 17.2, EnDys was 80.8 ± 30.4, BMI was 29.3 ± 6.9 kg/m2, WC was 37.5 ± 6.7 inches, and 6MWT was 495.4 ± 113.5 meters. EnDys was higher in females, 87.6 ± 30.5, vs males, 73.8 ± 28.9, p<0.01. EnDys did not differ by being sedentary, 80.8 ± 27.6, or active, 81.0 ± 33.0, p=0.97. Using bivariate analysis, a lower EnDys was associated with higher BMI, r=-0.23, higher WC, r=-0.33, lower 6MWT, r=0.32, and older age, r=-0.20, all p<0.02 or less. In a multivariate model, 6MWT (β=-0.06, p<0.01), WC (β=-1.02, p<.02), and sex (female, β=6.9, p<0.01) were each independently associated with EnDys. There were no interactions for sex with 6MWT and WC. CONCLUSION: Among measures of fitness, PA, and fatness, and in both sexes, a lower walking distance and a higher waist circumference were each independently associated with a lower EnDys, indicating worse endothelial function. A next logical step is to assess if EnDys improves with interventions like exercise and weight loss, thereby providing a novel and relatively simple way to track progress towards CVD risk reduction.

Industrial Time Series Data Forecasting of LSTM Neural Network Based on Attention Mechanism

At present, the maintenance mode of industrial equipment is still based on regular maintenance and after-the-fact maintenance, and with the development of industrial production intelligence, the production data of equipment has increased dramatically. In order to reasonably carry out maintenance activities in the use stage of CNC machine tools and other industrial equipment, this paper proposes an industrial time series data prediction method based on LSTM of attention mechanism. Firstly, based on the LSTM recurrent neural network, combined with the complex historical data of CNC machine tools, the overall characteristics of important time series are obtained. Secondly, the mechanism of attention analysis is introduced, and the algorithm of network structure design and prediction process is given. Finally, the accuracy of prediction is compared with experiments with standard BP neural network. The experimental results show that, compared with the standard BP neural network, the LSTM model based on the attention mechanism has higher prediction accuracy, which verifies the effectiveness of the method.

Kidney Disease, Intensive Hypertension Treatment, and Risk for Dementia and Mild Cognitive Impairment: The Systolic Blood Pressure Intervention Trial.

Intensively treating hypertension may benefit cardiovascular disease and cognitive function, but at the short-term expense of reduced kidney function. We investigated markers of kidney function and the effect of intensive hypertension treatment on incidence of dementia and mild cognitive impairment (MCI) in 9361 participants in the randomized Systolic Blood Pressure Intervention Trial, which compared intensive versus standard systolic BP lowering (targeting <120 mm Hg versus <140 mm Hg, respectively). We categorized participants according to baseline and longitudinal changes in eGFR and urinary albumin-to-creatinine ratio. Primary outcomes were occurrence of adjudicated probable dementia and MCI. Among 8563 participants who completed at least one cognitive assessment during follow-up (median 5.1 years), probable dementia occurred in 325 (3.8%) and MCI in 640 (7.6%) participants. In multivariable adjusted analyses, there was no significant association between baseline eGFR <60 ml/min per 1.73 m2 and risk for dementia or MCI. In time-varying analyses, eGFR decline ≥30% was associated with a higher risk for probable dementia. Incident eGFR <60 ml/min per 1.73 m2 was associated with a higher risk for MCI and a composite of dementia or MCI. Although these kidney events occurred more frequently in the intensive treatment group, there was no evidence that they modified or attenuated the effect of intensive treatment on dementia and MCI incidence. Baseline and incident urinary ACR ≥30 mg/g were not associated with probable dementia or MCI, nor did the urinary ACR modify the effect of intensive treatment on cognitive outcomes. Among hypertensive adults, declining kidney function measured by eGFR is associated with increased risk for probable dementia and MCI, independent of the intensity of hypertension treatment.

SOC Estimation of Mine Power Supply Based on Improved BP Neural Network Algorithm

Ni-MH, with high electric energy density and high stability, is the main backup battery of mine power supply. To improve the estimation accuracy of battery SOC (state of charge) and prolong its service life, this paper takes the single Ni-MH battery as the experimental object, and adopts an improved BP neural network that adds an adaptive rate momentum term to increase the discharged amount of electricity. The input characteristics of the network are the same as temperature, current and terminal voltage. According to the variance of the error between the expected output and the actual output of the training sample, the momentum term is adjusted; The adaptive learning rate is increased according to the change direction of the back propagation error. By comparing the standard BP neural network algorithm, the convergence speed is increased by 70%, and the generalization error is within 4%.

Deep neural networks based binary classification for single channel speaker independent multi-talker speech separation

Speech separation is an important task of separating a target speech from the mixture signals. Speaker-independent multi-talker speech separation is a challenging task due to unpredictability of the target and interfering speech in the target-interference mixtures. Conventionally, speech separation is used as a signal processing problem, but recently it is formulated as a deep learning problem and discriminative patterns of the speech are learned from the training data. In this paper, we consider the ideal binary mask (IBM) as a supervised binary classification training-target by using fully connected deep neural networks (DNN) for single-channel speaker-independent multi-talker speech separation. The train DNNs is used to estimate IBM training-target. The mean square error (MSE) is used as an objective cost function. Standard backpropagation and Monte-Carlo dropout regularization approaches are used for better generalization and overfitting during training. The estimated training-target is applied to the mixtures to obtain the separated target speech. We have addressed the over-smoothing problem and performed equalization of spectral variances to match the estimated and clean speech features. Our experimental results in various evaluating conditions report that the proposed method outperformed the competing methods in terms of the Perceptual Evaluation of Speech Quality (PESQ), Segmental SNR (SNRSeg), Short-time objective intelligibility (STOI), normalized Frequency weighted SNRSeg (nFwSNRSeg) and HIT-FA rates.

In patients with SBP ≥130 mm Hg and CV risk, intensive vs standard BP control did not increase fear of falling.

Source Citation Berlowitz DR, Foy C, Conroy M, et al. Impact of intensive blood pressure therapy on concern about falling: longitudinal results from the Systolic Blood Pressure Intervention Trial (...

Conjugate Gradient Polak Ribiere In Improving Performance in Predicting Population Backpropagation

Perform predictive data in the form of time series required a correct method, one method is now often used is the propagation of this method is a method that is able to minimize the error value of the output of the predicted number of people, but still generate quite a lot of iteration that needs to be optimized by minimize iterations and use of time, then the use of conjugate gradient polak Ribiere are expected to minimize the use of time, the number of the epoch of the results of standard backpropagation.

Estimating the Unit Weight of Local Organic Soils from Laboratory Tests Using Artificial Neural Networks

The estimation of the unit weight of soil is carried out using laboratory methods; however, it requires high-quality research material in the form of samples with undisturbed structures, the acquisition of which, especially in the case of organic soils, is extremely difficult, time-consuming and expensive. This paper presents a proposal to use artificial neural networks to estimate the unit weight of local organic soils as leading parameters in the process of checking the load capacity of subsoil, under a direct foundation in drained conditions, in accordance with current standards guidelines. The initial recognition of the subsoil, and the locating of organic soils at the Theological and Pastoral Institute in Rzeszow, was carried out using a mechanical cone penetration test (CPTM), using various interpretation criteria, and then, material for laboratory tests was obtained. The analysis of the usefulness of the artificial intelligence method, in this case, was based on data from laboratory tests. Standard multi-layer backpropagation networks were used to predict the soil unit weight based on two leading variables: the organic content LOIT and the natural water content w. The applied neural model provided reliable prediction results, comparable to the standard regression methods.