Reversal Of Gradient Research Articles

Proactive Privacy-preserving Learning for Cross-modal Retrieval

Deep cross-modal retrieval techniques have recently achieved remarkable performance, which also poses severe threats to data privacy potentially. Nowadays, enormous user-generated contents that convey personal information are released and shared on the Internet. One may abuse a retrieval system to pinpoint sensitive information of a particular Internet user, causing privacy leakage. In this article, we propose a data-centric Proactive Privacy-preserving Cross-modal Learning algorithm that fulfills the protection purpose by employing a generator to transform original data into adversarial data with quasi-imperceptible perturbations before releasing them. When the data source is infiltrated, the inside adversarial data can confuse retrieval models under the attacker’s control to make erroneous predictions. We consider the protection under a realistic and challenging setting where the prior knowledge of malicious models is agnostic. To handle this, a surrogate retrieval model is instead introduced, acting as the target to fool. The whole network is trained under a game-theoretical framework, where the generator and the retrieval model persistently evolve to fight against each other. To facilitate the optimization, a Gradient Reversal Layer module is inserted between two models, enabling a one-step learning fashion. Extensive experiments on widely used realistic datasets prove the effectiveness of the proposed method.

Enhancing the Generalization for Text Classification through Fusion of Backward Features.

Generalization has always been a keyword in deep learning. Pretrained models and domain adaptation technology have received widespread attention in solving the problem of generalization. They are all focused on finding features in data to improve the generalization ability and to prevent overfitting. Although they have achieved good results in various tasks, those models are unstable when classifying a sentence whose label is positive but still contains negative phrases. In this article, we analyzed the attention heat map of the benchmarks and found that previous models pay more attention to the phrase rather than to the semantic information of the whole sentence. Moreover, we proposed a method to scatter the attention away from opposite sentiment words to avoid a one-sided judgment. We designed a two-stream network and stacked the gradient reversal layer and feature projection layer within the auxiliary network. The gradient reversal layer can reverse the gradient of features in the training stage so that the parameters are optimized following the reversed gradient in the backpropagation stage. We utilized an auxiliary network to extract the backward features and then fed them into the main network to merge them with normal features extracted by the main network. We applied this method to the three baselines of TextCNN, BERT, and RoBERTa using sentiment analysis and sarcasm detection datasets. The results show that our method can improve the sentiment analysis datasets by 0.5% and the sarcasm detection datasets by 2.1%.

Mortality inequalities in France since the 1920s: Evidence of a reversal of the income gradient in mortality.

Many recent studies show that Europe has had a lower mortality inequality for most ages than the United States over the last thirty years. However, the evolution of the income gradient in mortality all along the twentieth century remains poorly understood. This article uses a unique dataset that gives the annual lifetables and fiscal income for the 90 administrative regions of mainland France from 1922 to 2020. The income gradients in mortality are computed across regions using a traditional method with calendar ages and, alternatively, with mortality milestones to control for the increase in life expectancy over time. The study reveals a systematic reversal of the gradient that occurred around the 1970s for both sexes and all ages or mortality groups when calculated at an aggregated level. Inequality in mortality amongst the oldest age groups has however returned to a level observed at least ten years earlier because of Covid-19, even after controlling for mortality improvements over the period.

Minimizing susceptibility-induced BOLD sensitivity loss in multi-band accelerated fMRI using point spread function mapping and gradient reversal

Objective. Interleaved reverse-gradient fMRI (RG-fMRI) with a point-spread-function (PSF) mapping-based distortion correction scheme has the potential to minimize signal loss in echo-planar-imaging (EPI). In this work, the RG-fMRI is further improved by imaging protocol optimization and application of reverse Fourier acquisition. Approach. Multi-band imaging was adapted for RG-fMRI to improve the temporal and spatial resolution. To better understand signal dropouts in forward and reverse EPIs, a simple theoretical relationship between echo shift and geometric distortion was derived and validated by the reliable measurements using PSF mapping method. After examining practical imaging protocols for RG-fMRI in three subjects on both a conventional whole-body and a high-performance compact 3 T, the results were compared and the feasibility to further improve the RG-fMRI scheme were explored. High-resolution breath-holding RG-fMRI was conducted with nine subjects on the compact 3 T and the fMRI reliability improvement in high susceptibility brain regions was demonstrated. Finally, reverse Fourier acquisition was applied to RG-fMRI, and its benefit was assessed by a simulation study based on the breath-holding RG-fMRI data. Main results. The temporal and spatial resolution of the multi-band RG-fMRI became feasible for whole-brain fMRI. Echo shift measurements from PSF mapping well estimated signal dropout effects in the EPI pair and were useful to further improve the RG-fMRI scheme. Breath-holding RG-fMRI demonstrated improved fMRI reliability in high susceptibility brain regions. Reverse partial Fourier acquisition omitting the late echoes could further improve the temporal or spatial resolution for RG-fMRI without noticeable signal degradation and spatial resolution loss. Significance. With the improved imaging scheme, RG-fMRI could reliably investigate the functional mechanisms of the human brain in the temporal and frontal areas suffering from susceptibility-induced functional sensitivity loss.

The Age–Stiffness Relationships of Elastic and Muscular Arteries in a Control Population and in End-Stage Renal Disease Patients

Arterial dysfunction is major risk factor for cardiovascular complications, and arterial stiffness is an independent risk factor in end-stage renal disease patients. As the distance from the heart increases, arterial stiffness (pulse wave velocity) becomes progressively more marked. This generates a centrifugal stiffness gradient, which leads to partial, continuous local wave reflections, which in turn attenuate the transmission of pulsatile pressure into the microcirculation, thus limiting the potentially deleterious outcomes both upstream (on the heart: left-ventricular hypertrophy and coronary perfusion) and downstream (on the renal and cerebral microcirculation: reduced glomerular filtration and impaired cognitive functions). The impact of arterial aging is greater on the aorta and central capacitive arteries, and it is characterized by a loss or reversal of the physiological stiffness gradient between central and peripheral arteries. Recently, however, in contrast to observations on the aorta, several studies have shown less pronounced, absent, or even negative associations between muscular peripheral arteries and age–stiffness relationships, which may be associated with a decrease in or reversal of the stiffness gradient. These findings point to a potential benefit of assessing the muscular peripheral arteries to predict the risk of cardiovascular disease and suggest that reversal of the stiffness gradient may be an independent risk factor for all-cause mortality.

Pressure gradient control in bidirectional switching between standby mode and production mode in biopharmaceutical cleanroom

The bidirectional switching between production mode and standby mode in biopharmaceutical clean workshop may lead to drastic fluctuation and reversal of pressure gradient, resulting in pollution risk, which limits the implementation of such switching operation in actual production. Therefore, a data relationship model is proposed between the supply air volume (valve opening) and the return air volume (valve opening) of the cleanroom, and the total air supply volume and the air supply volume of each cleanroom in the clean pharmaceutical plant are adjusted in a synchronized linear step-by-step manner, with a stepwise Bidirectional switching between production and standby modes. Experimental results show that the new mode switching strategy, when the production mode is switched to the standby mode (reducing the air volume), the step size of 20% can better maintain the stability of pressure gradient. When the standby mode is switched to the production mode (increasing the air volume), the step size of 10% can better maintain the stability of pressure gradient. During the whole bidirectional switching process, the pressure variation of each room can be suppressed within the range of 20%. It lays a foundation for the energy saving operation of biopharmaceutical cleanroom during the non-production period.

A vascularized crypt-patterned colon model for high-throughput drug screening and disease modelling.

The colon serves as a primary target for pharmaceutical compound screening and disease modelling. To better study colon diseases and develop treatments, engineered in vitro models with colon-specific physiological features are required. Existing colon models lack integration of colonic crypt structures with underlying perfusable vasculature, where vascular-epithelial crosstalk is affected by disease progression. We present a colon epithelium barrier model with vascularized crypts that recapitulates relevant cytokine gradients in both healthy and inflammatory conditions. Using our previously published IFlowPlate384 platform, we initially imprinted crypt topography and populated the patterned scaffold with colon cells. Proliferative colon cells spontaneously localized to the crypt niche and differentiated into epithelial barriers with a tight brush border. Toxicity of the colon cancer drug, capecitabine, was tested and showed a dose-dependent response and recovery from crypt-patterned colon epithelium exclusively. Perfusable microvasculature was then incorporated around the colon crypts followed by treatment with pro-inflammatory TNFα and IFNγ cytokines to simulate inflammatory bowel disease (IBD)-like conditions. We observed in vivo-like stromal basal-to-apical cytokine gradients in tissues with vascularized crypts and gradient reversals upon inflammation. Taken together, we demonstrated crypt topography integrated with underlying perfusable microvasculature has significant value for emulating colon physiology and in advanced disease modelling.

$L_{1}$-Regularized Reconstruction Model for Edge-Preserving Filtering

Smoothing images while preserving salient edges is a crucial task in computational photography. Existing edge-preserving filters suffer from various artifacts, such as halos, gradient reversals, and intensity shifts. Observing that various artifacts are strongly related to salient edges with large gradients, we propose a continuous mapping function to process the gradients. The proposed function is literally edge-preserving, i.e., it keeps large gradients intact while attenuating small gradients. We propose an L₁-regularized reconstruction model based on the processed gradients for edge-preserving image filtering. The L₁-regularization facilitates the edge-preserving property in the reconstructed results. To solve the proposed L₁-regularized model, we implement an efficient algorithm based on the alternating direction method of multipliers (ADMM) and Fourier domain optimization. We have conducted qualitative and quantitative experiments to evaluate the proposed filter. The results demonstrate that our filter better handles various artifacts and delivers superior image quality on various applications. The proposed filter is highly efficient, our GPU implementation takes 70ms to process a color image with 1 megapixel on an NVIDIA GTX 1070 GPU.

Determining the height of deep volcanic eruptions over the tropical western Pacific with Himawari-8

Volcanic eruptions are significant aviation hazards due to the formation of airborne volcanic ash clouds. Further, deep eruptions that reach the upper troposphere and lower stratosphere may have significant weather and climate impacts. A key variable for both dispersion model forecasting for aviation hazards and understanding climate impacts is the volcanic plume height. This work presents a method to quickly and reliably estimate the maximum plume heights of volcanic eruptions that interact with the tropical tropopause layer in the tropical western Pacific region. The method uses infrared (11.2 μm) data from Himawari-8 to identify ‘stratospheric warm spots’ in optically thick portions of the eruption cloud top by searching for reversals in the local-brightness temperature gradient. The brightness temperature of these warm spots is converted to height using seasonal stratospheric reference temperature profiles derived from 20 years of radiosonde data from 17 stations spread throughout the western Pacific. An approach for estimating the height of cold ‘overshooting tops’ is also adopted. Based on the radiosonde data, estimates of the uncertainty in the plume height depend on the height and range within 0.5–5.0 km. A case study of the 19 December 2021 eruption of Hunga Tonga-Hunga Ha’apai demonstrates the technique. The heights are robustly determined with this simple technique and compare well with height estimates of eruptions in the literature that use more complex satellite techniques.

Aligning Small Datasets Using Domain Adversarial Learning: Applications in Automated in Vivo Oral Cancer Diagnosis.

Deep learning approaches for medical image analysis are limited by small data set size due to factors such as patient privacy and difficulties in obtaining expert labelling for each image. In medical imaging system development pipelines, phases for system development and classification algorithms often overlap with data collection, creating small disjoint data sets collected at numerous locations with differing protocols. In this setting, merging data from different data collection centers increases the amount of training data. However, a direct combination of datasets will likely fail due to domain shifts between imaging centers. In contrast to previous approaches that focus on a single data set, we add a domain adaptation module to a neural network and train using multiple data sets. Our approach encourages domain invariance between two multispectral autofluorescence imaging (maFLIM) data sets of in vivo oral lesions collected with an imaging system currently in development. The two data sets have differences in the sub-populations imaged and in the calibration procedures used during data collection. We mitigate these differences using a gradient reversal layer and domain classifier. Our final model trained with two data sets substantially increases performance, including a significant increase in specificity. We also achieve a significant increase in average performance over the best baseline model train with two domains (p = 0.0341). Our approach lays the foundation for faster development of computer-aided diagnostic systems and presents a feasible approach for creating a robust classifier that aligns images from multiple data centers in the presence of domain shifts.

Fast Guided Median Filter.

Faster computation of a weighted median (WM) filter is impeded by the construction of a weighted histogram for every local window of data. Since the calculated weights vary for each local window, it is difficult, using a sliding window approach, to construct the weighted histogram efficiently. In this paper, we propose a novel WM filter that overcomes the difficulty of histogram construction. Our proposed method achieves real-time processing for higher resolution images and can be applied to multidimensional, multichannel, and high precision data. The weight kernel used in our WM filter is the pointwise guided filter, which is derived from the guided filter. The use of kernels based on the guided filter avoids gradient reversal artifacts and shows a higher denoising performance than the Gaussian kernel based on the color/intensity distance. The core idea of the proposed method is a formulation that allows the use of histogram updates with a sliding window approach to find the weighted median. For high precision data we propose an algorithm based on a linked list that can reduce the memory requirements of storing histograms and the computational cost of updating them. We present implementations of the proposed method that are suitable for both CPU and GPU. Experimental results show that the proposed method indeed realizes faster computation than conventional WM filters and is capable of filtering multidimensional, multichannel, and high precision data. This is an approach which is difficult to achieve with conventional methods.

An unsupervised intelligent fault diagnosis research for rotating machinery based on NND-SAM method

Currently, intelligent fault diagnostics of rotating machinery have significantly contributed to mechanical health monitoring. However, real-world labeled data obtained from high-value equipment such as gas turbine units, pumps, and other rotating components are occasionally insufficient for model training. This article proposes an unsupervised deep transfer learning model that can directly extract features from the data itself, thus reducing the number of training samples required. The well-designed neural network with a domain-specific antagonism mechanism aligns features between the source and target domains and so makes data-driven decisions more efficiently. The parameter-free gradient reversal layer is used as an optimizer, considerably reducing the cross-domain discrepancy and accelerating convergence. The average multi-classification accuracy under transferable conditions reaches 97%, 91%, and 95% over three cases of fault diagnosis. Moreover, the time consumption of the system improves by more than 3.5% compared to existing models. The results reveal that the suggested strategy is suitable for a challenging unlabeled dataset and represents a significant improvement over existing unsupervised learning techniques.

The 'Reverse FDUF' Mechanism of Atrial Excitation-Contraction Coupling Sustains Calcium Alternans-A Hypothesis.

Cardiac calcium alternans is defined as beat-to-beat alternations of Ca transient (CaT) amplitude and has been linked to cardiac arrhythmia, including atrial fibrillation. We investigated the mechanism of atrial alternans in isolated rabbit atrial myocytes using high-resolution line scan confocal Ca imaging. Alternans was induced by increasing the pacing frequency until stable alternans was observed (1.6-2.5 Hz at room temperature). In atrial myocytes, action potential-induced Ca release is initiated in the cell periphery and subsequently propagates towards the cell center by Ca-induced Ca release (CICR) in a Ca wave-like fashion, driven by the newly identified 'fire-diffuse-uptake-fire' (FDUF) mechanism. The development of CaT alternans was accompanied by characteristic changes of the spatio-temporal organization of the CaT. During the later phase of the CaT, central [Ca]i exceeded peripheral [Ca]i that was indicative of a reversal of the subcellular [Ca]i gradient from centripetal to centrifugal. This gradient reversal resulted in a reversal of CICR propagation, causing a secondary Ca release during the large-amplitude alternans CaT, thereby prolonging the CaT, enhancing Ca-release refractoriness and reducing Ca release on the subsequent beat, thus enhancing the degree of CaT alternans. Here, we propose the 'reverse FDUF' mechanism as a novel cellular mechanism of atrial CaT alternans, which explains how the uncoupling of central from peripheral Ca release leads to the reversal of propagating CICR and to alternans.

Meta-Learning for Mandarin-Tibetan Cross-Lingual Speech Synthesis

The paper proposes a meta-learning-based Mandarin-Tibetan cross-lingual text-to-speech (TTS) to realize both Mandarin and Tibetan speech synthesis under a unique framework. First, we build two kinds of Tacotron2-based Mandarin-Tibetan cross-lingual baseline TTS. One is a shared encoder Mandarin-Tibetan cross-lingual TTS, and another is a separate encoder Mandarin-Tibetan cross-lingual TTS. Both baseline TTS use the speaker classifier with a gradient reversal layer to disentangle speaker-specific information from the text encoder. At the same time, we design a prosody generator to extract prosodic information from sentences to explore syntactic and semantic information adequately. To further improve the synthesized speech quality of the Tacotron2-based Mandarin-Tibetan cross-lingual TTS, we propose a meta-learning-based Mandarin-Tibetan cross-lingual TTS. Based on the separate encoder Mandarin-Tibetan cross-lingual TTS, we use an additional dynamic network to predict the parameters of the language-dependent text encoder that could realize better cross-lingual knowledge sharing in the sequence-to-sequence TTS. Lastly, we synthesize Mandarin or Tibetan speech through the unique acoustic model. The baseline experimental results show that the separate encoder Mandarin-Tibetan cross-lingual TTS could handle the input of different languages better than the shared encoder Mandarin-Tibetan cross-lingual TTS. The experimental results further show that the proposed meta-learning-based Mandarin-Tibetan cross-lingual speech synthesis method could effectively improve the voice quality of synthesized speech in terms of naturalness and speaker similarity.

Direct electrochemical detection of trans-plasma membrane electron transfer: A possible alternative pathway for cell respiration

An electrochemical protocol was designed to enable Vero cells to transfer electrons to an electrode without any added redox mediator. The cells were cultured on the surface of electrodes polarized at the optimal potential of 400 mV/silver pseudo-reference. Gold, carbon, and CNT-coated carbon electrodes displayed similar current record patterns. Extracellular electron transfer was sustained for several days. Its intensity, up to 1.5 pA.cell−1, was in the range of the electron flows implemented by cell respiration. A large fraction of the current vanished as soon as anoxic conditions were established, which suggests a mitochondrial origin for a large proportion of the electrons. The current records always showed a two-phase pattern. The occurrence of the two phases was not due to an evolution of the cell mat structure, which was fully established during the first day of polarization and did not change significantly thereafter. Increasing the cell seeding density decreased the maximum current reached during the first phase and the duration of the phase. These observations put together lead us to propose a model, in which only the cells adhered on the electrode surface produced current by metabolizing glutamine during the first phase. The possible role of this extracellular electron transfer as an alternative cell respiration pathway is discussed. The key roles it could play in regulating pH and pO2 gradients are considered, specifically to explain the pH gradient reversal observed in cancer cells. These pioneering results pave the way for electrochemical sensors to directly address cellular metabolic pathways.

Understanding the zigzags of multi-echo phase signals by numerical simulations

Understanding the multi-echo phase zigzag signals is instrumental to assuring the quality of MRI phase data acquisition for ensuing phase exploration and exploitation. This paper provides a theoretical and computational mechanism for understanding the zigzag multi-echo phase formation that has been observed in numerical multi-echo gradient-recalled (GRE) simulations of clinical complex-valued brain MRI images. Based on intravoxel dephasing mechanism, we calculated a train of multi-GRE complex-valued voxel signals by simulating field gradient reversals under perturbations in either gradient strength (G±δG) or gradient duration (Δ±δΔ), as well as the simultaneous bi-variable gradient perturbations (δGδΔ). In this theoretical experiment, we observed a zigzag line of one-shot multi-echo phase signals at a voxel with respect to linear stepwise field gradient variations in δG ∝ n and δΔ ∝ n (where n denotes the echo index). However, the multi-echo magnitude signals were invariant to field gradient reversal, i.e. no multi-echo magnitude zigzags. To support our simulations, we analyzed the clinical one-shot multi-echo T2*-weighted MRI phase images and found similar multi-echo phase zigzags. In this way, we provide a theoretical and computational understanding of multi-echo phase zigzag artifacts, specifically for the eddy current effect on one-shot multi-GRE signals in practice.

L0 image smoothing via iterating truncated L1 gradient regularization

Edge-preserving image smoothing is a fundamental tool in computational photography and graphics. It aims to suppress insignificant details while maintaining salient structures. The classical L0 filter provides an elegant framework for a variety of applications. However, it inclines to sharpen the salient edges, thus suffering from gradient reversals and color deviations. We propose a solution toward the objective of optimizing summed squared error regularized by the L0-norm of the gradients. The proposed solution explores an iterative strategy, where each iteration is an optimization problem based on the truncated L1 regularization, which can be solved efficiently with the combination of alternating direction method of multipliers and Fourier domain optimization. Leveraging L1-norm limits the aggressive modifications of gradients during the iterative process, which alleviates various artifacts in classical L0 filter. Experiment results indicate that the proposed method achieves superior performance in various applications, including texture removal, detail enhancement, HDR tone mapping, and compression artifact removal. The proposed filter can be conveniently implemented on GPU. It takes our filter 0.51 s to process a 720P color image on a NVIDIA GTX 1080 GPU.

Framework to select refining parameters in Total fringe order photoelasticity (TFP)

What makes Total fringe order photoelasticity (TFP) the ideal digital photoelastic technique for industrial applications is the sufficiency of a single colour isochromatic image for fringe order evaluation and the convenience of capturing this image using easily accessible, relatively cheap hardware. In TFP, the colour information in the application image is matched with an experimental colour calibration table. A simple least squares colour matching is insufficient for this due to errors caused by the repetition of colour in the isochromatics. This is rectified through a process called fringe order refinement. Among the existing fringe order refining methods, the combination of – flexible seeding, window search method, and FRSTFP scanning – has proved to be the best choice for solving a variety of problems for the complete model domain. The successful application of this combination requires the right choice of parameter values for its constituent methods. A systematic parametric study of these refining parameters is a lacuna in the literature, which is addressed in this work, by selecting a variety of problems. The study has revealed the interconnection between the refining parameters, which has helped to formulate a set of recommendations for selecting the same. It is found that some of the existing understandings on refining parameters require a relook. The difficulty in refining fringe fields having multiple fringe gradient reversals with a single set of parameters is illustrated, and how this can be overcome by local refinement is brought out. Additionally, a methodology to track the source of refining error is proposed.

Integration of deep adaptation transfer learning and online sequential extreme learning machine for cross-person and cross-position activity recognition

Deep learning (DL) has been evolving to a prevalent method in human activity recognition (HAR). However, the performance of wearable sensor based HAR models decline significantly when training data come from different persons or sensor positions, and a time-consuming data annotation is indispensible to cater for the big-data driven DL models. In this paper we proposed a fast and robust hybrid model to handle the transfer issues of wearable sensor based HAR between different persons (cross-person) and different positions (cross-position) with just a few annotated data in target domain. The model consists of three parts: (1) A convolutional neural network (CNN) with global average pooling layer to facilitate the extraction of advanced common features in source domain and target domain; (2) A domain adaptive neural network with a gradient reversal layer (DANN) and deep domain confusion network with an adaptive layer (DDC) to reduce domain shift caused by the change of persons and sensor positions; (3) An adaptive classifier based on online sequential extreme learning machine (OS-ELM) to achieve fast and accurate classification with a few annotated data in target domain. Experimental results on four public datasets verified the superiority of the proposed hybrid model over standard CNN and deep transfer learning models in adapting the classifier to new sensor locations and subjects quickly, where the HAR accuracy can be improved by at least 12% for cross-person transfer and 20% for cross-position transfer, respectively.

Domain adaptation for staff-region retrieval of music score images

Optical music recognition (OMR) is the field that studies how to automatically read music notation from score images. One of the relevant steps within the OMR workflow is the staff-region retrieval. This process is a key step because any undetected staff will not be processed by the subsequent steps. This task has previously been addressed as a supervised learning problem in the literature; however, ground-truth data are not always available, so each new manuscript requires a preliminary manual annotation. This situation is one of the main bottlenecks in OMR, because of the countless number of existing manuscripts , and the associated manual labeling cost. With the aim of mitigating this issue, we propose the application of a domain adaptation technique, the so-called Domain-Adversarial Neural Network (DANN), based on a combination of a gradient reversal layer and a domain classifier in the inference neural architecture. The results from our experiments support the benefits of our proposed solution, obtaining improvements of approximately 29% in the F-score.