Bidirectional Mapping Research Articles

Mapping of Dynamics Between Mechanical and Electrical Ports in SG-IBR Composite Grids

Power grids are traditionally dominated by synchronous generators (SGs) but are currently undergoing a major transformation due to the increasing integration of inverter-based resources (IBRs). The state space method with transparent apparatus models can be readily used. However, models of IBRs are usually not disclosed by manufacturers. Alternatively, the port-based approach represents dynamics by input-output transfer functions without exposing internal states. These transfer functions at various ports are normally configured with a particular focus: an SG-dominated grid is traditionally analyzed in a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mechanical-centric</i> view which ignores fast electrical dynamics and focuses on the torque-speed dynamics, whereas the emergent IBR-dominated grid usually takes an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">electrical-centric</i> view which focuses on the voltage-current interaction. In this article, a new perspective called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">port-mapping method</i> is proposed to combine these two views. Specifically, the mechanical dynamics are mapped to the electrical impedance seen at the electrical port; and the electrical dynamics are also mapped to the torque coefficient seen at the mechanical port. The bidirectional mapping gives additional flexibility and insights to analyze the sub-system interactions in whole-system dynamics and guide the tuning of parameters. Application of the proposed method is illustrated in three cases with increasing scales.

Model-Based Image Signal Processors via Learnable Dictionaries

Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP). Computational photography tasks such as image denoising and colour constancy are commonly performed in the RAW domain, in part due to the inherent hardware design, but also due to the appealing simplicity of noise statistics that result from the direct sensor readings. Despite this, the availability of RAW images is limited in comparison with the abundance and diversity of available RGB data. Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping: handcrafted model-based methods that are interpretable and controllable usually require manual parameter fine-tuning, while end-to-end learnable neural networks require large amounts of training data, at times with complex training procedures, and generally lack interpretability and parametric control. Towards addressing these existing limitations, we present a novel hybrid model-based and data-driven ISP that builds on canonical ISP operations and is both learnable and interpretable. Our proposed invertible model, capable of bidirectional mapping between RAW and RGB domains, employs end-to-end learning of rich parameter representations, i.e. dictionaries, that are free from direct parametric supervision and additionally enable simple and plausible data augmentation. We evidence the value of our data generation process by extensive experiments under both RAW image reconstruction and RAW image denoising tasks, obtaining state-of-the-art performance in both. Additionally, we show that our ISP can learn meaningful mappings from few data samples, and that denoising models trained with our dictionary-based data augmentation are competitive despite having only few or zero ground-truth labels.

Visual-semantic graph neural network with pose-position attentive learning for group activity recognition

Video-based group activities typically contain interactive contexts among diverse visual modalities between multiple persons, and semantic relationships between individual actions. Nevertheless, majority of the existing methods for recognizing group activity either captures the relationships among different persons by utilizing a solely RGB modality or neglect to exploit the label hierarchies between individual actions and the group activity. To tackle these issues, we propose a visual-semantic graph neural network, with pose-position attentive learning (VSGNN-PAL), for group activity recognition. Specifically, we first extract the individual-level appearance and motion representations from RGB and optical-flow inputs, to build a bi-modal visual graph. Two attentive aggregators are further proposed to integrate both the pose and position information to measure the relevance scores between persons, and dynamically refine the representation of each visual node from both modality-specific and cross-modal perspectives. To model a semantic hierarchy from a label space, we construct a semantic graph based on the linguistic embeddings of individual actions and group activity labels. We further employ a bi-directional mapping learning scheme, to integrate the label-relation-aware semantic context into the visual representations. Besides, a global reasoning module is introduced to progressively generate the group-level representations with the scene description maintained. Furthermore, we formulate a semantic-preserving loss, to maintain the consistency between the learned high-level representations and the semantics of the ground-truth labels. Experimental results on three group activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance.

Data augmentation by a CycleGAN-based extra-supervised model for nondestructive testing

The deep learning method is widely used in computer vision tasks with large-scale annotated datasets. However, obtaining such datasets in most directions of the vision based nondestructive testing (NDT) field is very challenging. Data augmentation is proved as an efficient way of dealing with the lack of large-scale annotated datasets. In this paper, we propose a CycleGAN-based extra-supervised (CycleGAN-ES) model to generate synthetic NDT images, where the ES is used to ensure that the bidirectional mapping is learned for corresponding labels and defects. Furthermore, we show the effectiveness of using the synthesized images to train deep convolutional neural networks (DCNNs) for defect recognition. In the experiments, we extract a number of x-ray welding images with both defect and no defects from the published GDXray dataset, and CycleGAN-ES is used to generate the synthetic defect images based on a small number of extracted defect images and manually drawn labels that are used as a content guide. For quality verification of the synthesized defect images, we use a high-performance classifier pretrained using a big dataset to recognize the synthetic defects and show the comparability of the performances of classifiers trained using synthetic defects and real defects, respectively. To present the effectiveness of using the synthesized defects as an augmentation method, we train and evaluate the performances of DCNN for defect recognition with or without the synthesized defects.

Artificial neural Network-Based approaches for Bi-directional modelling of robotic wire arc additive manufacturing

Additive manufacturing (AM) process has seen its usage go astronomically up in the recent years. One of the oldest additive manufacturing processes namely, wire arc additive manufacturing (WAAM) has seen many recent developments due to the urge of attaining high efficiency in manufacturing field. Not only is it a very simple process among the other AM processes, but also it can be used for manufacturing very complex structures with a very low budget. The process of robotic WAAM demands precise modelling of the relationship between the input and response parameters so as to guarantee desired output of fine quality. To meet the demands of the process, hybridized artificial neural network (ANN) models were developed in this study and used for forward and backward mappings. Gradient descent with momentum and grey wolf optimization (GWO) are the algorithms that have been coupled with the neural networks in this paper. The novelty of this paper lies with the performance comparison in bi-directional mapping by hybridizing ANN models with the aforementioned algorithms in robotic WAAM domain. The gradient descent with momentum algorithm has shown better results than the metaheuristic algorithm in both the mappings while taking much lesser execution time.

Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning.

Bidirectional mapping-based generalized zero-shot learning (GZSL) methods rely on the quality of synthesized features to recognize seen and unseen data. Therefore, learning a joint distribution of seen-unseen classes and preserving the distinction between seen-unseen classes is crucial for GZSL methods. However, existing methods only learn the underlying distribution of seen data, although unseen class semantics are available in the GZSL problem setting. Most methods neglect retaining seen-unseen classes distinction and use the learned distribution to recognize seen and unseen data. Consequently, they do not perform well. In this work, we utilize the available unseen class semantics alongside seen class semantics and learn joint distribution through a strong visual-semantic coupling. We propose a bidirectional mapping coupled generative adversarial network (BMCoGAN) by extending the concept of the coupled generative adversarial network into a bidirectional mapping model. We further integrate a Wasserstein generative adversarial optimization to supervise the joint distribution learning. We design a loss optimization for retaining distinctive information of seen-unseen classes in the synthesized features and reducing bias towards seen classes, which pushes synthesized seen features towards real seen features and pulls synthesized unseen features away from real seen features. We evaluate BMCoGAN on benchmark datasets and demonstrate its superior performance against contemporary methods.

Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis.

Fusing multi-modality medical images, such as magnetic resonance (MR) imaging and positron emission tomography (PET), can provide various anatomical and functional information about the human body. However, PET data is not always available for several reasons, such as high cost, radiation hazard, and other limitations. This paper proposes a 3D end-to-end synthesis network called Bidirectional Mapping Generative Adversarial Networks (BMGAN). Image contexts and latent vectors are effectively used for brain MR-to-PET synthesis. Specifically, a bidirectional mapping mechanism is designed to embed the semantic information of PET images into the high-dimensional latent space. Moreover, the 3D Dense-UNet generator architecture and the hybrid loss functions are further constructed to improve the visual quality of cross-modality synthetic images. The most appealing part is that the proposed method can synthesize perceptually realistic PET images while preserving the diverse brain structures of different subjects. Experimental results demonstrate that the performance of the proposed method outperforms other competitive methods in terms of quantitative measures, qualitative displays, and evaluation metrics for classification.

Operational Effectiveness Evaluation of EW Equipment System Based on Framework Design

The currently used evaluating method has such problems as complex evaluation index space, complexity of computation, low evaluation efficiency, and low systematicness and intellectualization, which is difficult to meet the requirements of operational effectiveness evaluation for ECM equipment. In order to guarantee the efficiency and accuracy of operational effectiveness evaluation, an integrated and systematic evaluation method was provided based on multi-views system’s simulation model architecture, which can show the actual frameworks of equipment system for evaluation, we describe the element, structure, operational process and index system in a standardized manner. Furthermore, a framework of Flexible-intelligent Evaluation towards Combat-capability Space in System-of-systems (FECSS) was proposed. The FECSS method builds a bi-directional mapping relation between the upper and low operational effectiveness indicators by basic indicators-formed space in system-of-systems which combined the complex evaluation index, evaluation process with intelligent evaluation model. The proposed method can effectively analysis the effectiveness results in condition of different types and numbers of equipment. It also makes the electronic countermeasure equipment project demonstration and operational application more feasibly and practically.

Digital twin-driven clamping force control for thin-walled parts

Clamping quality is one of the main factors that will affect the deformation of thin-walled parts during their processing, which can then directly affect parts’ performance. However, traditional clamping force settings are based on manual experience, which is a random and inaccurate manner. In addition, dynamic clamping force adjustment according to clamping deformation is rarely considered in clamping force control process, which easily causes large clamping deformation and low machining accuracy. To address these issues, this study proposes a digital twin-driven clamping force control approach to improve the machining accuracy of thin-walled parts. The total factor information model of clamping system is built to integrate the dynamic information of the clamping process. The virtual space model is constructed based on finite element simulation and deep neural network algorithm. To ensure bidirectional mapping of physical-virtual space, the workflow of clamping force control and interoperability method between digital twin models are elaborated. Finally, a case study is used to verify the effectiveness and feasibility of the proposed method.

Discrete hashing with triple supervision learning

In recent years, discrete supervised hashing methods have attracted increasing attention because of their high retrieval efficiency and precision. However, in these methods, some effective semantic information is typically neglected, which means that all the information is not sufficiently utilized. Moreover, these methods often only decompose the first-order features of the original data, ignoring the more fine-grained higher-order features. To address these problems, we propose a supervised hashing learning method called discrete hashing with triple supervision learning (DHTSL). Specifically, we integrate three aspects of semantic information into this method: (1) the bidirectional mapping of semantic labels; (2) pairwise similarity relations; (3) second-order features from the original data. We also design a discrete optimization method to solve the proposed objective function. Moreover, an out-of-sample extension strategy that can better maintain the independence and balance of hash codes is employed to improve retrieval performance. Extensive experiments on three widely used datasets demonstrate its superior performance.

Bi-directional mapping for multi-label learning of label-specific features

Bi-directional mapping for multi-label learning of label-specific features

Bidirectional Mapping Between the Symbolic Number System and the Approximate Number System.

Previous studies have discussed the symmetry of bidirectional mapping between approximate number system (ANS) and symbolic number system (SNS). However, these studies neglected the essential significance of bidirectional mapping in the development of numerical cognition. That is, with age, the connection strength between the ANS and SNS in ANS-SNS mapping could be higher than that in SNS-ANS mapping. Therefore, this study attempted to explore the symmetry of bidirectional mapping by examining whether the connection between the ANS and SNS is the same. Using two types of dot array materials (extensive and intensive) and sequence priming paradigms, this study found a stable negative priming effect in the ANS-SNS priming task, but no priming effect in the SNS-ANS priming task. In addition, although sensory cues (extensive and intensive) could affect performance in the ANS-SNS mapping task, these cues did not affect performance in the ANS-SNS priming task. In general, this study provides valuable insight into the symmetry of bidirectional mapping.

Label Disentangled Analysis for unsupervised visual domain adaptation

Labeling a large number of samples in machine learning is often a prohibitively time-consuming task. Unsupervised domain adaptation (UDA) aims to learn a model to classify or represent the target domain by borrowing sufficiently labeled data from semantic adjacent but differently distributed source domains. However, existing DA models ignore an essential problem, i.e. the label dominated supervision signal easily makes the learning model/network tend to memorize the data, but not to learn essential image features, which makes DA unexplainable on what makes transfer a success. With this motivation, we have a core idea to disentangle/erase the label information from image features for easier DA task. To that end, this paper for the first time proposes the “label-erased” and “label-reconstructed” feature distributions by establishing their bidirectional mapping, and models them respectively for domain adaptation with the philosophy of divide and conquer. Specifically, in this paper, we propose a Label Disentangled Analysis (LDA) approach, consisting of three novel parts: (1) disentangling (decoupling) the labels for label-erased (label-irrelevant) features from domain data, (2) aligning the label-erased image features and label-reconstructed features between domains, and (3) cross-associating the aligned image and label features together to ensure the class discrimination of data. The above process can be summarized as three actions: label disentanglement (decoupling), feature-label joint adaptation (alignment), and feature-label cross association (cross-coupling). Extensive experiments verify that the proposed LDA outperforms the SOTAs on a number of benchmarks and is also comparable to several advanced deep transfer methods.

Deep cross-view co-regularized representation learning for glioma subtype identification

The new subtypes of diffuse gliomas are recognized by the World Health Organization (WHO) on the basis of genotypes, e.g., isocitrate dehydrogenase and chromosome arms 1p/19q, in addition to the histologic phenotype. Glioma subtype identification can provide valid guidances for both risk-benefit assessment and clinical decision. The feature representations of gliomas in magnetic resonance imaging (MRI) have been prevalent for revealing underlying subtype status. However, since gliomas are highly heterogeneous tumors with quite variable imaging phenotypes, learning discriminative feature representations in MRI for gliomas remains challenging. In this paper, we propose a deep cross-view co-regularized representation learning framework for glioma subtype identification, in which view representation learning and multiple constraints are integrated into a unified paradigm. Specifically, we first learn latent view-specific representations based on cross-view images generated from MRI via a bi-directional mapping connecting original imaging space and latent space, and view-correlated regularizer and output-consistent regularizer in the latent space are employed to explore view correlation and derive view consistency, respectively. We further learn view-sharable representations which can explore complementary information of multiple views by projecting the view-specific representations into a holistically shared space and enhancing via adversary learning strategy. Finally, the view-specific and view-sharable representations are incorporated for identifying glioma subtype. Experimental results on multi-site datasets demonstrate the proposed method outperforms several state-of-the-art methods in detection of glioma subtype status.

Digital twin-driven framework for improving self-management of ergonomic risks

PurposeThe physically-demanding and repetitive nature of construction work often exposes workers to work-related musculoskeletal injuries. Real-time information about the ergonomic consequences of workers' postures can enhance their ability to control or self-manage their exposures. This study proposes a digital twin framework to improve self-management ergonomic exposures through bi-directional mapping between workers' postures and their corresponding virtual replica.Design/methodology/approachThe viability of the proposed approach was demonstrated by implementing the digital twin framework on a simulated floor-framing task. The proposed framework uses wearable sensors to track the kinematics of workers' body segments and communicates the ergonomic risks via an augmented virtual replica within the worker's field of view. Sequence-to-sequence long short-term memory (LSTM) network is employed to adapt the virtual feedback to workers' performance.FindingsResults show promise for reducing ergonomic risks of the construction workforce through improved awareness. The experimental study demonstrates feasibility of the proposed approach for reducing overexertion of the trunk. Performance of the LSTM network improved when trained with augmented data but at a high computational cost.Research limitations/implicationsSuggested actionable feedback is currently based on actual work postures. The study is experimental and will need to be scaled up prior to field deployment.Originality/valueThis study reveals the potentials of digital twins for personalized posture training and sets precedence for further investigations into opportunities offered by digital twins for improving health and wellbeing of the construction workforce.

Normalization of breast MRIs using cycle-consistent generative adversarial networks

Objectives: Dynamic Contrast Enhanced-Magnetic Resonance Imaging (DCE-MRI) is widely used to complement ultrasound examinations and x-ray mammography for early detection and diagnosis of breast cancer. However, images generated by various MRI scanners (e.g., GE Healthcare, and Siemens) differ both in intensity and noise distribution, preventing algorithms trained on MRIs from one scanner to generalize to data from other scanners. In this work, we propose a method to solve this problem by normalizing images between various scanners.Methods: MRI normalization is challenging because it requires normalizing intensity values and mapping noise distributions between scanners. We utilize a cycle-consistent generative adversarial network to learn a bidirectional mapping and perform normalization between MRIs produced by GE Healthcare and Siemens scanners in an unpaired setting. Initial experiments demonstrate that the traditional CycleGAN architecture struggles to preserve the anatomical structures of the breast during normalization. Thus, we propose two technical innovations in order to preserve both the shape of the breast as well as the tissue structures within the breast. First, we incorporate mutual information loss during training in order to ensure anatomical consistency. Second, we propose a modified discriminator architecture that utilizes a smaller field-of-view to ensure the preservation of finer details in the breast tissue.Results: Quantitative and qualitative evaluations show that the second innovation consistently preserves the breast shape and tissue structures while also performing the proper intensity normalization and noise distribution mapping.Conclusion: Our results demonstrate that the proposed model can successfully learn a bidirectional mapping and perform normalization between MRIs produced by different vendors, potentially enabling improved diagnosis and detection of breast cancer. All the data used in this study are publicly available at https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=70226903.

Adversarial Partial Multi-Label Learning with Label Disambiguation

Partial multi-label learning (PML), which tackles the problem of learning multi-label prediction models from instances with overcomplete noisy annotations, has recently started gaining attention from the research community. In this paper, we propose a novel adversarial learning model, PML-GAN, under a generalized encoder-decoder framework for partial multi-label learning. The PML-GAN model uses a disambiguation network to identify irrelevant labels and uses a multi-label prediction network to map the training instances to their disambiguated label vectors, while deploying a generative adversarial network as an inverse mapping from label vectors to data samples in the input feature space. The learning of the overall model corresponds to a minimax adversarial game, which enhances the correspondence of input features with the output labels in a bi-directional mapping. Extensive experiments are conducted on both synthetic and real-world partial multi-label datasets, while the proposed model demonstrates the state-of-the-art performance.

Bidirectional Mapping of Brain MRI and PET With 3D Reversible GAN for the Diagnosis of Alzheimer's Disease.

Combining multi-modality data for brain disease diagnosis such as Alzheimer’s disease (AD) commonly leads to improved performance than those using a single modality. However, it is still challenging to train a multi-modality model since it is difficult in clinical practice to obtain complete data that includes all modality data. Generally speaking, it is difficult to obtain both magnetic resonance images (MRI) and positron emission tomography (PET) images of a single patient. PET is expensive and requires the injection of radioactive substances into the patient’s body, while MR images are cheaper, safer, and more widely used in practice. Discarding samples without PET data is a common method in previous studies, but the reduction in the number of samples will result in a decrease in model performance. To take advantage of multi-modal complementary information, we first adopt the Reversible Generative Adversarial Network (RevGAN) model to reconstruct the missing data. After that, a 3D convolutional neural network (CNN) classification model with multi-modality input was proposed to perform AD diagnosis. We have evaluated our method on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, and compared the performance of the proposed method with those using state-of-the-art methods. The experimental results show that the structural and functional information of brain tissue can be mapped well and that the image synthesized by our method is close to the real image. In addition, the use of synthetic data is beneficial for the diagnosis and prediction of Alzheimer’s disease, demonstrating the effectiveness of the proposed framework.

Bidirectional Mapping Augmentation Algorithm for Synthetic Images Based on Generative Adversarial Network

Bidirectional Mapping Augmentation Algorithm for Synthetic Images Based on Generative Adversarial Network

Relation-Induced Multi-Modal Shared Representation Learning for Alzheimer's Disease Diagnosis.

The fusion of multi-modal data (e.g., magnetic resonance imaging (MRI) and positron emission tomography (PET)) has been prevalent for accurate identification of Alzheimer's disease (AD) by providing complementary structural and functional information. However, most of the existing methods simply concatenate multi-modal features in the original space and ignore their underlying associations which may provide more discriminative characteristics for AD identification. Meanwhile, how to overcome the overfitting issue caused by high-dimensional multi-modal data remains appealing. To this end, we propose a relation-induced multi-modal shared representation learning method for AD diagnosis. The proposed method integrates representation learning, dimension reduction, and classifier modeling into a unified framework. Specifically, the framework first obtains multi-modal shared representations by learning a bi-directional mapping between original space and shared space. Within this shared space, we utilize several relational regularizers (including feature-feature, feature-label, and sample-sample regularizers) and auxiliary regularizers to encourage learning underlying associations inherent in multi-modal data and alleviate overfitting, respectively. Next, we project the shared representations into the target space for AD diagnosis. To validate the effectiveness of our proposed approach, we conduct extensive experiments on two independent datasets (i.e., ADNI-1 and ADNI-2), and the experimental results demonstrate that our proposed method outperforms several state-of-the-art methods.