Auxiliary Network Research Articles

Learning-based data-driven optimal deployment control of tethered space robot

To avoid complex constraints of the traditional nonlinear method for tethered space robot (TSR) deployment, a data-driven optimal control framework with an improve deep learning based Koopman operator is proposed in this work. In consideration of nonlinearity of tethered space robot dynamics, its finite dimensional global linear representation called lifted linear system is derived with the Koopman operator theory. A deep learning scheme is adopted to find the embedding functions associate with Koopman operator. And an auxiliary neural network is developed to encode the nonlinear control term of finite dimensional lifted system. Then a controllability constraint is considered for learning a controllable lifted linear system. Besides two loss functions that relate to reconstruction and prediction ability of lifted linear system are designed for training the deep neural network. With the learned lifted linear dynamics, Linear Quadratic Regulator (LQR) is applied to derive the optimal control policy for the tethered space robot deployment. Finally, simulation results verify the effectiveness of proposed framework and show that it could deploy tethered space robot more quickly with less swing of in-plane angle.

Infrared-Visible Synthetic Data from Game Engine for Image Fusion Improvement

Limited by the shooting scenes and angles of fixed cameras, the existing datasets generally lack many detailed pedestrian models in diverse scenarios. Existing deep learning-based image fusion methods, for this reason, bring about overfitting or insufficient information of fusion results in varying degrees. To address this challenge, A new infrared-visible pedestrian synthetic dataset (GIVF) with a synthetic data tagger (GSDT) is constructed and an improved end-to-end image fusion network (FSGAN) is proposed to validate infrared and visible fusion. In the model, the method uses an auxiliary network to extract features that complement the cascade network of the main path, effectively improving the ability to extract pedestrian texture details. Experimental results show that FSGAN can be well applied to GIVF. By conducting extensive comparative experiments with eight state-of-the-art image fusion methods. FSGAN shows better performance than those comparison methods, especially in the two evaluation indexes visual information fidelity (VIF) and structural similarity measurement (SSIM). Besides, by comparing the quantitative analysis results of various methods, and evaluating the fusion results of real images in complex environments on other three datasets, we conclude that FSGAN can be better applied to GIVF datasets than other popular methods, and has outstanding performance in generalization.

Visible-hidden hybrid automatic feature engineering via multi-agent reinforcement learning

Feature engineering is one of the most important and time-consuming steps of machine learning algorithms. In recent years, automatic feature engineering (AFE) methods have received a lot of attention due to their low cost and scalability. However, the existing AFE methods do not take into account the interactions of features in the evolution process and cannot achieve effective feature selection, which weakens the performance of these methods. To tackle the above issues, a novel visible-hidden hybrid automatic feature engineering (VHAFE) method is proposed in this work. Specifically, a visible-hidden hybrid feature transformation graph (VHFTG) is devised to represent both various kinds of feature transformation functions and the feature selection process. Afterward, a multi-pointer state identification method is proposed, which enables the interactions among derived features and the efficient utilization of historically derived features. Furthermore, the multi-agent reinforcement learning algorithm is introduced to optimize the evolution process of VHFTG, where an input variable selection network is devised as the auxiliary policy network to avoid the generation of excessive noise features. Additionally, the VHAFE achieves the state-of-the-art results on a total of 19 public datasets and 1 dataset collected from the actual industrial operation process of gas turbines, which demonstrates the effectiveness of the proposed method.

Information bottleneck-based Hebbian learning rule naturally ties working memory and synaptic updates.

Deep neural feedforward networks are effective models for a wide array of problems, but training and deploying such networks presents a significant energy cost. Spiking neural networks (SNNs), which are modeled after biologically realistic neurons, offer a potential solution when deployed correctly on neuromorphic computing hardware. Still, many applications train SNNs offline, and running network training directly on neuromorphic hardware is an ongoing research problem. The primary hurdle is that back-propagation, which makes training such artificial deep networks possible, is biologically implausible. Neuroscientists are uncertain about how the brain would propagate a precise error signal backward through a network of neurons. Recent progress addresses part of this question, e.g., the weight transport problem, but a complete solution remains intangible. In contrast, novel learning rules based on the information bottleneck (IB) train each layer of a network independently, circumventing the need to propagate errors across layers. Instead, propagation is implicit due the layers' feedforward connectivity. These rules take the form of a three-factor Hebbian update a global error signal modulates local synaptic updates within each layer. Unfortunately, the global signal for a given layer requires processing multiple samples concurrently, and the brain only sees a single sample at a time. We propose a new three-factor update rule where the global signal correctly captures information across samples via an auxiliary memory network. The auxiliary network can be trained a priori independently of the dataset being used with the primary network. We demonstrate comparable performance to baselines on image classification tasks. Interestingly, unlike back-propagation-like schemes where there is no link between learning and memory, our rule presents a direct connection between working memory and synaptic updates. To the best of our knowledge, this is the first rule to make this link explicit. We explore these implications in initial experiments examining the effect of memory capacity on learning performance. Moving forward, this work suggests an alternate view of learning where each layer balances memory-informed compression against task performance. This view naturally encompasses several key aspects of neural computation, including memory, efficiency, and locality.

Bridging knowledge distillation gap for few-sample unsupervised semantic segmentation

Due to privacy, security, and costly labeling of images, unsupervised semantic segmentation with very few samples has become a promising direction, but still remains unexplored. This inspires us to introduce the few-sample unsupervised semantic segmentation task, which is very challenging because generalizing the segmentation model from only a few unlabeled images is far from sufficient. We address this problem in the knowledge distillation perspective, by proposing a medium-sized auxiliary network as the bridge, which narrows down the semantic knowledge gap between teacher network (large) and student network (small). To this end, we develop the Knowledge Distillation Bridge (KDB) framework for few-sample unsupervised semantic segmentation. In particular, it consists of the teacher-auxiliary-student architecture, which adopts the block-wise distillation that encourages the auxiliary to imitate the teacher and the student to imitate the auxiliary. In this way, the knowledge gap between the source feature distribution and the target one is reduced, allowing the student with the smaller network to be readily deployed in highly-demanding environment. Meanwhile, each channel characterizes different semantics in feature map, which motivates us to distill the features of decoder in a channel-wise manner. Extensive experiments on two benchmarks including Pascal VOC2012 and Cityscapes demonstrate the promising performance of the proposed method, which strikes a good balance between precision and speed, e.g., it achieves the inference speed of 230 fps for a 512×512 image.

Gaze-assisted visual grounding via knowledge distillation for referred object grasping with under-specified object referring

Understanding human referring intention is the key challenge in referred object grasping tasks. Current Referring Expression Comprehension (REC) and Referring Expression Segmentation (RES) methods enable robots to ground the referred object through precise language commands while ignoring the under-specified referring expressions in practical scenarios. In this paper, we propose a gaze-assisted visual grounding network, which efficiently estimates the human gaze via knowledge distillation techniques to provide visual guidance for inferring human referring intention. Specifically, a gaze auxiliary network is introduced to incorporate the human gaze to implicitly eliminate the referring ambiguity. A cross-attention-based multi-modal fusion module is designed with a balanced structure to project the diverse modalities into a common feature space to balance the inter-modal information. A tabletop object dataset with human gaze and under-specified referring expressions was established to evaluate our method. Experimental results show that our method achieves advanced results compared to the state-of-the-art visual grounding method, with a gain of 8.23% on REC precision and 10.44% on RES mIoU (mean Intersection over Union). The effectiveness and superiority of our method are also demonstrated through real-world referred object grasping experiments. The code is now available at https://github.com/robotseu/GVGNet_KD.

Smooth-Guided Implicit Data Augmentation for Domain Generalization.

The training process of a domain generalization (DG) model involves utilizing one or more interrelated source domains to attain optimal performance on an unseen target domain. Existing DG methods often use auxiliary networks or require high computational costs to improve the model's generalization ability by incorporating a diverse set of source domains. In contrast, this work proposes a method called Smooth-Guided Implicit Data Augmentation (SGIDA) that operates in the feature space to capture the diversity of source domains. To amplify the model's generalization capacity, a distance metric learning (DML) loss function is incorporated. Additionally, rather than depending on deep features, the suggested approach employs logits produced from cross entropy (CE) losses with infinite augmentations. A theoretical analysis shows that logits are effective in estimating distances defined on original features, and the proposed approach is thoroughly analyzed to provide a better understanding of why logits are beneficial for DG. Moreover, to increase the diversity of the source domain, a sampling-based method called smooth is introduced to obtain semantic directions from interclass relations. The effectiveness of the proposed approach is demonstrated through extensive experiments on widely used DG, object detection, and remote sensing datasets, where it achieves significant improvements over existing state-of-the-art methods across various backbone networks.

Implementation and Analysis of an Efficient Soft-Switching Battery Wireless Charger with Re-Configurable Rectifier

To significantly reduce switching loss of the high frequency inverter in battery wireless charger (BWC), improve the soft-switching stability and system reliability, a novel soft-switching BWC with low loss auxiliary network and re-configurable rectifier is presented. The presented soft-switching BWC can implement the smooth switching between constant current mode (CCM) and constant voltage mode (CVM) without frequency switching and relays, which is beneficial to enhancing the system stability. With the help of a novel low energy consumption auxiliary circuit, the power switch tubes in the H-bridge inverter can implement zero-voltage soft switch-ON and zero-current soft switch-OFF in the full range, completely eliminating switching loss of the inverter. The proposed scheme can enhance the system efficiency while refraining from the soft-switching range limitation and soft-switching failure caused by the system parameter perturbation. Finally, a 1-kW soft-switching BWC principle prototype is developed to validate the effectiveness and correctness of the proposed scheme.

BIMO: Bootstrap Inter–Intra Modality at Once Unsupervised Learning for Multivariate Time Series

It is difficult to learn meaningful representations of time-series data since they are sparsely labeled and unpredictable. Hence, we propose bootstrap inter–intra modality at once (BIMO), an unsupervised representation learning method based on time series. Unlike previous works, the proposed BIMO method learns both inter-sample and intra-temporal modality representations simultaneously without negative pairs. BIMO comprises a main network and two auxiliary networks, namely inter-auxiliary and intra-auxiliary networks. The main network is trained to learn inter–intra modality representations sequentially by regulating the use of each auxiliary network dynamically. Thus, BIMO thoroughly learns inter–intra modality representations simultaneously. The experimental results demonstrate that the proposed BIMO method outperforms the state-of-the-art unsupervised methods and achieves comparable performance to existing supervised methods.

S2CANet: A self-supervised infrared and visible image fusion based on co-attention network

Existing methods for infrared and visible image fusion (IVIF) often overlook the analysis of common and distinct features among source images. Consequently, this study develops A self-supervised infrared and visible image fusion based on co-attention network, incorporating auxiliary networks and backbone networks in its design. The primary concept is to transform both common and distinct features into common features and reconstructed features, subsequently deriving the distinct features through their subtraction. To enhance the similarity of common features, we designed the fusion block based on co-attention (FBC) module specifically for this purpose, capturing common features through co-attention. Moreover, fine-tuning the auxiliary network enhances the image reconstruction effectiveness of the backbone network. It is noteworthy that the auxiliary network is exclusively employed during training to guide the self-supervised completion of IVIF by the backbone network. Additionally, we introduce a novel estimate for weighted fidelity loss to guide the fused image in preserving more brightness from the source image. Experiments conducted on diverse benchmark datasets demonstrate the superior performance of our S2CANet over state-of-the-art IVIF methods.

Attribute-level Sentiment Analysis Incorporating Auxiliary Information

In order to obtain the sentiment tendency of multiple attributes contained in the data reviews, this paper uses an attribute-level sentiment analysis model RBT-BiAtt-GCN, which incorporates auxiliary information and attention mechanism, to analyze the attribute-level sentiment analysis of e-commerce reviews of citrus. The model incorporates auxiliary information, attention mechanism, and GCN network on the basis of the previous one, and the final macro F1 value obtained can reach 78.21%, which achieves good results in comparison with other models.

A method of evaluating cell state based on data augmentation and ViT16

In this paper, based on the model of data augmentation and Vision Transformer 16 (ViT16), a method of assessment for electrolysis cell state is presented to get the real-time information of the current cell state, so as to improve current efficiency of process. Firstly, in order to solve the issue of the small sample data and improve classification accuracy, the method of data augmentation is performed on the flame hole images by using convolutional block attention module to improve auxiliary classifier generativhyhee adversarial network. Secondly, the deep feature data of the flame hole images is extracted by the method of ViT16, and the genetic algorithm is applied to eliminate the redundant feature data to improve the accuracy. Thirdly, the support vector machines model is employed to classify the feature data, and the aluminum cells are classified into cold, hot, and normal. Finally, the actual data are applied to the experiments of the above method, the results of experiments show that this method is better than other methods, and the accuracy of classifying the cell state is as high as 98.677%. This is of great significance for the guidance of aluminum electrolysis production process.

Novel Series-Parallel Phase-Shifted Full-Bridge Converters with Auxiliary LC Networks to Achieve Wide Lagging-Leg ZVS Range

Under light load conditions, the phase-shifted full-bridge (PSFB) converter often has difficulty in realizing the zero-voltage switching (ZVS) of the lagging-leg by relying on the energy of its resonant inductor; however, for the series-parallel PSFB converter applied in high-power applications, the lagging-leg still has the problem of difficult realization of ZVS. Based on this, the paper analyzes the reasons why the series-parallel PSFB converter has difficulty in achieving ZVS for the lagging-leg under light and heavy loads. Under interleaved control, the ZVS of the lagging-leg over the full load range is realized by adding an auxiliary LC branch at the midpoint of the lagging-leg of both submodules. Based on the double-bridge input-parallel-output-series (IPOS) PSFB converter, analyzing the working principle of the circuit after adding the auxiliary LC branch and extending it to the series-parallel PSFB converter. The design requirements of the LC auxiliary branch of the dual-bridge series-parallel PSFB converter are given and the effects of the LC auxiliary branch on the module operating state and device stress are analyzed. On this basis, an extension is carried out to give the working principle and design method of the auxiliary LC branch of the N-bridge series-parallel PSFB converter. Finally, a 100 kW Matlab/Simulink simulation model verifies the superior performance of the proposed LC auxiliary branch to realize the lagging-leg ZVS of the series-parallel PSFB converter under light and heavy loads and achieves a 1.09% peak efficiency improvement at rated load.

Breast Tumor Tissue Image Classification Using Single-Task Meta Learning with Auxiliary Network.

Breast cancer has a high mortality rate among cancers. If the type of breast tumor can be correctly diagnosed at an early stage, the survival rate of the patients will be greatly improved. Considering the actual clinical needs, the classification model of breast pathology images needs to have the ability to make a correct classification, even in facing image data with different characteristics. The existing convolutional neural network (CNN)-based models for the classification of breast tumor pathology images lack the requisite generalization capability to maintain high accuracy when confronted with pathology images of varied characteristics. Consequently, this study introduces a new classification model, STMLAN (Single-Task Meta Learning with Auxiliary Network), which integrates Meta Learning and an auxiliary network. Single-Task Meta Learning was proposed to endow the model with generalization ability, and the auxiliary network was used to enhance the feature characteristics of breast pathology images. The experimental results demonstrate that the STMLAN model proposed in this study improves accuracy by at least 1.85% in challenging multi-classification tasks compared to the existing methods. Furthermore, the Silhouette Score corresponding to the features learned by the model has increased by 31.85%, reflecting that the proposed model can learn more discriminative features, and the generalization ability of the overall model is also improved.

TRA-ACGAN: A motor bearing fault diagnosis model based on an auxiliary classifier generative adversarial network and transformer network

Motor bearing fault diagnosis is essential to guarantee production efficiency and avoid catastrophic accidents. Deep learning-based methods have been developed and widely used for fault diagnosis, and these methods have proven to be very effective in accurately diagnosing bearing faults. In this paper, study the application of generative adversarial networks (GANs) in motor bearing fault diagnosis to address the practical issue of insufficient fault data in industrial testing. Focus on the auxiliary classifier generative adversarial network (ACGAN), and the data expansion is carried out for small datasets. This paper present a novel transformer network and auxiliary classifier generative adversarial network (TRA-ACGAN) for motor bearing fault diagnosis, where the TRA-ACGAN combines an ACGAN with a transformer network to avoid the traditional iterative and convolutional structures. The attention mechanism is fully utilized to extract more effective features, and the dual-task coupling problem encountered in classical ACGANs is avoided. Experimental results with the CWRU dataset and the PU dataset in the field of motor bearing fault diagnosis demonstrate the suitability and superiority of the TRA-ACGAN.

Narrative and Space: Reconstruction of Heritage Network for South Guangdong Ancient Post Road

The ancient post road in Southern Guangdong is a channel of communication inside and outside Lingnan in different historical periods. It is also known as a corridor of population migration, a channel of commodity exchange and an ancient road of civilization and culture. In view of the limitations of focusing on space while ignoring narrative in the previous research of ancient post road, the analysis method of narrative spatial structure was introduced herein. Based on the four main functions of the ancient post road in Southern Guangdong, including postal exchange, military war, commerce and trade, and official travel, the holistic and systematic narrative space study of the ancient post road in Southern Guangdong was carried out in order to construct a relatively complete narrative space structure of the heritage corridor network. Research showed that the four functional narrative points all rely on corridors. Its spatial structure took Guangzhou as the main core, with Leizhou and Yangjiang in western Guangdong and Chaozhou in eastern Guangdong as the sub-core. A waterway-based land as the auxiliary network system was formed; The post road shifted from the original military function to the economic function, experienced the internal communication with the Central Plains to the coastal areas, and then to the world through the Maritime Silk Road. It became the epitome of the historical development process of Lingnan region. The narrative spatial structure analysis method can deeply explore the historical characteristics and cultural connotation of the heritage corridor, and promote the overall cognition of the post road. The analysis results have certain guiding significance for the construction and protection of the heritage corridor network of the ancient post road in Southern Guangdong. It can also provide a reference for other scholars to carry out the research on similar heritage corridors.

Age Encoded Adversarial Learning for Pediatric CT Segmentation.

Organ segmentation from CT images is critical in the early diagnosis of diseases, progress monitoring, pre-operative planning, radiation therapy planning, and CT dose estimation. However, data limitation remains one of the main challenges in medical image segmentation tasks. This challenge is particularly huge in pediatric CT segmentation due to children's heightened sensitivity to radiation. In order to address this issue, we propose a novel segmentation framework with a built-in auxiliary classifier generative adversarial network (ACGAN) that conditions age, simultaneously generating additional features during training. The proposed conditional feature generation segmentation network (CFG-SegNet) was trained on a single loss function and used 2.5D segmentation batches. Our experiment was performed on a dataset with 359 subjects (180 male and 179 female) aged from 5 days to 16 years and a mean age of 7 years. CFG-SegNet achieved an average segmentation accuracy of 0.681 dice similarity coefficient (DSC) on the prostate, 0.619 DSC on the uterus, 0.912 DSC on the liver, and 0.832 DSC on the heart with four-fold cross-validation. We compared the segmentation accuracy of our proposed method with previously published U-Net results, and our network improved the segmentation accuracy by 2.7%, 2.6%, 2.8%, and 3.4% for the prostate, uterus, liver, and heart, respectively. The results indicate that our high-performing segmentation framework can more precisely segment organs when limited training images are available.

Differentiable Auxiliary Learning for Sketch Re-Identification

Sketch re-identification (Re-ID) seeks to match pedestrians' photos from surveillance videos with corresponding sketches. However, we observe that existing works still have two critical limitations: (i) cross- and intra-modality discrepancies hinder the extraction of modality-shared features, (ii) standard triplet loss fails to constrain latent feature distribution in each modality with inadequate samples. To overcome the above issues, we propose a differentiable auxiliary learning network (DALNet) to explore a robust auxiliary modality for Sketch Re-ID. Specifically, for (i) we construct an auxiliary modality by using a dynamic auxiliary generator (DAG) to bridge the gap between sketch and photo modalities. The auxiliary modality highlights the described person in photos to mitigate background clutter and learns sketch style through style refinement. Moreover, a modality interactive attention module (MIA) is presented to align the features and learn the invariant patterns of two modalities by auxiliary modality. To address (ii), we propose a multi-modality collaborative learning scheme (MMCL) to align the latent distribution of three modalities. An intra-modality circle loss in MMCL brings learned global and modality-shared features of the same identity closer in the case of insufficient samples within each modality. Extensive experiments verify the superior performance of our DALNet over the state-of-the-art methods for Sketch Re-ID, and the generalization in sketch-based image retrieval and sketch-photo face recognition tasks.

Exploiting Auxiliary Caption for Video Grounding

Video grounding aims to locate a moment of interest matching the given query sentence from an untrimmed video. Previous works ignore the sparsity dilemma in video annotations, which fails to provide the context information between potential events and query sentences in the dataset. In this paper, we contend that exploiting easily available captions which describe general actions, i.e., auxiliary captions defined in our paper, will significantly boost the performance. To this end, we propose an Auxiliary Caption Network (ACNet) for video grounding. Specifically, we first introduce dense video captioning to generate dense captions and then obtain auxiliary captions by Non-Auxiliary Caption Suppression (NACS). To capture the potential information in auxiliary captions, we propose Caption Guided Attention (CGA) project the semantic relations between auxiliary captions and query sentences into temporal space and fuse them into visual representations. Considering the gap between auxiliary captions and ground truth, we propose Asymmetric Cross-modal Contrastive Learning (ACCL) for constructing more negative pairs to maximize cross-modal mutual information. Extensive experiments on three public datasets (i.e., ActivityNet Captions, TACoS and ActivityNet-CG) demonstrate that our method significantly outperforms state-of-the-art methods.

A novel rolling bearing fault diagnosis method for limited data

The ability of deep learning based bearing fault diagnosis methods is developing rapidly. However, it is difficult to obtain sufficient and comprehensive fault data in industrial applications, and changes in vibration signals caused by machine operating conditions can also hinder the accuracy of the model. The problem of limited data and frequent changes in operating conditions can seriously affect the effectiveness of deep learning methods. To tackle these challenges, a novel transformer model named the Differential Window Transformer (Dwin Transformer), which employs a new differential window self-attention mechanism, is presented in this paper. Meanwhile, the model introduces a hierarchical structure and a new patch merging to further improve performance. Furthermore, a new fault diagnosis model dealing with limited training data is proposed, which combines the Auxiliary Classifier Generative Adversarial Network with the Dwin Transformer(DT-ACGAN). The DT-ACGAN model can generate high-quality fake samples to facilitate training with limited data, significantly improving diagnostic capabilities. The proposed model can achieve excellent results under the dual challenges of limited data and variable working conditions by combining Dwin Transformer with GAN. The DT-ACGAN owns superior diagnostic accuracy and generalization performance under limited sample data and varying working environments when compared with other existing models. A comparative test about cross-domain ability is conducted on the Case Western Reserve University dataset and Jiang Nan University dataset. The results show that the proposed method achieves an average accuracy of 11.3% and 3.76% higher than other existing methods with limited data respectively.