Rectification Network Research Articles

A simplified adversarial architecture for cross-subject silent speech recognition using electromyography

Objective. The decline in the performance of electromyography (EMG)-based silent speech recognition is widely attributed to disparities in speech patterns, articulation habits, and individual physiology among speakers. Feature alignment by learning a discriminative network that resolves domain offsets across speakers is an effective method to address this problem. The prevailing adversarial network with a branching discriminator specializing in domain discrimination renders insufficiently direct contribution to categorical predictions of the classifier.Approach. To this end, we propose a simplified discrepancy-based adversarial network with a streamlined end-to-end structure for EMG-based cross-subject silent speech recognition. Highly aligned features across subjects are obtained by introducing a Nuclear-norm Wasserstein discrepancy metric on the back end of the classification network, which could be utilized for both classification and domain discrimination. Given the low-level and implicitly noisy nature of myoelectric signals, we devise a cascaded adaptive rectification network as the front-end feature extraction network, adaptively reshaping the intermediate feature map with automatically learnable channel-wise thresholds. The resulting features effectively filter out domain-specific information between subjects while retaining domain-invariant features critical for cross-subject recognition.Main results. A series of sentence-level classification experiments with 100 Chinese sentences demonstrate the efficacy of our method, achieving an average accuracy of 89.46% tested on 40 new subjects by training with data from 60 subjects. Especially, our method achieves a remarkable 10.07% improvement compared to the state-of-the-art model when tested on 10 new subjects with 20 subjects employed for training, surpassing its result even with three times training subjects.Significance. Our study demonstrates an improved classification performance of the proposed adversarial architecture using cross-subject myoelectric signals, providing a promising prospect for EMG-based speech interactive application.

Coronary artery segmentation in CCTA images based on multi-scale feature learning.

Coronary artery segmentation is a prerequisite in computer-aided diagnosis of Coronary Artery Disease (CAD). However, segmentation of coronary arteries in Coronary Computed Tomography Angiography (CCTA) images faces several challenges. The current segmentation approaches are unable to effectively address these challenges and existing problems such as the need for manual interaction or low segmentation accuracy. A Multi-scale Feature Learning and Rectification (MFLR) network is proposed to tackle the challenges and achieve automatic and accurate segmentation of coronary arteries. The MFLR network introduces a multi-scale feature extraction module in the encoder to effectively capture contextual information under different receptive fields. In the decoder, a feature correction and fusion module is proposed, which employs high-level features containing multi-scale information to correct and guide low-level features, achieving fusion between the two-level features to further improve segmentation performance. The MFLR network achieved the best performance on the dice similarity coefficient, Jaccard index, Recall, F1-score, and 95% Hausdorff distance, for both in-house and public datasets. Experimental results demonstrate the superiority and good generalization ability of the MFLR approach. This study contributes to the accurate diagnosis and treatment of CAD, and it also informs other segmentation applications in medicine.

Depthwise Convolution for Multi-Agent Communication With Enhanced Mean-Field Approximation.

Multi-Agent settings remain a fundamental challenge in the reinforcement learning (RL) domain due to the partial observability and the lack of accurate real-time interactions across agents. In this article, we propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge within a large number of agents coexisting. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations and learn local communication between neighboring agents. To facilitate multi-agent coordination, we explicitly learn the effect of joint actions by taking the policies of neighboring agents as inputs. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions. To more effectively coordinate behaviors of neighboring agents, we enhance the mean-field approximation by a supervised policy rectification network (PRN) for rectifying real-time agent interactions and by a learnable compensation term for correcting the approximation bias. The proposed method enables efficient coordination as well as outperforms several baseline approaches on the adaptive traffic signal control (ATSC) task and the StarCraft II multi-agent challenge (SMAC).

AIR-CNN: a lightweight automatic image rectification CNN used for barrel distortion

Barrel distortions often exist in images captured by wide-angle lenses, and the presence of barrel distortions reduces the label-making accuracy of algorithms and the precision rate of final target detection and semantic recognition. To reduce the interference of barrel distortions on imaging, we have proposed a lightweight image rectification network automatic image rectification CNN (AIR-CNN) for barrel distortion. The network is based on a parameter sharing (PS) convolutional neural network structure, which is trained on the distorted image dataset to predict the pixel displacement field between the distorted image and the rectified image, and finally restores the rectified image based on the predicted pixel displacement field. The experimental results show that the AIR-CNN can greatly reduce the number of network parameters through the PS mechanism and implicitly learns the texture features by asymmetric convolution kernels to obtain higher rectification accuracy at a lower computational cost, and automatically obtain the distortion parameters of the camera without special calibration methods. The AIR-CNN outperforms previous image rectification methods in both intuitive and quantitative comparisons (EPE, PSNR, NRMSE, SSIM).

Level set guided region prototype rectification network for retinal vessel segmentation

Retinal vessel segmentation refers to extracting the vessel region with continuous and smooth boundaries from retinal images, which is of great significance in clinical practices. However, due to the weak and blurry edges of targets as well as interference (such as optic cup and disc) in the background, current deep neural network-based methods struggle in extracting features with discriminative semantics while preserving continuous edges. To enforce continuous predictions of weak edges, we propose a level set guided region prototype rectification (LSRPR) framework and a novel level set loss (LS-loss) with learnable and self-guided mechanisms. Specifically, the LSRPR firstly takes features of the last layer from the decoders of a U-Net version as input and rectified the region prototype by an auxiliary self-supervised level set loss, then the pre-trained model is fine-tuned by using supervised level set loss. The LS-loss facilitates the model to generate reliable guidance and enhances the continuous of edges among the decoders of neural network model. The proposed method is simple, yet effective, which can easily be extended to other frameworks. The quantitative and qualitative experimental results on public retinal vessel datasets indicate the effectiveness of the region prototype rectification compared to other SOTA models. Our code is available at Github:https://github.com/tweedlemoon/LSRPR.

A dual-band, wide-angle absorbing metasurface for EM energy harvesting and wireless power transfer

In this paper, a dual-band metasurface is designed for electromagnetic (EM) applications in the sub-6G band. The metasurface consists of a hexagonal Complementary Split Ring Resonator (CSRR) absorber array, a dual-band impedance-matched rectifier network, and a load. The CSRR metasurface absorber is designed by combining a hexagonal prism and a hexagonal ring. The proposed absorber is surrounded by metal vias to improve the operating stability of each cell. By integrating the absorber array with the rectification network, the system redundancy and power loss are reduced. The designed metasurface is capable of harvesting EM energy in different frequencies using one port, even at a wide incidence angle. Dual-band rectification within the Sub-6G range is achieved by the rectification network at low input power density. We fabricated and measured a 3 × 3 hexagonal CSRR metasurface array, and the experimental result shows that the optimal input power is 2 dBm (the incident power density is 169 μW/cm2), the radio frequency(RF)-to-direct current(DC) efficiencies are 57.6% and 47% at 3.8 and 5.2 GHz, respectively.

Image projective transformation rectification with synthetic data for smartphone-captured chest X-ray photos classification

Automatic interpretation of chest X-ray (CXR) photos taken by smartphones at the same performance level as with digital CXRs is challenging, due to the projective transformation caused by the non-ideal camera position. Existing rectification methods for other camera-captured photos (document photos, license plate photos, etc.) cannot precisely rectify the projective transformation of CXR photos, due to its specific projective transformation type. In this paper, we propose an innovative deep learning-based Projective Transformation Rectification Network (PTRN) to automatically rectify the projective transformation of CXR photos by predicting the projective transformation matrix. Additionally, synthetic CXR photos are generated for training with the consideration of visual artifacts of natural images. The effectiveness of the proposed classification pipeline with PTRN is evaluated in the CheXphoto smartphone-captured CXR photo classification competition. It achieves first place with a huge performance improvement (ours 0.850, second-best 0.762, in AUC). Moreover, experimental results show that our approach successfully achieves the same performance level of digital CXR classification (AUC 0.893) on CXR photo classification (AUC 0.893).

FishRecGAN: An End to End GAN Based Network for Fisheye Rectification and Calibration

We propose an end-to-end deep learning approach to rectify fisheye images and simultaneously calibrate camera intrinsic and distortion parameters. Our method consists of two parts: a Quick Image Rectification Module developed with a Pix2Pix GAN and Wasserstein GAN (W-Pix2PixGAN), and a Calibration Module with a CNN architecture. Our Quick Rectification Network performs robust rectification with good resolution, making it suitable for constant calibration in camera-based surveillance equipment. To achieve highquality calibration, we use the straightened output from the Quick Rectification Module as a guidance-like semantic feature map for the Calibration Module to learn the geometric relationship between the straightened feature and the distorted feature. We train and validate our method with a large synthesized dataset labeled with well-simulated parameters applied to a perspective image dataset. Our solution has achieved robust performance in high-resolution with a significant PSNR value of 22.343. 1

A Two-Level Rectification Attention Network for Scene Text Recognition

Scene text recognition is a challenging task in the computer vision field due to the diversity of text styles and the complexity of the image backgrounds. In recent decades, numerous text rectification and recognition methods have been proposed to solve these problems. However, most of these methods rectify texts at the geometry level or pixel level. The former is limited by geometric constraints, and the latter is prone to blurring the text. In this paper, we propose a two-level rectification attention network (TRAN) to rectify and recognize texts. This network consists of two parts: a two-level rectification network (TORN) and an attention-based recognition network (ABRN). Specifically, the TORN first rectifies texts at the geometry level and then performs a pixel-level adjustment, which not only eliminates the geometric constraints but also renders clear texts. The ABRN's role is to recognize text in the rectified images. To improve the feature extraction ability of our model, we design a new channel-wise and kernel-wise attention unit, which enables the network to handle significant variations of character size and channel interdependencies. Furthermore, we propose a skip training strategy to make our model converge smoothly. We conduct experiments on various benchmarks, including regular and irregular datasets. The experimental results show that our method achieves a state-of-the-art performance.

Revisiting Radial Distortion Rectification in Polar-Coordinates: A New and Efficient Learning Perspective

Fisheye cameras can capture a large field-of-view (Fov) scene but it introduces severe radial distortion in images. Thus, distortion rectification is a crucial step for subsequent computer vision tasks using fisheye cameras. A prevalent type of method predicts the displacement field between the input and output to rectify the distorted images. However, it is challenging to estimate the accurate flow in Cartesian coordinates (both <inline-formula> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$y$ </tex-math></inline-formula> directions), in which the sampling strategy of the convolution kernel ignores the radial symmetry of distortion. In general, the pixel&#x2019;s distortion at the same radius from the center is the same, while the radius corresponds to one parameter in polar coordinates. Motivated by this fact, we exploit the radial symmetry of distortion to predict a more straightforward one-dimensional flow, transforming the distorted image into the polar coordinates domain instead of predicting two-dimensional flow in <inline-formula> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$y$ </tex-math></inline-formula> directions. Specifically, we propose a Polar coordinates Distortion Rectification Network (PCDRN), whose sampling strategy corresponds to the radial distortion characteristic so that a more accurate flow can be predicted. To eliminate the blurs and ring artifacts induced by the coordinates transformation, a Polar-To-Cartesian Appearance Enhancement Network is designed to enhance the local appearance of rectified images. Experimental results on the synthesized dataset and real-world dataset demonstrate the superiority of our approach in both quantitative and qualitative evaluations.

An attention-based network for serial number recognition on banknotes

The serial number recognition (SNR) on banknotes is essential for currency circulation. The performance of the existing SNR methods is significantly influenced by character segmentation, which is challenging due to uneven illumination and complex background. In this paper, we apply deep learning techniques to SNR by proposing an attention-based network, which can be end-to-end trained to avoid the problem of character segmentation. The proposed framework contains two parts: rectification and recognition. First, the rectification network, which can be trained in a weakly supervised manner without additional manual annotations, is built to automatically rectify the tilted and loosely-bounded images and reduce the difficulty of recognition. Then, the recognition network, an attention-based sequence model, recognizes serial number characters from the rectified image holistically, rather than segmenting and recognizing individual characters. To address the problem of complex textures on banknotes, we integrate the deformable convolution into the recognition network, which adaptively focuses on the character regions by using flexible receptive fields to accurately extract optimal character features, while ignoring redundant background information. Extensive experiments conducted on CNY, KRW, EUR and JPY banknotes, demonstrate that the proposed method achieves higher accuracy than the existing methods.

Adaptive rectification based adversarial network with spectrum constraint for high-quality PET image synthesis.

Positron emission tomography (PET) is a typical nuclear imaging technique, which can provide crucial functional information for early brain disease diagnosis. Generally, clinically acceptable PET images are obtained by injecting a standard-dose radioactive tracer into human body, while on the other hand the cumulative radiation exposure inevitably raises concerns about potential health risks. However, reducing the tracer dose will increase the noise and artifacts of the reconstructed PET image. For the purpose of acquiring high-quality PET images while reducing radiation exposure, in this paper, we innovatively present an adaptive rectification based generative adversarial network with spectrum constraint, named AR-GAN, which uses low-dose PET (LPET) image to synthesize standard-dose PET (SPET) image of high-quality. Specifically, considering the existing differences between the synthesized SPET image by traditional GAN and the real SPET image, an adaptive rectification network (AR-Net) is devised to estimate the residual between the preliminarily predicted image and the real SPET image, based on the hypothesis that a more realistic rectified image can be obtained by incorporating both the residual and the preliminarily predicted PET image. Moreover, to address the issue of high-frequency distortions in the output image, we employ a spectral regularization term in the training optimization objective to constrain the consistency of the synthesized image and the real image in the frequency domain, which further preserves the high-frequency detailed information and improves synthesis performance. Validations on both the phantom dataset and the clinical dataset show that the proposed AR-GAN can estimate SPET images from LPET images effectively and outperform other state-of-the-art image synthesis approaches.

Robust Correlation Tracking in Unmanned Aerial Vehicle Videos via Deep Target-Specific Rectification Networks

Unmanned Aerial Vehicle (UAV) Videos have received active research attention in the remote sensing field by taking full advantage of the bird&#x2019;s eye view offered by UAV. At the same time, visual tracking approaches based on discriminative correlation filters (DCF) have recently achieved increasing popularity and success. Despite their success, the tracking robustness and accuracy of existing DCF-based trackers are hard to promote in challenging tracking scenarios of aerial videos due to excessive reliance on the response map and fixed linear model update strategy. To resolve these issues, we propose a robust DCF-based tracking framework via an effective pretrained rectification network for UAV-based remote sensing. Specifically, the target-specific rectification network is offline trained to discriminatively classify the target and background. During the online tracking stage, the DCF module performs fast inference to obtain the potential locations of the target. After that, the deep rectification network evaluates the correlation-specific proposals offered by the DCF module and provides precise tracking results. Besides, to achieve a robust and adaptive model update strategy, we propose to finetune both the DCF module and rectification network according to the classification confidence of the estimated result. Extensive experimental results on recent UAV benchmarks demonstrate that our method achieves better performance than other competing algorithms.

PETR: Rethinking the Capability of Transformer-Based Language Model in Scene Text Recognition.

The exploration of linguistic information promotes the development of scene text recognition task. Benefiting from the significance in parallel reasoning and global relationship capture, transformer-based language model (TLM) has achieved dominant performance recently. As a decoupled structure from the recognition process, we argue that TLM's capability is limited by the input low-quality visual prediction. To be specific: 1) The visual prediction with low character-wise accuracy increases the correction burden of TLM. 2) The inconsistent word length between visual prediction and original image provides a wrong language modeling guidance in TLM. In this paper, we propose a Progressive scEne Text Recognizer (PETR) to improve the capability of transformer-based language model by handling above two problems. Firstly, a Destruction Learning Module (DLM) is proposed to consider the linguistic information in the visual context. DLM introduces the recognition of destructed images with disordered patches in the training stage. Through guiding the vision model to restore patch orders and make word-level prediction on the destructed images, visual prediction with high character-wise accuracy is obtained by exploring inner relationship between the local visual patches. Secondly, a new Language Rectification Module (LRM) is proposed to optimize the word length for language guidance rectification. Through progressively implementing LRM in different language modeling steps, a novel progressive rectification network is constructed to handle some extremely challenging cases (e.g. distortion, occlusion, etc.). By utilizing DLM and LRM, PETR enhances the capability of transformer-based language model from a more general aspect, that is, focusing on the reduction of correction burden and rectification of language modeling guidance. Compared with parallel transformer-based methods, PETR obtains 1.0% and 0.8% improvement on regular and irregular datasets respectively while introducing only 1.7M additional parameters. The extensive experiments on both English and Chinese benchmarks demonstrate that PETR achieves the state-of-the-art results.

Adversarial learning based attentional scene text recognizer

Abstract In this paper, we propose an adversarial learning based attentional scene text recognizer to solve the distortion problem of scene text image. We choose a rectification module which can rectify images in both horizontal and vertical directions, and use a recognizer based on the attention mechanism. Through the adversarial learning of the rectification network and the recognition network, we iteratively improve the rectification effect and the recognition performance. The entire network is trained with weak supervision, so only images and corresponding text labels are needed. Our method achieves high performance for both regular and irregular scene text images, and the experimental results tested on multiple benchmarks prove that our method achieves the performance of state-of-the-art.

Progressive rectification network for irregular text recognition

Scene text recognition has received increasing attention in the research community. Text in the wild often possesses irregular arrangements, which typically include perspective, curved, and oriented texts. Most of the existing methods do not work well for irregular text, especially for severely distorted text. In this paper, we propose a novel progressive rectification network (PRN) for irregular scene text recognition. Our PRN progressively rectifies the irregular text to a front-horizontal view and further boosts the recognition performance. The distortions are removed step by step by leveraging the observation that the intermediate rectified result provides good guidance for subsequent higher quality rectification. Additionally, by decomposing the rectification process into multiple procedures, the difficulty of each step is considerably mitigated. First, we specifically perform a rough rectification, and then adopt iterative refinement to gradually achieve optimal rectification. Additionally, to avoid the boundary damage problem in direct iterations, we design an envelope-refinement structure to maintain the integrity of the text during the iterative process. Instead of the rectified images, the text line envelope is tracked and continually refined, which implicitly models the transformation information. Then, the original input image is consistently utilized for transformation based on the refined envelope. In this manner, the original character information is preserved until the final transformation. These designs lead to optimal rectification to boost the performance of succeeding recognition. Extensive experiments on eight challenging datasets demonstrate the superiority of our method, especially on irregular benchmarks.

Learning Rich Part Hierarchies with Progressive Attention Networks for Fine-Grained Image Recognition.

We investigate the localization of subtle yet discriminative parts for fine-grained image recognition. Based on the observation that such parts typically exist within a hierarchical structure (e.g., from a coarse-scale "head" to a fine-scale "eye" when recognizing bird species), we propose a novel progressive-attention convolutional neural network (PA-CNN) to progressively localize parts at multiple scales. The PA-CNN localizes parts in two steps, where a part proposal network (PPN) generates multiple local attention maps, and a part rectification network (PRN) learns part-specific features from each proposal and provides the PPN with refined part locations. This coupling of the PPN and PRN allows them to be optimized in a mutually reinforcing manner, leading to improved pinpointing of fine-grained parts. Moreover, the convolutional parameters for a PPN at a finer scale can be inherited from the PRN at a coarser scale, enabling a rich part hierarchy (e.g., eye and beak in a bird's head) to be learned in a stacked fashion. Case studies show that PA-CNN can precisely identify parts without using bounding box/part annotations. In addition, quantitative evaluations demonstrate that PA-CNN yields state-of-the-art performance in three challenging fine-grained recognition tasks. i.e., CUB-200-2011, FGVC-Aircraft, and Stanford Cars.

MORAN: A Multi-Object Rectified Attention Network for scene text recognition

Irregular text is widely used. However, it is considerably difficult to recognize because of its various shapes and distorted patterns. In this paper, we thus propose a multi-object rectified attention network (MORAN) for general scene text recognition. The MORAN consists of a multi-object rectification network and an attention-based sequence recognition network. The multi-object rectification network is designed for rectifying images that contain irregular text. It decreases the difficulty of recognition and enables the attention-based sequence recognition network to more easily read irregular text. It is trained in a weak supervision way, thus requiring only images and corresponding text labels. The attention-based sequence recognition network focuses on target characters and sequentially outputs the predictions. Moreover, to improve sensitivity of the attention-based sequence recognition network, a fractional pickup method is proposed for an attention-based decoder in the training phase. With the rectification mechanism, the MORAN can read both regular and irregular scene text. Extensive experiments on various benchmarks are conducted, which show that the MORAN achieves state-of-the-art performance. The source code is available.11https://github.com/Canjie-Luo/MORAN_v2.

ASTER: An Attentional Scene Text Recognizer with Flexible Rectification.

A challenging aspect of scene text recognition is to handle text with distortions or irregular layout. In particular, perspective text and curved text are common in natural scenes and are difficult to recognize. In this work, we introduce ASTER, an end-to-end neural network model that comprises a rectification network and a recognition network. The rectification network adaptively transforms an input image into a new one, rectifying the text in it. It is powered by a flexible Thin-Plate Spline transformation which handles a variety of text irregularities and is trained without human annotations. The recognition network is an attentional sequence-to-sequence model that predicts a character sequence directly from the rectified image. The whole model is trained end to end, requiring only images and their groundtruth text. Through extensive experiments, we verify the effectiveness of the rectification and demonstrate the state-of-the-art recognition performance of ASTER. Furthermore, we demonstrate that ASTER is a powerful component in end-to-end recognition systems, for its ability to enhance the detector.

The Heartmate II: design and development of a fully sealed axial flow left ventricular assist system.

Our group is developing the control and power transmission components required to implement a permanent and fully sealed left ventricular assist system (LVAS). Starting with the percutaneously powered HeartMate II blood pump, our development efforts are focused in the following areas: a complete redesign of the transcutaneous energy transmission system (TETS) to include a rectification network and autonomous voltage regulation within the secondary coil, a hermetically sealed electronics package containing a miniaturized implementation of the existing redundant drive and control electronics with several power-input options, an implanted rechargeable lithium ion battery pack capable of providing up to 1 h of untethered operation, implantable electrical connectors that allow components to be connected after placement in the body or to be replaced if needed, and a radio telemetry subsystem to transmit diagnostic information and to permit remote adjustment of selected parameters.