Mapping Module Research Articles

A real-time ultra-low light color imaging system based on FPGA

This article shows a low light color image acquisition system, The core components of the system are the Fairchild’s SCMOS image sensor CIS1910F1111 and XILINX’s Artix-7 XC7A100T-2CSG324I FPGA, the remarkable advantage of the system is that it can obtain better color imaging effect under lower illumination environment, and the image noise is much less than other similar products. Based on the excellent imaging performance of the image detector, a high performance real-time low-light level color imaging system is developed. This imaging system can obtain the characteristic information of the targets under ultra-low illuminance environment, including the details, colors and so on. The hardware of the low light level imaging system mainly contains a color SCMOS image sensor and a FPGA, a driving circuit of a combination of DDR3, the ultra-low noise power conversion circuit and a Camera-Link and a 3G-SDI interface circuits. The SCMOS chip is used for photoelectric conversion of the shot scene and the FPGA is used for the control of the whole imaging system, image acquisition and image processing, etc, The FPGA software system consists of SCMOS initialize configuration and timing control module, automatic exposure control module, real-time color image processing module, imaging tone mapping module, image denoising module and image enhancement module. The automatic exposure control (AEC) module adaptively adjusts the average gray value of the region of interest. The module automatically calculates the exposure time and gain value of the next frame according to the current frame image data value. The real-time color image processing module includes color restoration, automatic white balance and color spaces conversion, etc. The image denoising module uses the advanced real-time guide-filter algorithm. The image tone mapping module and enhancement module are proposed based on an improved automatic threshold logarithmic and enhancement algorithm. Combining the hardware and FPGA soft algorithm with excellent performance, the imaging results show that the system can get good color image effect of the ultra-low light level about 10-2lx.

DV-LOAM: Direct Visual LiDAR Odometry and Mapping

Self-driving cars have experienced rapid development in the past few years, and Simultaneous Localization and Mapping (SLAM) is considered to be their basic capabilities. In this article, we propose a direct vision LiDAR fusion SLAM framework that consists of three modules. Firstly, a two-staged direct visual odometry module, which consists of a frame-to-frame tracking step, and an improved sliding window based thinning step, is proposed to estimate the accurate pose of the camera while maintaining efficiency. Secondly, every time a keyframe is generated, a dynamic objects considered LiDAR mapping module is utilized to refine the pose of the keyframe to obtain higher positioning accuracy and better robustness. Finally, a Parallel Global and Local Search Loop Closure Detection (PGLS-LCD) module that combines visual Bag of Words (BoW) and LiDAR-Iris feature is applied for place recognition to correct the accumulated drift and maintain a globally consistent map. We conducted a large number of experiments on the public dataset and our mobile robot dataset to verify the effectiveness of each module in our framework. Experimental results show that the proposed algorithm achieves more accurate pose estimation than the state-of-the-art methods.

Incorporating learnt local and global embeddings into monocular visual SLAM

Traditional approaches for Visual Simultaneous Localization and Mapping (VSLAM) rely on low-level vision information for state estimation, such as handcrafted local features or the image gradient. While significant progress has been made through this track, under more challenging configuration for monocular VSLAM, e.g., varying illumination, the performance of state-of-the-art systems generally degrades. As a consequence, robustness and accuracy for monocular VSLAM are still widely concerned. This paper presents a monocular VSLAM system that fully exploits learnt features for better state estimation. The proposed system leverages both learnt local features and global embeddings at different modules of the system: direct camera pose estimation, inter-frame feature association, and loop closure detection. With a probabilistic explanation of keypoint prediction, we formulate the camera pose tracking in a direct manner and parameterize local features with uncertainty taken into account. To alleviate the quantization effect, we adapt the mapping module to generate 3D landmarks better to guarantee the system’s robustness. Detecting temporal loop closure via deep global embeddings further improves the robustness and accuracy of the proposed system. The proposed system is extensively evaluated on public datasets (Tsukuba, EuRoC, and KITTI), and compared against the state-of-the-art methods. The competitive performance of camera pose estimation confirms the effectiveness of our method.

Computer vision tool for detection, mapping, and fault classification of photovoltaics modules in aerial IR videos

AbstractIncreasing deployment of photovoltaics (PV) plants demands for cheap and fast inspection. A viable tool for this task is thermographic imaging by unmanned aerial vehicles (UAV). In this work, we develop a computer vision tool for the semi‐automatic extraction of PV modules from thermographic UAV videos. We use it to curate a dataset containing 4.3 million IR images of 107,842 PV modules from thermographic videos of seven different PV plants. To demonstrate its use for automated PV plant inspection, we train a ResNet‐50 to classify ten common module anomalies with more than 90% test accuracy. Experiments show that our tool generalizes well to different PV plants. It successfully extracts PV modules from 512 out of 561 plant rows. Failures are mostly due to an inappropriate UAV trajectory and erroneous module segmentation. Including all manual steps our tool enables inspection of 3.5 MWp to 9 MWp of PV installations per day, potentially scaling to multi‐gigawatt plants due to its parallel nature. While we present an effective method for automated PV plant inspection, we are also confident that our approach helps to meet the growing demand for large thermographic datasets for machine learning tasks, such as power prediction or unsupervised defect identification.

Microelectrode Arrays: Flexible and Transparent Metal Nanowire Microelectrode Arrays and Interconnects for Electrophysiology, Optogenetics, and Optical Mapping (Adv. Mater. Technol. 7/2021)

In article number 2100225, Igor R. Efimov, Luyao Lu, and co-workers develop flexible and transparent metal nanowire-based microelectrode arrays (MEAs) and interconnects from a simple solution-processing approach. The MEAs enable precision co-localized electrical recording, optogenetic modulation, and optical mapping of cell activity. The devices are promising for large-area multifunctional interrogation of complex biological systems in basic and translational research.

TWC-Net: A SAR Ship Detection Using Two-Way Convolution and Multiscale Feature Mapping

Synthetic aperture radar (SAR) is an active earth observation system with a certain surface penetration capability and can be employed to observations all-day and all-weather. Ship detection using SAR is of great significance to maritime safety and port management. With the wide application of in-depth learning in ordinary images and good results, an increasing number of detection algorithms began entering the field of remote sensing images. SAR image has the characteristics of small targets, high noise, and sparse targets. Two-stage detection methods, such as faster regions with convolution neural network (Faster RCNN), have good results when applied to ship target detection based on the SAR graph, but their efficiency is low and their structure requires many computing resources, so they are not suitable for real-time detection. One-stage target detection methods, such as single shot multibox detector (SSD), make up for the shortage of the two-stage algorithm in speed but lack effective use of information from different layers, so it is not as good as the two-stage algorithm in small target detection. We propose the two-way convolution network (TWC-Net) based on a two-way convolution structure and use multiscale feature mapping to process SAR images. The two-way convolution module can effectively extract the feature from SAR images, and the multiscale mapping module can effectively process shallow and deep feature information. TWC-Net can avoid the loss of small target information during the feature extraction, while guaranteeing good perception of a large target by the deep feature map. We tested the performance of our proposed method using a common SAR ship dataset SSDD. The experimental results show that our proposed method has a higher recall rate and precision, and the F-Measure is 93.32%. It has smaller parameters and memory consumption than other methods and is superior to other methods.

Moving-mass-based station keeping of stratospheric airships

ABSTRACTStratospheric airships are lighter-than-air vehicles that work at an altitude of 20km in the lower calm portion of the stratosphere. They can be used as real-time surveillance platforms for environment monitoring and civil communication. Solar energy is the ideal power choice for long-endurance stratospheric airships. Attitude control is important for airships so that they can point at a target for observation or adjust the attitude to improve the output performance of solar panels. Stratospheric airships have a large volume and semi-flexible structure. The typical actuators used are aerodynamic surfaces, vectored thrust and ballonets. However, not all these actuators can work well under special working conditions, such as low density and low speed. In this study, moving-mass control is introduced to stratospheric airships because its control efficiency is independent of airspeed and atmospheric density. A nonlinear feedback controller based on generalised inverse with a nonlinear mapping module is designed to implement moving-mass control. Such a new station keeping scheme with moving masses is proposed for airships with different working situations.

Improving distortion correction for isotropic high-resolution 3D diffusion MRI by optimizing Jacobian modulation.

To improve distortion correction for isotropic high-resolution whole-brain 3D diffusion MRI when in a time-saving acquisition scenario. Data were acquired using simultaneous multi-slab (SMSlab) acquisitions, with a b = 0 image pair encoded by reversed polarity gradients (RPG) for phase encoding (PE) and diffusion weighted images encoded by a single PE direction. Eddy current-induced distortions were corrected first. During the following susceptibility distortion correction, image deformation was first corrected by the field map estimated from the b = 0 image pair. Intensity variation was subsequently corrected by Jacobian modulation. Two Jacobian modulation methods were compared. They calculated the Jacobian modulation map from the field map, or from the deformation corrected b = 0 image pair, termed as JField and JRPG , respectively. A modified version of the JRPG method, with proper smoothing, was further proposed for improved correction performance, termed as JRPG-smooth . Compared to JField modulation, less remaining distortions are observed when using the JRPG and JRPG-smooth methods, especially in areas with large B0 field inhomogeneity. The original JRPG method causes signal-to-noise ratio (SNR) deficiency problem, which manifests as degraded SNR of the diffusion weighted images, while the JRPG-smooth method maintains the original image SNR. Less estimation errors of diffusion metrics are observed when using the JRPG-smooth method. This study improves the distortion correction for isotropic high-resolution whole-brain 3D diffusion MRI by optimizing Jacobian modulation. The optimized method outperforms the conventional JField method regarding intensity variation correction and accuracy of diffusion metrics estimation, and outperforms the original JRPG method regarding SNR performance.

Increasing Timeliness of Satellite-Based Flood Mapping Using Early Warning Systems in the Copernicus Emergency Management Service

Remotely sensed images have become an important source of information for actors involved in disaster management and satellite-based emergency mapping (SEM) is increasingly used to support the response phase in the first hours and days after a disaster occurs. The delivery timeliness of the crisis information is key to the success of SEM. In the Copernicus Emergency Management Service (CEMS), a procedure was tested during the past 5 years which links the European Flood Awareness System (EFAS) with the on-demand Rapid Mapping module in order to anticipate satellite tasking in view of an upcoming activation of the service for riverine floods. This study aims at assessing if the procedure has helped to improve the delivery timeliness of the first flood impact information. For the assessment, we used the Rapid Mapping performance statistics recorded from June 2016 to December 2020. Standard Rapid Mapping activations for floods were compared with those preceded by an EFAS based pre-tasking request. The focus was on essential time stamps such as activation start, provision of imagery and the availability of derived information products. For the pre-tasking-related activations, we further compared the EFAS predictions with the Rapid Mapping user request and compared flood predictions with actual observations. Our results show that the EFAS based pre-tasking improves the timeliness of the first product delivery due to the fact that satellite images could be acquired earlier compared to activations without pre-tasking.

Soil Sample Assay Uncertainty and the Geographic Distribution of Contaminants: Error Impacts on Syracuse Trace Metal Soil Loading Analysis Results.

A research team collected 3609 useful soil samples across the city of Syracuse, NY; this data collection fieldwork occurred during the two consecutive summers (mid-May to mid-August) of 2003 and 2004. Each soil sample had fifteen heavy metals (As, Cr, Cu, Co, Fe, Hg, Mo, Mn, Ni, Pb, Rb, Se, Sr, Zn, and Zr), measured during its assaying; errors for these measurements are analyzed in this paper, with an objective of contributing to the geography of error literature. Geochemistry measurements are in milligrams of heavy metal per kilogram of soil, or ppm, together with accompanying analytical measurement errors. The purpose of this paper is to summarize and portray the geographic distribution of these selected heavy metals measurement errors across the city of Syracuse. Doing so both illustrates the value of the SAAR software’s uncertainty mapping module and uncovers heavy metal characteristics in the geographic distribution of Syracuse’s soil. In addition to uncertainty visualization portraying and indicating reliability information about heavy metal levels and their geographic patterns, SAAR also provides optimized map classifications of heavy metal levels based upon their uncertainty (utilizing the Sun-Wong separability criterion) as well as an optimality criterion that simultaneously accounts for heavy metal levels and their affiliated uncertainty. One major outcome is a summary and portrayal of the geographic distribution of As, Cr, Cu, Co, Fe, Hg, Mo, Mn, Ni, Pb, Rb, Se, Sr, Zn, and Zr measurement error across the city of Syracuse.

Smooth Surface Defect Detection by Deep Learning Based on Wrapped Phase Map

Object detection is widely used in the industry for visual inspection of defects. But it fails to fully reflect the defects on the smooth surface due to the difficulty in illuminating, and the high performance of defect detection tends to be a challenge. In this paper, we proposed a novel defect detection method for laptop panels. On the one hand, we utilized the wrapped phase map obtained by phase-shift reflection fringes for defect detection and used the DenseFuse network to fuse the wrapped phase map with the fringe modulation map to optimize the dataset. On the other hand, we proposed an improved deep learning-based network Vovecwnet. The experiments indicated that our image-based multi-information fusion system could achieve excellent performance in defect detection on the smooth surface (e.g., laptop panels surface), with high accuracy of 97.5% and a few reference time 0.0762s on GTX 2080ti GPU. Thus, our proposed method outperformed and tackled the challenges encountered in the defect detection of smooth surfaces.

Design and development of ground station for UAV/UGV heterogeneous collaborative system

Unmanned Aerial Vehicle/Unmanned Ground Vehicle heterogeneous collaborative system can complete complex tasks. In order to monitor and control the two agents simultaneously, a comprehensive ground station is designed. First, the preliminary work is introduced. Then, the architecture of the ground station of the system is proposed. In data communication module, since the amount of data from the system is large, the data of structure and multi-threaded processing are designed in detail. In data display module, the ground station provides various ways for displaying data. In image display module, in order to balance the accuracy and real-time performance of the system, an Efficient Convolution Operators tracking algorithm is improved for tracking moving target. In digital map module, a three-dimensional path planning method based on improved Ant Colony is proposed. The experiment results show that the ground station works well in real-time performance and accuracy.

Medical Image Encryption Based on 2D Zigzag Confusion and Dynamic Diffusion

For the security of medical image, a new algorithm of medical image encryption is designed. The novel algorithm is based on a chaotic system composed of the two-dimensional Sine Logistic modulation map (2D-SLMM) and the two-dimensional Hénon-Sine map (2D-HSM). The main encryption procedure includes zigzag scan scramble, pixel grey value transformation, and dynamic diffusion. On the pixel grey value transformation stage, a password feedback is added. This makes the relationship between password and key more complicated. The proposed scheme is lossless for medical image encryption and decryption. It avoids the problems of low-dimensional chaotic map such as narrow interval and few parameters, as well as the problem of the special texture and contour of medical images. The key space of the novel algorithm is big enough, and the encryption and decryption processing are sensitive to the key. Simulation and experiments validate the effectiveness and efficiency of the novel algorithm. Security analysis proves the algorithm is resistant to common attacks.

SPARC: Development of a Transgenic Pig Expressing mCherry and CRE Recombinase Fusion Proteins Under the Control of the Murine Choline Acetyltransferase Promoter

Pigs can serve as useful models for advancing the biomedical field due to anatomy and physiology that more closely resembles human anatomy and physiology compared to traditional rodent models. However, there is a paucity of porcine CRE-driver lines available, limiting their utility for refined neural mapping and neural modulation studies. Here we describe the development of pigs with mCherry and CRE recombinase proteins expressed in cholinergic neurons. A 6,419 bp DNA fragment containing the Mus musculus choline acetyltransferase (ChAT) promoter was used to drive production of a mCherry–P2A–CRE recombinase fusion protein (mmChAT–mCherry–P2A–CRE). Linearized mmChAT–mCherry–P2A–CRE plasmid DNA was introduced into male porcine fetal fibroblasts by electroporation. Positive colonies were selected with 400µg/mL G-418 and screened for transgene insertion by PCR. Two colonies showing targeted insertion were used for somatic cell nuclear transfer and the resulting blastocysts were transferred to surrogate gilts. Two early pregnancies were established, and one of them went to term. Six Duroc-Landrace male piglets were born by natural delivery. PCR analysis of genomic DNA showed that one piglet carried the transgene whereas the others were determined to be wild-type. The transgenic piglet was phenotypically normal with no obvious difference in appearance compared to the wild-type littermates. The piglet was euthanized at 7-weeks of age for further characterization. Major tissues known to have cholinergic cell bodies and cholinergic innervation (hippocampus, cerebellum, spinal cord, basal ganglion, dorsal root ganglion, nodose ganglion, vagus nerve, trachea, bronchus, heart, bladder, duodenum and pancreas) were evaluated regarding mCherry and CRE recombinase mRNA presence. End-point PCR revealed CRE recombinase mRNA in basal ganglion, nodose ganglion, dorsal root ganglion and heart. mCherry mRNA was found in bronchus homogenates. Studies are ongoing to confirm integration sites into the genome, and mCherry and CRE recombinase expression at the protein level in cholinergic neurons. If successful, this model will allow for simultaneous dissection of cholinergic circuit connectivity and control of neural activity through CRE-mediated recombination.

Securing multiple-single-channel color image using unequal spectrum decomposition and 2D-SLIM biometric keys

A novel multiple-single-channel color image cryptosystem based on unequal spectrum decomposition (USD) and 2D sine improved 1ogistic iterative chaotic map with infinite collapse modulation map (2D-SLIM) is proposed. In this method, R, G, and B channels of each color image of an authorized user is fused with corresponding LL sub-band of a gray-scale carrier image by inverse discrete wavelet transform (DWT) to obtain single-channel and watermarked image. The individual biometric key of an authorized user is produced by his/her phase-encoded irisprint and then modulated by chaotic random phase mask (CRPM). So the parameters of CRPM are used as decryption keys with the uniqueness of the irisprint. The watermarked image as input image is normalized, multiplied with biometric key, and then Fresnel transformed. The Fresnel spectrum is divided into two complex-value masks by using unequal spectrum decomposition in which one phase mask is exploited as decryption key and other as ciphertext. The final ciphertext is obtained by adding individual ciphertexts. The encryption and decryption process can be realized with a hybrid optoelectronic system. Numerical simulations have been performed to verify the validity and feasibility of the proposed system.

Novel Approaches in Cardiac Imaging for Non-invasive Assessment of Left Heart Myocardial Fibrosis.

In the past, the identification of myocardial fibrosis was only possible through invasive histologic assessment. Although endomyocardial biopsy remains the gold standard, recent advances in cardiac imaging techniques have enabled non-invasive tissue characterization of the myocardium, which has also provided valuable insights into specific disease processes. The diagnostic accuracy, incremental yield and prognostic value of speckle tracking echocardiography, late gadolinium enhancement and parametric mapping modules by cardiac magnetic resonance and cardiac computed tomography have been validated against tissue samples and tested in broad patient populations, overall providing relevant clinical information to the cardiologist. This review describes the patterns of left ventricular and left atrial fibrosis, and their characterization by advanced echocardiography, cardiac magnetic resonance and cardiac computed tomography, allowing for clinical applications in sudden cardiac death and management of atrial fibrillation.

Depth segmentation using disparity and self-adaption fringe modulation analysis

In this paper, we propose a depth segmentation method combining disparity information with fringe modulation information, which aims at improving the robustness under the noisy scene. Firstly, we investigate the continuity characteristics in the U-V disparity map and distinguishing modulation distribution in the wrapped phase images, respectively. Then, the disparity-based method is optimized to detect and classify the valid regions containing objects and their supporting surfaces. Besides, we provide an evaluation criterion to reduce the noise represented as scattered points. Furthermore, through the analysis of the different fringe modulation distributions between objects and their shadow areas, a method with adjustable thresholds is presented to suppress shadow noise. Our approach is evaluated on both simulation scenes and real-world scenes to verify the availability and robustness. Sufficient experimental results indicate that the proposed method can effectively separate the valid region into objects and detect their incomplete supporting surfaces.

Stacked Pyramid Attention Network for Object Detection

Scale variation is one of the primary challenges in object detection. Recently, different strategies have been introduced to address this challenge, achieving promising performance. However, limitations still exist in these detectors. On the one hand, as for the large-scale deep layers, the localizing power of the features is relatively low. On the other hand, as for the small-scale shallow layers, the categorizing ability of the features is relatively weak. Actually, the limitations are self-solving, as the above two aspects can be mutually beneficial to each other. Therefore, we propose the Stacked Pyramid Attention Network (SPANet) to bridge the gap between different scales. In SPANet, two lightweight modules, i.e. top-down feature map attention module (TDFAM) and bottom-up feature map attention module (BUFAM), are designed. Via learning the channel attention and spatial attention, each module effectively builds connections between features from adjacent scales. By progressively integrating BUFAM and TDFAM into two encoder–decoder structures, two novel feature aggregating branches are built. In this way, the branches fully complement the localizing power from small-scale features and the categorizing power from large-scale features, therefore enhancing the detection accuracy while keeping lightweight. Extensive experiments on two challenging benchmarks (PASCAL VOC and MS COCO datasets) demonstrate the effectiveness of our SPANet, showing that our model reaches a competitive trade-off between accuracy and speed.

The linkage of cell cycle and DNA replication with growth difference in female Chinese tongue sole (Cynoglossus semilaevis): Analysis from transcriptomic study and WGCNA

In addition to the typical sexual size dimorphism, considerable size differences within the female population of the Chinese tongue sole (Cynoglossus semilaevis) have become a further bottleneck of the improvement of sole aquaculture. To identify the internal mechanism, transcriptomic analysis and weighted gene co-expression network analysis (WGCNA) were employed simultaneously. Transcriptomic analyses of brain, pituitary gland, liver, gonad, and muscle tissues from two female groups with size differences identified 109, 698, 1325, 2299, and 2141 differentially expressed genes (DEGs), respectively. The results of these enrichment analyses suggest that the up-regulation of neuroactive ligand-receptor interaction, cell cycle, DNA replication, and MAPK signaling pathway in the group with larger females may be involved in the regulation of the observed growth differences. WGCNA of DEGs showed that cell cycle and DNA replication might be crucial pathways for accelerating cell growth in the groups with larger females. Finally, a series of hub genes including 6-phosphofructokinase type C (pfkp), ribosome biogenesis protein (wdr12), bleomycin hydrolase (blmh), and semaphorin-3A (sema3a) were recognized by the illustrated network map of modules. The linkage of cell cycle, DNA replication, and hub genes in the growth regulation of C. semilaevis provides further information for a better understanding of growth differences in fish.

R-LOAM: Improving LiDAR Odometry and Mapping With Point-to-Mesh Features of a Known 3D Reference Object

LiDAR-based odometry and mapping is used in many robotic applications to retrieve the robot's position in an unknown environment and allows for autonomous operation in GPS-denied (e.g., indoor) environments. With a 3D LiDAR sensor, highly accurate localization becomes possible, which enables high quality 3D reconstruction of the environment. In this letter we extend the well-known LOAM framework by leveraging prior knowledge about a reference object in the environment to further improve the localization accuracy. This requires a known 3D model of the reference object and its known position in a global coordinate frame. Instead of only relying on the point features in the mapping module of LOAM, we also include mesh features extracted from the 3D triangular mesh of the reference object in the optimization problem. For fast correspondence computation of mesh features, we use the Axis-Aligned-Bounding-Box-Tree (AABB) structure. Essentially, our approach not only makes use of the previously built map for absolute localization in the environment, but also takes the relative position to the reference object into account, effectively reducing long-term drift. To validate the proposed concept, we generated datasets using the Gazebo simulation environment in exemplary visual inspection scenarios of an airplane inside a hangar and the Eiffel Tower. An actuated 3D LiDAR sensor is mounted via a 1-DoF gimbal on a UAV capturing 360° scans. We benchmark our approach against the state-of-the-art open-source LOAM framework. The results show that the proposed joint optimization using both point and mesh features yields a significant reduction in Absolute Pose Error (APE) and therefore improves the map and 3D reconstruction quality during long-term operations.