High-resolution Details Research Articles

Laser curve extraction of a train wheelset based on an encoder-decoder network.

An algorithm of laser curve segmentation for a train wheelset based on an encoder- decoder network is proposed. Aiming at the rich local features and simple semantic features of the train wheelset laser curve image, a neural network with shallow depth, high resolution, and good detail performance was designed. The proposed neural network makes full use of the dense connection mechanism and the upsampling module to enhance feature reuse and feature propagation. It can extract context semantic information at multiple scales with fewer parameters. Experimental results show that the encoder-decoder network has better performance than other neural networks in laser curve extraction of train wheelset. Based on the encoder-decoder neural network, mIOU, Recall, Accuracy, and F1_score of the laser curve dataset of the train wheelset, the score index reached 86.5%, 89.2%, 99.9%, and 85.0%, which can accurately extract the laser stripe of the train wheelset. Additionally, the encoder-decoder network can diminish the influence of noise on the extraction of laser fringes of a train wheelset to a certain extent. Therefore, it has good application in railway safety.

Symptoms: Sudden Hearing Loss and Facial Paralysis

Symptoms: Sudden Hearing Loss and Facial Paralysis

Optimized cryo-EM data-acquisition workflow by sample-thickness determination.

Sample thickness is a known key parameter in cryo-electron microscopy (cryo-EM) and can affect the amount of high-resolution information retained in the image. Yet, common data-acquisition approaches in single-particle cryo-EM do not take it into account. Here, it is demonstrated how the sample thickness can be determined before data acquisition, allowing the identification of optimal regions and the restriction of automated data collection to images with preserved high-resolution details. This quality-over-quantity approach almost entirely eliminates the time- and storage-consuming collection of suboptimal images, which are discarded after a recorded session or during early image processing due to a lack of high-resolution information. It maximizes the data-collection efficiency and lowers the electron-microscopy time required per data set. This strategy is especially useful if the speed of data collection is restricted by the microscope hardware and software, or if microscope access time, data transfer, data storage and computational power are a bottleneck.

Spatio-Temporal Downscaling of Climate Data Using Convolutional and Error-Predicting Neural Networks

Numerical weather and climate simulations nowadays produce terabytes of data, and the data volume continues to increase rapidly since an increase in resolution greatly benefits the simulation of weather and climate. In practice, however, data is often available at lower resolution only, for which there are many practical reasons, such as data coarsening to meet memory constraints, limited computational resources, favoring multiple low-resolution ensemble simulations over few high-resolution simulations, as well as limits of sensing instruments in observations. In order to enable a more insightful analysis, we investigate the capabilities of neural networks to reconstruct high-resolution data from given low-resolution simulations. For this, we phrase the data reconstruction as a super-resolution problem from multiple data sources, tailored toward meteorological and climatological data. We therefore investigate supervised machine learning using multiple deep convolutional neural network architectures to test the limits of data reconstruction for various spatial and temporal resolutions, low-frequent and high-frequent input data, and the generalization to numerical and observed data. Once such downscaling networks are trained, they serve two purposes: First, legacy low-resolution simulations can be downscaled to reconstruct high-resolution detail. Second, past observations that have been taken at lower resolutions can be increased to higher resolutions, opening new analysis possibilities. For the downscaling of high-frequent fields like precipitation, we show that error-predicting networks are far less suitable than deconvolutional neural networks due to the poor learning performance. We demonstrate that deep convolutional downscaling has the potential to become a building block of modern weather and climate analysis in both research and operational forecasting, and show that the ideal choice of the network architecture depends on the type of data to predict, i.e., there is no single best architecture for all variables.

Structure from Motion and Archaeological Excavation: Experiences of the Via Consolare Project in Pompeii

The last decade of advances in Image-Based Modeling (IBM) data acquisition based on Structure from Motion (SfM) have made it possible as never before to record excavated archaeological deposits, historical architectural remains, artifacts, and geographical surroundings in the field. Armed only with digital cameras and low-cost or open-source software, researchers can now produce accurate point clouds of millions of points, capturing archaeological information in high-resolution detail. But what changes will IBM really bring to the standards, requirements, and expectations of practical field methodology for projects operating on shoe-string budgets? Since 2010, the Via Consolare Project, a small archaeological research project from a State level University, has employed an entirely open-source and “free for academic use” IBM pipeline to record a variety of archaeological features in Insula VII 6 and the “Villa delle Colonne a mosaico” in Pompeii. Ranging from surviving architecture, to rubble fill layers, to the interiors of inaccessible cisterns and drains, this work has been carried out in preparation for the eventual coordination of these data into a 3D GIS of all recorded stratigraphy. Rarely were sufficient resources available for dedicated equipment or personnel to be devoted to this task. While practical implementation, even in a low-budget excavation environment, has confirmed that this technology can indeed augment archaeological field documentation and provide investigation opportunities that would otherwise be impossible, it failed to replace traditional handdrafted recording techniques and was found to present significant challenges and a number of hidden costs. This emphasizes a need for appropriate and cautious planning in implementation, especially in projects with limited means.

Infrared and visible image fusion based on edge-preserving guided filter and infrared feature decomposition

Infrared (IR) images can distinguish salient targets from their backgrounds based on the radiation difference in all-weather conditions. By contrast, visible (VIS) images can contain high-resolution texture and details information, which is more suitable for human observation. Therefore, it is quite necessary to combine both imaging advantages of these two kinds of images. Compared with the existing methods, we believe that scale decomposition based methods are the most active and efficient image fusion methods, which also have the best fusion effects. Inspired by the present scale decomposition methods, we propose a new feature decomposition method. Firstly, we propose an improved guided filter called edge-preserving guided filter (EPGF), which adopts the image gradient map for further improving the filtering effect. Subsequently, by using the EPGF, we decompose the IR and VIS images into three kinds of layers, including salient feature layers, luminance layers and detail layers. At the same time, we combine all the layers together to get an initial fusion result. Finally, we optimize the initial fusion image according to a new image fusion optimization model and ADMM, and the final fusion result will be obtained after several iterations. Compared with other scale decomposition methods, our proposed feature decomposition based method takes the IR salient targets, IR and VIS background illumination, as well as VIS details into consideration which is more in line with human visual observation, besides the computational efficiency is also superior. Experimental results indicate that this method has better subjective and objective evaluation results compared with other state-of-the-art methods.

Small-Scale Condensation Simulation on Complicated Surface

<p indent=0mm>Vapor condensation is a typical phenomenon in the nature, whose realistic simulation is of great significance to virtual reality, video effects, game entertainment, etc. Traditional physically-based methods failed to preserve the small-scale details during condensation. A novel hybrid framework combining smoothed particle hydrodynamics and adaptive fluid implicit particle method is proposed. The heat transfer of air is modeled with smoothed particle hydrodynamics, with phase transition controlled by relative humidity and dew point. The fluid implicit particle method is combined with an octree-based adaptive grid to achieve high-resolution details of small-scale drops. Finally, surface tension, adhesion and resistance are introduced to simulate the motion of drops on complicated surfaces realistically. It is demonstrated that proposed method provides plausible results of condensation on intricate surfaces efficiently through several experiments.

An overview of the recent advances in cryo-electron microscopy for life sciences.

Cryo-electron microscopy (CryoEM) has superseded X-ray crystallography and NMR to emerge as a popular and effective tool for structure determination in recent times. It has become indispensable for the characterization of large macromolecular assemblies, membrane proteins, or samples that are limited, conformationally heterogeneous, and recalcitrant to crystallization. Besides, it is the only tool capable of elucidating high-resolution structures of macromolecules and biological assemblies in situ. A state-of-the-art electron microscope operable at cryo-temperature helps preserve high-resolution details of the biological sample. The structures can be determined, either in isolation via single-particle analysis (SPA) or helical reconstruction, electron diffraction (ED) or within the cellular environment via cryo-electron tomography (cryoET). All the three streams of SPA, ED, and cryoET (along with subtomogram averaging) have undergone significant advancements in recent times. This has resulted in breaking the boundaries with respect to both the size of the macromolecules/assemblies whose structures could be determined along with the visualization of atomic details at resolutions unprecedented for cryoEM. In addition, the collection of larger datasets combined with the ability to sort and process multiple conformational states from the same sample are providing the much-needed link between the protein structures and their functions. In overview, these developments are helping scientists decipher the molecular mechanism of critical cellular processes, solve structures of macromolecules that were challenging targets for structure determination until now, propelling forward the fields of biology and biomedicine. Here, we summarize recent advances and key contributions of the three cryo-electron microscopy streams of SPA, ED, and cryoET.

Efficient Parallel Branch Network With Multi-Scale Feature Fusion for Real-Time Overhead Power Line Segmentation

Image-based segmentation of overhead power lines is critical for power line inspection. Real-time segmentation helps the inspection robot avoid obstacles or land on the wire during the inspection task. It is challenging for several studies to achieve real-time overhead power line segmentation with high accuracy. In addition, cluttered background brings great difficulties to overhead power lines segmentation. To address these issues, an efficient parallel branch network for real-time overhead power line segmentation is proposed. Our framework combines a context branch that generates useful global information with a spatial branch that preserves high-resolution segmentation details. The asymmetric factorized depth-wise bottleneck (AFDB) module is designed in the context branch to achieve more efficient short-range feature extraction and provide a large receptive field. Furthermore, the subnetwork-level skip connections in the classifier are proposed to fuse long-range features and lead to high accuracy. Experiments demonstrate that our framework achieves more than 90% segmentation accuracy.

A mass spectrometry-based glycotope-centric cellular glycomics is the more fruitful way forward to see the forest for the trees.

The nature of protein glycosylation renders cellular glycomics a very challenging task in having to deal with all the disparate glycans carried on membrane glycoproteins. Rapid mapping by mass spectrometry analysis provides only a coarse sketch of the glycomic complexity based primarily on glycosyl compositions, whereby the missing high-resolution structural details require a combination of multi-mode separations and multi-stages of induced fragmentation to gain sufficiently discriminative precision, often at the expenses of throughput and sensitivity. Given the available technology and foreseeable advances in the near future, homing in on resolving the terminal fucosylated, sialylated and/or sulfated structural units, or glycotopes, maybe a more pragmatic and ultimately more rewarding approach to gain insights into myriad biological processes mediated by these terminal coding units carried on important glycoproteins, to be decoded by a host of endogenous glycan-binding proteins and antibodies. A broad overview of recent technical advances and limitations in cellular glycomics is first provided as a backdrop to the propounded glycotope-centric approach based on advanced nanoLC-MS2/MS3 analysis of permethylated glycans. To prioritize analytical focus on the more tangible glycotopes is akin to first identifying the eye-catching and characteristic-defining flowers and fruits of the glyco-forest, to see the forest for the trees. It has the best prospects of attaining the much-needed balance in sensitivity, structural precision and analytical throughput to match advances in other omics.

Development and Application of the Sample Support in Transmission Electron Microscopy

Cryogenic electron microscopy is becoming an increasingly popular technology that allows us to observe the molecules in high-resolution details. With this refined way to study the structures in scientific researches, the observation of the molecules will be expected to make improvements. The conventional methods, such as the X-ray crystallography and the nuclear magnetic resonance spectroscopy, provide the basic observation of ground-state molecules and the measurement of various small molecules. But both of them contain flaws discovered in the continuing development process. Therefore, the application of cryo-EM becomes a broader area to explore. In this review, we will give an overall understanding of the cryogenic electron microscopy. We will discuss the intention behind the invention of the cryo-EM and introduce the main workflow of the experiments. In addition, we will discuss the methods to prepare the sample in a well-condition and compare them among different groups of experiments. More importantly, we will discuss the application of the graphene in cryo-EM and evaluate possible improvements in future development.

The Use of Artificial Intelligence‐Based Optical Remote Sensing and Positioning Technology in Microelectronic Processing Technology

The purpose is to make defect detection in microelectronic processing technology fast, accurate, reliable, and efficient. A new optical remote sensing‐optical beam induced resistance change (ORS‐OBIRCH) target recognition and location defect detection method is proposed based on an artificial intelligence algorithm, optical remote sensing (ORS), and optical beam induced resistance change (OBIRCH) location technology using deep convolutional neural network. This method integrates the characteristics of high resolution and rich details of the image obtained by ORS technology and combines the advantages of photosensitive temperature characteristics in OBIRCH positioning technology. It can be adopted to identify, capture, and locate the defects of microdevices in the process of microelectronic processing. Simulation results show that this method can quickly reduce the detection range and locate defects accurately and efficiently. The experimental results reveal that the ORS‐OBIRCH target recognition defect location detection method can complete the dynamic synchronization of the IC detection system and obtain high‐quality images by changing the laser beam irradiation cycle. Moreover, it can analyze and process the detection results to quickly, accurately, and efficiently locate the defect location. Unlike the traditional detection methods, the success rate of detection has been greatly improved, which is about 95.8%, an increase of nearly 40%; the detection time has been reduced by more than half, from 5.5 days to 1.9 days, and the improvement rate has reached more than 65%. In a word, this method has good practical application value in the field of microelectronic processing.

Identification and functional characterization of the Piezo1 channel pore domain

Mechanotransduction is the process by which cells convert physical forces into electrochemical responses. On a molecular scale, these forces are detected by mechanically activated ion channels, which constitute the basis for hearing, touch, pain, cold, and heat sensation, among other physiological processes. Exciting high-resolution structural details of these channels are currently emerging that will eventually allow us to delineate the molecular determinants of gating and ion permeation. However, our structural–functional understanding across the family remains limited. Piezo1 is one of the largest and least understood of these channels, with various structurally identified features within its trimeric assembly. This study seeks to determine the modularity and function of Piezo1 channels by constructing deletion proteins guided by cryo EM structural knowledge. Our comprehensive functional study identified, for the first time, the minimal amino acid sequence of the full-length Piezo1 that can fold and function as the channel’s pore domain between E2172 and the last residue E2547. While the addition of an anchor region has no effect on permeation properties. The Piezo1 pore domain is not pressure-sensitive and the appending of Piezo Repeat-A did not restore pressure-dependent gating, hence the sensing module must exist between residues 1 to 1952. Our efforts delineating the permeation and gating regions within this complex ion channel have implications in identifying small molecules that exclusively regulate the activity of the channel’s pore module to influence mechanotransduction and downstream processes.

Crystal structures capture multiple stoichiometric states of an aqueous self-assembling oligourea foldamer

We describe here an oligourea foldamer able to self-assemble in aqueous conditions into helix bundles of multiple stoichiometries, with accompanying crystal structures providing high-resolution details of the structural polymorphism of this foldamer.

Realistic Speech-Driven Talking Video Generation with Personalized Pose

In this work, we propose a method to transform a speaker’s speech information into a target character’s talking video; the method could make the mouth shape synchronization, expression, and body posture more realistic in the synthesized speaker video. This is a challenging task because changes of mouth shape and posture are coupled with audio semantic information. The model training is difficult to converge, and the model effect is unstable in complex scenes. Existing speech-driven speaker methods cannot solve this problem well. The method proposed in this paper first generates the sequence of key points of the speaker’s face and body postures from the audio signal in real time and then visualizes these key points as a series of two-dimensional skeleton images. Subsequently, we generate the final real speaker video through the video generation network. We take a random sampling of audio clips, encode audio contents and temporal correlations using a more effective network structure, and optimize and iterate network outputs using differential loss and attitude perception loss, so as to obtain a smoother pose key-point sequence and better performance. In addition, by inserting a specified action frame into the synthesized human pose sequence window, action poses of the synthesized speaker are enriched, making the synthesis effect more realistic and natural. Then, the final speaker video is generated by the obtained gesture key points through the video generation network. In order to generate realistic and high-resolution pose detail videos, we insert a local attention mechanism into the key point network of the generated pose sequence and give higher attention to the local details of the characters through spatial weight masks. In order to verify the effectiveness of the proposed method, we used the objective evaluation index NME and user subjective evaluation methods, respectively. Experiment results showed that our method could vividly use audio contentsto generate corresponding speaker videos, and its lip-matching accuracy and expression postures are better than those of previous work. Compared with existing methods in the NME index and user subjective evaluation, our method showed better results.

Cell Volume (3D) Correlative Microscopy Facilitated by Intracellular Fluorescent Nanodiamonds as Multi-Modal Probes.

Three-dimensional correlative light and electron microscopy (3D CLEM) is attaining popularity as a potential technique to explore the functional aspects of a cell together with high-resolution ultrastructural details across the cell volume. To perform such a 3D CLEM experiment, there is an imperative requirement for multi-modal probes that are both fluorescent and electron-dense. These multi-modal probes will serve as landmarks in matching up the large full cell volume datasets acquired by different imaging modalities. Fluorescent nanodiamonds (FNDs) are a unique nanosized, fluorescent, and electron-dense material from the nanocarbon family. We hereby propose a novel and straightforward method for executing 3D CLEM using FNDs as multi-modal landmarks. We demonstrate that FND is biocompatible and is easily identified both in living cell fluorescence imaging and in serial block-face scanning electron microscopy (SB-EM). We illustrate the method by registering multi-modal datasets.

Novel diffusion-based MRI approaches reveal high resolution details of aortic vessel wall architecture with advanced inflammatory atherosclerosis

Background and Aims: Atherosclerosis is the initiated uptake of cholesterol, mainly in an esterified form cholesteryl ester (CE) present in plasma low density lipoprotein LDL. Its initial accumulation in the intima leads to disease of the entire vessel wall and damage to the structural cells and the extracellular matrix (ECM), which regulate the mechanical properties of the vessel wall. During atherosclerosis progression, the damaged the smooth muscle cells acquire inflammatory features, with detrimental effects on vessel wall structure and function. This study applied diffusion-based MRI to investigate new linkages between the plaque lipids and vascular-tissue structural properties.

Hybrid real- and reciprocal-space full-field imaging with coherent illumination

We present a novel diffractive imaging method that harnesses a low-resolution real-space image to guide the phase retrieval. A computational algorithm is developed to utilise such prior knowledge as a real-space constraint in the iterative phase retrieval procedure. Numerical simulations and proof-of-concept experiments are carried out, demonstrating our method’s capability of reconstructing high-resolution details that are otherwise inaccessible with traditional phasing algorithms. With the present method, we formulate a conceptual design for the coherent imaging experiments at a next-generation x-ray light source.

Enhancing feature fusion for human pose estimation

Current human pose estimation methods mainly rely on designing efficient Convolutional Neural Networks (CNN) frameworks. These CNN architectures typically consist of high-to-low resolution sub-networks to learn semantic information, and then followed by low-to-high sub-networks to raise the resolution to locate the keypoints. Because low-level features have high resolution but less semantic information, while high-level features have rich semantic information but less high resolution details, so it is important to fuse different level features to improve the final performance. However, most existing models implement feature fusion by simply concatenate low-level and high-level features without considering the gap between spatial resolution and semantic levels. In this paper, we propose a new feature fusion method for human pose estimation. We introduce high level semantic information into low-level features to enhance feature fusion. Further, to keep both the high-level semantic information and high-resolution location details, we use Global Convolutional Network blocks to bridge the gap between low-level and high-level features. Experiments on MPII and LSP human pose estimation datasets demonstrate that efficient feature fusion can significantly improve the performance. The code is available at: https://github.com/tongjiangwei/FeatureFusion .

Structural basis of redox modulation on chloroplast ATP synthase

In higher plants, chloroplast ATP synthase has a unique redox switch on its γ subunit that modulates enzyme activity to limit ATP hydrolysis at night. To understand the molecular details of the redox modulation, we used single-particle cryo-EM to determine the structures of spinach chloroplast ATP synthase in both reduced and oxidized states. The disulfide linkage of the oxidized γ subunit introduces a torsional constraint to stabilize the two β hairpin structures. Once reduced, free cysteines alleviate this constraint, resulting in a concerted motion of the enzyme complex and a smooth transition between rotary states to facilitate the ATP synthesis. We added an uncompetitive inhibitor, tentoxin, in the reduced sample to limit the flexibility of the enzyme and obtained high-resolution details. Our cryo-EM structures provide mechanistic insight into the redox modulation of the energy regulation activity of chloroplast ATP synthase.