Extract High-level Information Research Articles

Fearless steps Apollo: Advancements in robust speech technologies and naturalistic corpus development from Earth to the Moon

Recent developments in deep learning strategies have revolutionized Speech and Language Technologies(SLT). Deep learning models often rely on massive naturalistic datasets to produce the necessary complexity required for generating superior performance. However, most massive SLT datasets are not publicly available, limiting the potential for academic research. Through this work, we showcase the CRSS-UTDallas led efforts to recover, digitize, and openly distribute over 50,000 hrs of speech data recorded during the 12 NASA Apollo manned missions, and outline our continuing efforts to digitize and create meta-data through diarization of the remaining 100,000hrs. We present novel deep learning-based speech processing solutions developed to extract high-level information from this massive dataset. Fearless-Steps APOLLO resource is a 50,000 hrs audio collection from 30-track analog tapes originally used to document Apollo missions 1,7,8,10,11,&13. A customized tape read-head developed to digitize all 30 channels simultaneously has been deployed to expedite digitization of remaining mission tapes. Diarized transcripts for these unlabeled audio communications have also been generated to facilitate open research from speech sciences, historical archives, education, and speech technology communities. Robust technologies developed to generate human-readable transcripts include: (i) speaker diarization, (ii) speaker tracking, and (iii) text output from speech recognition systems.

Object Detection Based on Adaptive Feature-Aware Method in Optical Remote Sensing Images

Object detection is used widely in remote sensing image interpretation. Although most models used for object detection have achieved high detection accuracy, computational complexity and low detection speeds limit their application in real-time detection tasks. This study developed an adaptive feature-aware method of object detection in remote sensing images based on the single-shot detector architecture called adaptive feature-aware detector (AFADet). Self-attention is used to extract high-level semantic information derived from deep feature maps for spatial localization of objects and the model is improved in localizing objects. The adaptive feature-aware module is used to perform adaptive cross-scale depth fusion of different-scale feature maps to improve the learning ability of the model and reduce the influence of complex backgrounds in remote sensing images. The focal loss is used during training to address the positive and negative sample imbalance problem, reduce the influence of the loss value dominated by easily classified samples, and enhance the stability of model training. Experiments are conducted on three object detection datasets, and the results are compared with those of the classical and recent object detection algorithms. The mean average precision(mAP) values are 66.12%, 95.54%, and 86.44% for three datasets, which suggests that AFADet can detect remote sensing images in real-time with high accuracy and can effectively balance detection accuracy and speed.

An Image Quality Adjustment Framework for Object Detection on Embedded Cameras

Automatic analysis tools are ubiquitously applied on wireless embedded cameras to extract high-level information from raw data. The quality of images may be degraded by factors such as noise and blur introduced during the sensing process, which could affect the performance of automatic analysis. Object detection is the first and the most fundamental step for the automatic analysis of visual information. This paper introduces a quality adjustment framework to provide satisfactory object detection performance on wireless embedded cameras. Key components of the framework include a blind regression model for predicting the performance of object detection and two distortion type classifiers for determining the presence of noise and blur in an image. Experimental results show that the proposed framework achieves accurate estimations of image distortion types, and it can be easily applied on embedded cameras with low computational complexity to improve the quality of captured images.

Context Residual Attention Network for Remote Sensing Scene Classification

Remote sensing image scene classification has attracted much attention due to its wide application. In this paper, a new end-to-end attentive context network under the guidance of human visual system has been proposed. The network can focus on some critical regions of the image selectively and then extract high-level feature information so as to generalize the whole image. The contributions of this letter are as follows: firstly, a novel attention structure which expresses the attention of image by stacking attention modules is designed; secondly, the context information of the image is utilized to make a holistic analysis of the attention mechanism on the basis of the top-down feedforward structure. Experiments performed on several datasets demonstrate that the proposed framework can obtain outstanding performance compared with state-of-the-art approaches.

Automated newborn cry diagnostic system using machine learning approach

Researchers have found that crying is an acoustic symptom among unhealthy newborns. This study aims to develop a non-invasive newborn cry diagnostic system (NCDS) using information at different levels of the cry audio signal (CAS) of infants. The unhealthy newborn group in our experiment consists of 34 clinical cases. The proposed machine learning (ML) techniques include the extraction of feature sets of Mel frequency cepstral coefficients (MFCC), auditory-inspired amplitude modulation (AAM) features, and a prosody feature set of tilt, intensity, and rhythm features. The training models are probabilistic neural networks and support vector machine algorithms. The feature sets of AAM and MFCC extract low-level patterns, whereas the prosody feature set of tilt, intensity, and rhythm extracts high-level information in an infant CAS. The AAM feature set in the NCDS has never yet been examined. As an innovative aspect of this study, the AAM feature set is included in the NCDS, and this feature set is fused with the feature sets of MFCC and prosody. As another innovation, we reproduce real-world problems by including many pathologies in the unhealthy group. Among the evaluated frameworks proposed, the fusion of all feature sets improves the system performance. The best result is obtained with the fusion of AAM and MFCC with an F-measure of over 80%. The results of this experiment reveal the usefulness of information at different levels within the CASs of newborns, which vary among healthy and unhealthy groups. Moreover, to identify unhealthy newborns, this information can be captured noninvasively by applying ML methods to the NCDS.

Extracting datums to reconstruct CSG models from 2D engineering sketches of polyhedral shapes

Our goal is to automatically generate CAD 3D models from 2D sketches as part of a design chain where models should be procedural, containing features arranged in a model tree and linked to suitable datums. Current procedural models capture much about the design intent and are easy to edit, but must be created from scratch during the detailed design state—given conceptual sketches as used by designers in the early part of the design process, current sketch-based modeling approaches only output explicit models. Thus, we describe an approach to extract high-level information directly from 2D engineering wireframe sketches and use it to complete a CSG feature tree, which serves as a model tree for a procedural 3D CAD model.Our method extracts procedural model information directly from 2D sketches in the form of a set of features, plus a set of datums and relationships between these features. We detect and analyze features of 2D sketches in isolation, and define the CSG feature tree by the parent–child relationships between features, and combine this information to obtain a complete and consistent CSG feature tree that can be transferred to a 3D modeler, which reconstructs the model. This paper focuses on how to extract the feature datums and the extrusion operation from an input 2D sketch.

Research Progress of Ultrasound Radiomics in Predicting Axillary Lymph Node Metastasis of Breast Cancer

Breast cancer, as the most common malignant tumor in women worldwide, has become the focus of global attention. Axillary lymph node tumor burden is an important prognostic indicator of it. Ultrasound is the most commonly used imaging method, but its sensitivity is not high, especially for the diagnosis of small and micro lymph node metastasis. In recent years, the emerging Radiomics in machine learning field has been widely used in the field of medical imaging. Because it can extract high-level information of images that is difficult to be recognized by human eyes, it has been used to establish clinical prediction models. This paper introduced the value of preoperative sonography, sketched Radiomics, and summarized the research progress of this method in predicting lymph node metastasis of breast cancer. This new method is expected to provide a reliable basis for individualized and accurate diagnosis and treatment of breast cancer.

Android Malware Detection via Graph Representation Learning

With the widespread usage of Android smartphones in our daily lives, the Android platform has become an attractive target for malware authors. There is an urgent need for developing an automatic malware detection approach to prevent the spread of malware. The low code coverage and poor efficiency of the dynamic analysis limit the large-scale deployment of malware detection methods based on dynamic features. Therefore, researchers have proposed a plethora of detection approaches based on abundant static features to provide efficient malware detection. This paper explores the direction of Android malware detection based on graph representation learning. Without complex feature graph construction, we propose a new Android malware detection approach based on lightweight static analysis via the graph neural network (GNN). Instead of directly extracting Application Programming Interface (API) call information, we further analyze the source code of Android applications to extract high-level semantic information, which increases the barrier of evading detection. Particularly, we construct approximate call graphs from function invocation relationships within an Android application to represent this application and further extract intrafunction attributes, including required permission, security level, and Smali instructions’ semantic information via Word2Vec, to form the node attributes within graph structures. Then, we use the graph neural network to generate a vector representation of the application, and then malware detection is performed on this representation space. We conduct experiments on real-world application samples. The experimental results demonstrate that our approach implements high effective malware detection and outperforms state-of-the-art detection approaches.

Automatic Lung Segmentation Algorithm on Chest X-ray Images Based on Fusion Variational Auto-Encoder and Three-Terminal Attention Mechanism

Automatic segmentation of the lungs in Chest X-ray images (CXRs) is a key step in the screening and diagnosis of related diseases. There are many opacities in the lungs in the CXRs of patients, which makes the lungs difficult to segment. In order to solve this problem, this paper proposes a segmentation algorithm based on U-Net. This article introduces variational auto-encoder (VAE) in each layer of the decoder-encoder. VAE can extract high-level semantic information, such as the symmetrical relationship between the left and right thoraxes in most cases. The fusion of the features of VAE and the features of convolution can improve the ability of the network to extract features. This paper proposes a three-terminal attention mechanism. The attention mechanism uses the channel and spatial attention module to automatically highlight the target area and improve the performance of lung segmentation. At the same time, the three-terminal attention mechanism uses the advanced semantics of high-scale features to improve the positioning and recognition capabilities of the attention mechanism, suppress background noise, and highlight target features. Experimental results on two different datasets show that the accuracy (ACC), recall (R), F1-Score and Jaccard values of the algorithm proposed in this paper are the highest on the two datasets, indicating that the algorithm in this paper is better than other state-of-the-art algorithms.

Line–Circle–Square (LCS): A multilayered geometric filter for edge-based detection

This paper presents a state-of-the-art filter that reduces the complexity in object detection, tracking and mapping applications. Existing edge detection and tracking methods are proposed to create suitable autonomy for mobile robots, however, many of them face overconfidence and large computations at the entrance to scenarios with an immense number of landmarks. The method in this work, the Line–Circle–Square (LCS) filter, claims that mobile robots without a large database for object recognition and highly advanced prediction methods can deal with incoming objects that the camera captures in real-time. The proposed filter applies detection, tracking and learning to each defined expert to extract higher level information for judging scenes without over-calculation. The interactive learning feed between each expert increases the consistency of detected landmarks that works against overwhelming detected features in crowded scenes. Our experts are dependent on trust factors’ covariance under the geometric definitions to ignore, emerge and compare detected landmarks. The experiment validates the effectiveness of the proposed filter in terms of detection precision and resource usage in both experimental and real-world scenarios.

DAR-Net: Dense Attentional Residual Network for Vehicle Detection in Aerial Images.

With the rapid development of deep learning and the wide usage of Unmanned Aerial Vehicles (UAVs), CNN-based algorithms of vehicle detection in aerial images have been widely studied in the past several years. As a downstream task of the general object detection, there are some differences between the vehicle detection in aerial images and the general object detection in ground view images, e.g., larger image areas, smaller target sizes, and more complex background. In this paper, to improve the performance of this task, a Dense Attentional Residual Network (DAR-Net) is proposed. The proposed network employs a novel dense waterfall residual block (DW res-block) to effectively preserve the spatial information and extract high-level semantic information at the same time. A multiscale receptive field attention (MRFA) module is also designed to select the informative feature from the feature maps and enhance the ability of multiscale perception. Based on the DW res-block and MRFA module, to protect the spatial information, the proposed framework adopts a new backbone that only downsamples the feature map 3 times; i.e., the total downsampling ratio of the proposed backbone is 8. These designs could alleviate the degradation problem, improve the information flow, and strengthen the feature reuse. In addition, deep-projection units are used to reduce the impact of information loss caused by downsampling operations, and the identity mapping is applied to each stage of the proposed backbone to further improve the information flow. The proposed DAR-Net is evaluated on VEDAI, UCAS-AOD, and DOTA datasets. The experimental results demonstrate that the proposed framework outperforms other state-of-the-art algorithms.

Plane-Wave Image Reconstruction via Generative Adversarial Network and Attention Mechanism

Plane-wave imaging (PWI) has been designed and widely applied in ultrasound imaging in recent years. This technique achieves high frame rate while compromising with the image quality, such as spatial resolution and contrast. Although the compounding method is capable to improve reconstruction effect, this method limits the temporal resolution. The existing generative adversarial network (GAN)-based methods are not capable to fully extract shallow, high-frequency, and high-level information. To reconstruct high-resolution plane-wave images, we propose a reconstruction method via attention mechanism and Unet-based GAN (AUGAN). The generator of AUGAN adopts the Unet-based architecture that containing skip concatenate pathways, which can capture low-level features of different scales. Besides, in order to excavate image information comprehensively, we adopt the pixel attention to strengthen the extraction of shallow features and local aware attention to put more attention to limited high-frequency features. Moreover, perceptual loss is introduced to make the model keep more high-level information of target images. The proposed method are evaluated on datasets acquired from simulation, experimental phantom, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in vivo</i> targets provided by the PWI Challenge in Medical Ultrasound (PICMUS). AUGAN can achieve maximal improvements of full-width at half-maximum (FWHM) by 8.27% (simulation) and 33.21% (experiment) compared with coherent plane-wave compounding (CPWC) while 2.46% (simulation) and 5.05% (experiment) compared with Unet-based GAN (UnetGAN). The maximum contrast ratio (CR) improvements are 11.32% upon CPWC and 2.3% upon UnetGAN. For <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in vivo</i> data, AUGAN shows its advantage in preserving anatomical structures compared with other state-of-the-art methods in plane-wave image reconstruction. The attention ablation experiment illustrates that both pixel attention and local aware attention have further improved the plane-wave image reconstruction.

An SOE-Based Learning Framework Using Multisource Big Data for Identifying Urban Functional Zones

Identifying urban functional zones is of great significance for understanding urban structure and urban planning. The rapid growth and open accessibility of multisource big data, including remote sensing imagery and social sensing data, lead to a new way for dynamic identification of urban functional zones. In this article, we propose an SOE (scene-object-economy) based learning framework which integrates scene features from remote sensing imagery, object features from building footprints, and economy features from POIs (points of interest). From these three perspectives, rich information hidden in urban zone is excavated for function identification. Convolutional neural networks are used to extract high-level scene information from remote sensing images with different resolutions. Object features comprising a series of building indicators are constructed by measuring the area, perimeter, floor number, and year of the building. Moreover, we extract socioeconomic characteristics from POIs, which reflect different types of human activities in the urban zone. Last, random forest is used to identify functional zones based on SOE features. We apply the SOE-based framework to Shenzhen datasets and achieve 90.8% in accuracy with remote sensing images of 0.3-m spatial resolution. The experimental results show that the predicting performance of SOE-based framework is significantly better than other traditional methods, and the quantitative contribution of SOE factors is also revealed in determining functionality of urban zones.

Sequential Gating Ensemble Network for Noise Robust Multiscale Face Restoration.

Face restoration from low resolution and noise is important for applications of face analysis recognition. However, most existing face restoration models omit the multiple scale issues in the face restoration problem, which is still not well solved in the research area. In this paper, we propose a sequential gating ensemble network (SGEN) for a multiscale noise robust face restoration issue. To endow the network with multiscale representation ability, we first employ the principle of ensemble learning for SGEN network architecture design. The SGEN aggregates multilevel base-encoders and base-decoders into the network, which enables the network to contain multiple scales of receptive field. Instead of combining these base-en/decoders directly with nonsequential operations, the SGEN takes base-en/decoders from different levels as sequential data. Specifically, it is visualized that SGEN learns to sequentially extract high-level information from base-encoders in a bottom-up manner and restore low-level information from base-decoders in a top-down manner. Besides, we propose realizing bottom-up and top-down information combination and selection with a sequential gating unit (SGU). The SGU sequentially takes information from two different levels as inputs and decides the output based on one active input. Experimental results on the benchmark dataset demonstrate that our SGEN is more effective at multiscale human face restoration with more image details and less noise than state-of-the-art image restoration models. Further utilizing an adversarial training scheme, SGEN also produces more visually preferred results than other models under subjective evaluation.

FSS-2019-nCov: A deep learning architecture for semi-supervised few-shot segmentation of COVID-19 infection

The newly discovered coronavirus (COVID-19) pneumonia is providing major challenges to research in terms of diagnosis and disease quantification. Deep-learning (DL) techniques allow extremely precise image segmentation; yet, they necessitate huge volumes of manually labeled data to be trained in a supervised manner. Few-Shot Learning (FSL) paradigms tackle this issue by learning a novel category from a small number of annotated instances. We present an innovative semi-supervised few-shot segmentation (FSS) approach for efficient segmentation of 2019-nCov infection (FSS-2019-nCov) from only a few amounts of annotated lung CT scans. The key challenge of this study is to provide accurate segmentation of COVID-19 infection from a limited number of annotated instances. For that purpose, we propose a novel dual-path deep-learning architecture for FSS. Every path contains encoder–decoder (E-D) architecture to extract high-level information while maintaining the channel information of COVID-19 CT slices. The E-D architecture primarily consists of three main modules: a feature encoder module, a context enrichment (CE) module, and a feature decoder module. We utilize the pre-trained ResNet34 as an encoder backbone for feature extraction. The CE module is designated by a newly introduced proposed Smoothed Atrous Convolution (SAC) block and Multi-scale Pyramid Pooling (MPP) block. The conditioner path takes the pairs of CT images and their labels as input and produces a relevant knowledge representation that is transferred to the segmentation path to be used to segment the new images. To enable effective collaboration between both paths, we propose an adaptive recombination and recalibration (RR) module that permits intensive knowledge exchange between paths with a trivial increase in computational complexity. The model is extended to multi-class labeling for various types of lung infections. This contribution overcomes the limitation of the lack of large numbers of COVID-19 CT scans. It also provides a general framework for lung disease diagnosis in limited data situations.

Conditional Generative Adversarial Network-Based Training Sample Set Improvement Model for the Semantic Segmentation of High-Resolution Remote Sensing Images

To achieve high segmentation quality, deep semantic segmentation neural networks (DSSNNs) need to be trained on diverse direction, location, and neighboring category combinations for each pixel in the input-output image patches. Achieving this goal requires a large sample set. However, in many practical application scenarios, a very large training sample set is too expensive to achieve, or insufficient remote sensing image data are available. These limitations directly affect the quality of the results of DSSNNs. To address the above-mentioned problem, this article proposes a conditional generative adversarial network (CGAN)-based training sample set improvement model (CGAN-TSIM) for the semantic segmentation of high-resolution remote sensing images. In CGAN-TSIM, the generator model of the CGAN can generate a sample image when a ground-truth image is an input as a “condition.” A condition generation mechanism is designed to create ground-truth images, and these ground-truth conditions are used to drive the CGAN to generate samples containing more diverse object combinations, directions, and locations. These generated images can be added to the original training sample set to improve their spatial information diversity. Rather than simply relying on passively finding samples that contain diverse spatial information, CGAN-TSIM extracts high-level spatial information from the original training images and actively generates new sample images. Experiments show that the samples generated by CGAN-TSIM can improve the quality of the sample set. Compared with other traditional methods, CGAN-TSIM enables better classification accuracy.

Development of Frequency-Mixed Point-Focusing Shear Horizontal Guided-Wave EMAT for Defect Inspection Using Deep Neural Network

We propose a new frequency-mixed point-focusing shear horizontal (SH) guided-wave electromagnetic acoustic transducer (EMAT) in this work to obtain the defect positions and plate thickness simultaneously and accurately. Compared with other guided-wave detection methods, it is not required to measure the plate thickness in advance because we can easily obtain it during the test. We use the variational mode decomposition method to decompose the received frequency-mixed defect signal into subsignals with different center frequencies and to remove the noise. Furthermore, we use the continuous wavelet transform to analyze these subsignals using the time-frequency method and to obtain the time-of-flight information of the guided wave under different frequencies and modes. Therefore, we can obtain accurate defect positions and plate thicknesses via the new transducer and signal processing methods while improving the signal intensities. In the identification of defect types, we first constructed a database set containing three types of defects of different sizes using data enhancement methods. Then, the dense network, convolutional neural network, recurrent neural network, and newly proposed deep GFresNet are studied to analyze the defect classification performance of each structure. The results show that the proposed GFresNet has very good defect identification accuracy, which is about 95% along any depth of the defects, and that it can automatically extract high-level information without sophisticated feature engineering.

Wood Defect Detection Based on Depth Extreme Learning Machine

The deep learning feature extraction method and extreme learning machine (ELM) classification method are combined to establish a depth extreme learning machine model for wood image defect detection. The convolution neural network (CNN) algorithm alone tends to provide inaccurate defect locations, incomplete defect contour and boundary information, and inaccurate recognition of defect types. The nonsubsampled shearlet transform (NSST) is used here to preprocess the wood images, which reduces the complexity and computation of the image processing. CNN is then applied to manage the deep algorithm design of the wood images. The simple linear iterative clustering algorithm is used to improve the initial model; the obtained image features are used as ELM classification inputs. ELM has faster training speed and stronger generalization ability than other similar neural networks, but the random selection of input weights and thresholds degrades the classification accuracy. A genetic algorithm is used here to optimize the initial parameters of the ELM to stabilize the network classification performance. The depth extreme learning machine can extract high-level abstract information from the data, does not require iterative adjustment of the network weights, has high calculation efficiency, and allows CNN to effectively extract the wood defect contour. The distributed input data feature is automatically expressed in layer form by deep learning pre-training. The wood defect recognition accuracy reached 96.72% in a test time of only 187 ms.

An End-to-End Framework for Clothing Collocation Based on Semantic Feature Fusion

In this article, we develop an end-to-end clothing collocation learning framework based on a bidirectional long short-term memories (Bi-LTSM) model, and propose new feature extraction and fusion modules. The feature extraction module uses Inception V3 to extract low-level feature information and the segmentation branches of Mask Region Convolutional Neural Network (RCNN) to extract high-level semantic information; whereas the feature fusion module creates a new reference vector for each image to fuse the two types of image feature information. As a result, the feature can involve both low-level image and high-level semantic feature information, so that the performance of Bi-LSTM can be enhanced. Extensive simulations are conducted based on Ployvore and DeepFashion2 datasets. Simulation results verify the effectiveness of the proposed method compared with other state-of-the-art clothing collocation methods.

The polar sea ice topography reconstruction system

AbstractThe Polar Sea Ice Topography REconstruction System, or PSITRES, is a 3D camera system designed to continuously monitor an area of ice and water adjacent to an ice-going vessel. Camera systems aboard ships in the polar regions are common; however, the application of computer vision techniques to extract high-level information from the imagery is infrequent. Many of the existing systems are built for human involvement throughout the process and lack automation necessary for round the clock use. The PSITRES was designed with computer vision in mind. It can capture images continuously for days on end with limited oversight. We have applied the system in different ice observing scenarios. The PSITRES was deployed on three research expeditions in the Arctic and Subarctic, and we present applications in measuring ice concentration, melt pond fraction and presence of algae. Systems like PSITRES and the computer vision algorithms applied represent steps toward automatically observing, evaluating and analyzing ice and the environment around ships in ice-covered waters.