Single Titan Research Articles

Deep learning based prediction of traffic peaks in mobile networks

In mobile networks, it is essential to configure networks more efficiently to provide mobile users with services having better quality. For the adjacent cells, sometimes the mobile traffic concentrates in a single cell, leading to traffic imbalance; in many cases, the detection of imbalance is strongly related to the prediction of traffic peaks, which is extremely difficult because many peaks appear suddenly for no apparent reason. To better predict the peaks and traffic imbalance, we propose two novel mobile traffic predictors. The first is a Mixture of Experts (MoE) model which yields significantly better peak prediction along with excellent interpretability by establishing a cooperation mechanism between different experts, whereas the second predictor is a lightweight Multilayer Perceptron (MLP) which can obtain similar peak forecasting performance but operating more flexibly and consuming less computational power. The obtained predictions are then used to aid the predictive detection of traffic imbalance. To this end, we first perform a large-scale analysis of mobile traffic and then propose different approaches to detect the future traffic imbalance based on the predictions. To evaluate the performance of the proposed approaches, we have conducted extensive experiments on real-world mobile network datasets; all models are implemented in Python 3.8.8 with Pytorch 1.7.0, and they were run on a computer with i7-7700K CPU @ 4.2 GHz running Linux, 64 GB memory and a single Nvidia TITAN X Pascal 12GB GPU. The proposed models are compared with 10 widely used deep learning based predictors, and the results show that our approaches have significantly improved the sensitivity (peak prediction accuracy) from 39.3% to 58.5% (i.e., roughly a 50% increase) along with excellent interpretability. Moreover, the predictions are further used by the predictive detection approach, where the best multi-model approach improves the predictive classification accuracy of congestion (imbalance prediction accuracy) by 10.3% in contrast with the naive approach; this result also verifies the effectiveness of our peak predictions.

A GPU-based fast Monte Carlo code that supports proton transport in magnetic field for radiation therapy.

This paper presents the effort to extend a previously reported code ARCHER, a GPU-based Monte Carlo (MC) code for coupled photon and electron transport, into protons including the consideration of magnetic fields. The proton transport is modeled using a Class-II condensed-history algorithm with continuous slowing-down approximation. The model includes ionization, multiple scattering, energy straggling, elastic and inelastic nuclear interactions, as well as deflection due to the Lorentz force in magnetic fields. An additional direction change is added for protons at the end of each step in the presence of the magnetic field. Secondary charge particles, except for protons, are terminated depositing kinetic energies locally, whereas secondary neutral particles are ignored. Each proton is transported step by step until its energy drops to below 0.5 MeV or when the proton leaves the phantom. The code is implemented using the compute unified device architecture (CUDA) platform for optimized GPU thread-level parallelism and efficiency. The code is validated by comparing it against TOPAS. Comparisons of dose distributions between our code and TOPAS for several exposure scenarios, ranging from single square beams in water to patient plan with magnetic fields, show good agreement. The 3D-gamma pass rate with a 2mm/2% criterion in the region with dose greater than 10% of the maximum dose is computed to be over 99% for all tested cases. Using a single NVIDIA TITAN V GPU card, the computational time of ARCHER is found to range from 0.82 to 4.54 seconds for 1×107 proton histories. Compared to a few hours running on TOPAS, this speed improvement is significant. This work presents, for the first time, the performance of a GPU-based MC code to simulate proton transportation magnetic fields, demonstrating the feasibility of accurate and efficient dose calculations in potential magnetic resonance imaging (MRI)-guided proton therapy.

MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation

In recent years, how to strike a good trade-off between accuracy, inference speed, and model size has become the core issue for real-time semantic segmentation applications, which plays a vital role in real-world scenarios such as autonomous driving systems and drones. In this study, we devise a novel lightweight network using a multi-scale context fusion (MSCFNet) scheme, which explores an asymmetric encoder-decoder architecture to alleviate these problems. More specifically, the encoder adopts some developed efficient asymmetric residual (EAR) modules, which are composed of factorization depth-wise convolution and dilation convolution. Meanwhile, instead of complicated computation, simple deconvolution is applied in the decoder to further reduce the amount of parameters while still maintaining the high segmentation accuracy. Also, MSCFNet has branches with efficient attention modules from different stages of the network to well capture multi-scale contextual information. Then we combine them before the final classification to enhance the expression of the features and improve the segmentation efficiency. Comprehensive experiments on challenging datasets have demonstrated that the proposed MSCFNet, which contains only 1.15M parameters, achieves 71.9% Mean IoU on the Cityscapes testing dataset and can run at over 50 FPS on a single Titan XP GPU configuration.

FastRoadSeg: Fast Monocular Road Segmentation Network

Monocular road segmentation is a fundamental component of autonomous driving. High accuracy has always been a focus of the community for this task. However, the exploration of high-speed methods is still lacking. This limits the prevalence of the monocular road segmentation method. In this paper, an efficient encoder-decoder network is proposed for segmenting the road from a single image. Specifically, to achieve high-speed inference, we leverage a shallow encoder for feature extraction and design a lightweight decoder for feature resolution recovery. To avoid the accuracy drop caused by network simplification, we introduce an efficient asymmetric dilated block to improve the ability of features to distinguish drivable/nondrivable roads to maintain the performance without excess computation. Experiments on three datasets verify the effectiveness of our method. On KITTI-Road, our approach ranks second among all monocular methods and even achieves a competitive performance compared to multimodal methods, with an inference speed over 135 FPS on a single TITAN Xp, which is at least 20 times faster than the state-of-the-art monocular method. Furthermore, our approach can be deployed on the embedded platform Jetson TX2 and achieves a speed of over 80 FPS. Codes will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/gongshichina/FastRoadSeg</uri> .

MFNet: Multi-Feature Fusion Network for Real-Time Semantic Segmentation in Road Scenes

Although high-accuracy networks have been applied to semantic segmentation at present, their inference speeds remain slow. A trade-off between accuracy and speed is demanded for real-time applications. To approach this problem, we propose Multi-Feature Fusion Network (MFNet) with real-time efficient prediction capacity. MFNet adopts three branches (attention, semantic and spatial information) to capture low-level and high-level features. Additionally, MFNet exerts asymmetric factorized (AF) blocks to extract local and long-range features. As a result, without any pre-training or post-processing, MFNet using only 1.34 M parameters, achieves 72.1% mean intersection over union (mIoU) on the Cityscapes test set at a speed of 116 frames per second (FPS), with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$512\times 1024$ </tex-math></inline-formula> high resolution on a single Titan Xp graphics card. Our network’s performance stands out from other state-of-the-art networks on four datasets (Cityscapes, CamVid, KITTI, and Gatech).

Directional Deep Embedding and Appearance Learning for Fast Video Object Segmentation.

Most recent semisupervised video object segmentation (VOS) methods rely on fine-tuning deep convolutional neural networks online using the given mask of the first frame or predicted masks of subsequent frames. However, the online fine-tuning process is usually time-consuming, limiting the practical use of such methods. We propose a directional deep embedding and appearance learning (DDEAL) method, which is free of the online fine-tuning process, for fast VOS. First, a global directional matching module (GDMM), which can be efficiently implemented by parallel convolutional operations, is proposed to learn a semantic pixel-wise embedding as an internal guidance. Second, an effective directional appearance model-based statistics is proposed to represent the target and background on a spherical embedding space for VOS. Equipped with the GDMM and the directional appearance model learning module, DDEAL learns static cues from the labeled first frame and dynamically updates cues of the subsequent frames for object segmentation. Our method exhibits the state-of-the-art VOS performance without using online fine-tuning. Specifically, it achieves a J & F mean score of 74.8% on DAVIS 2017 data set and an overall score G of 71.3% on the large-scale YouTube-VOS data set, while retaining a speed of 25 fps with a single NVIDIA TITAN Xp GPU. Furthermore, our faster version runs 31 fps with only a little accuracy loss.

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate high-resolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both deep and shallow features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) at the inference speeds of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

Real-Time Semantic Segmentation With Fast Attention

In deep CNN based models for semantic segmentation, high accuracy relies on rich spatial context (large receptive fields) and fine spatial details (high resolution), both of which incur high computational costs. In this letter, we propose a novel architecture that addresses both challenges and achieves state-of-the-art performance for semantic segmentation of high-resolution images and videos in real-time. The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism and captures the same rich spatial context at a small fraction of the computational cost, by changing the order of operations. Moreover, to efficiently process high-resolution input, we apply an additional spatial reduction to intermediate feature stages of the network with minimal loss in accuracy thanks to the use of the fast attention module to fuse features. We validate our method with a series of experiments, and show that results on multiple datasets demonstrate superior performance with better accuracy and speed compared to existing approaches for real-time semantic segmentation. On Cityscapes, our network achieves 74.4% mIoU at 72 FPS and 75.5% mIoU at 58 FPS on a single Titan X GPU, which is ~50% faster than the state-of-the-art while retaining the same accuracy.

FASSD: A Feature Fusion and Spatial Attention-Based Single Shot Detector for Small Object Detection

Deep learning methods have significantly improved object detection performance, but small object detection remains an extremely difficult and challenging task in computer vision. We propose a feature fusion and spatial attention-based single shot detector (FASSD) for small object detection. We fuse high-level semantic information into shallow layers to generate discriminative feature representations for small objects. To adaptively enhance the expression of small object areas and suppress the feature response of background regions, the spatial attention block learns a self-attention mask to enhance the original feature maps. We also establish a small object dataset (LAKE-BOAT) of a scene with a boat on a lake and tested our algorithm to evaluate its performance. The results show that our FASSD achieves 79.3% mAP (mean average precision) on the PASCAL VOC2007 test with input 300 × 300, which outperforms the original single shot multibox detector (SSD) by 1.6 points, as well as most improved algorithms based on SSD. The corresponding detection speed was 45.3 FPS (frame per second) on the VOC2007 test using a single NVIDIA TITAN RTX GPU. The test results of a simplified FASSD on the LAKE-BOAT dataset indicate that our model achieved an improvement of 3.5% mAP on the baseline network while maintaining a real-time detection speed (64.4 FPS).

YOLACT++ Better Real-Time Instance Segmentation.

We present a simple, fully-convolutional model for real-time ( fps) instance segmentation that achieves competitive results on MS COCO evaluated on a single Titan Xp, which is significantly faster than any previous state-of-the-art approach. Moreover, we obtain this result after training on only one GPU. We accomplish this by breaking instance segmentation into two parallel subtasks: (1) generating a set of prototype masks and (2) predicting per-instance mask coefficients. Then we produce instance masks by linearly combining the prototypes with the mask coefficients. We find that because this process doesn't depend on repooling, this approach produces very high-quality masks and exhibits temporal stability for free. Furthermore, we analyze the emergent behavior of our prototypes and show they learn to localize instances on their own in a translation variant manner, despite being fully-convolutional. We also propose Fast NMS, a drop-in 12 ms faster replacement for standard NMS that only has a marginal performance penalty. Finally, by incorporating deformable convolutions into the backbone network, optimizing the prediction head with better anchor scales and aspect ratios, and adding a novel fast mask re-scoring branch, our YOLACT++ model can achieve 34.1 mAP on MS COCO at 33.5 fps, which is fairly close to the state-of-the-art approaches while still running at real-time.

Pyramid Constrained Self-Attention Network for Fast Video Salient Object Detection

Spatiotemporal information is essential for video salient object detection (VSOD) due to the highly attractive object motion for human's attention. Previous VSOD methods usually use Long Short-Term Memory (LSTM) or 3D ConvNet (C3D), which can only encode motion information through step-by-step propagation in the temporal domain. Recently, the non-local mechanism is proposed to capture long-range dependencies directly. However, it is not straightforward to apply the non-local mechanism into VSOD, because i) it fails to capture motion cues and tends to learn motion-independent global contexts; ii) its computation and memory costs are prohibitive for video dense prediction tasks such as VSOD. To address the above problems, we design a Constrained Self-Attention (CSA) operation to capture motion cues, based on the prior that objects always move in a continuous trajectory. We group a set of CSA operations in Pyramid structures (PCSA) to capture objects at various scales and speeds. Extensive experimental results demonstrate that our method outperforms previous state-of-the-art methods in both accuracy and speed (110 FPS on a single Titan Xp) on five challenge datasets. Our code is available at https://github.com/guyuchao/PyramidCSA.

DENAO: Monocular Depth Estimation Network With Auxiliary Optical Flow.

Estimating depth from multi-view images captured by a localized monocular camera is an essential task in computer vision and robotics. In this study, we demonstrate that learning a convolutional neural network (CNN) for depth estimation with an auxiliary optical flow network and the epipolar geometry constraint can greatly benefit the depth estimation task and in turn yield large improvements in both accuracy and speed. Our architecture is composed of two tightly-coupled encoder-decoder networks, i.e., an optical flow net and a depth net, the core part being a list of exchange blocks between the two nets and an epipolar feature layer in the optical flow net to improve predictions of both depth and optical flow. Our architecture allows to input arbitrary number of multiview images with a linearly growing time cost for optical flow and depth estimation. Experimental result on five public datasets demonstrates that our method, named DENAO, runs at 38.46fps on a single Nvidia TITAN Xp GPU which is 5.15X ∼142X faster than the state-of-the-art depth estimation methods Meanwhile, our DENAO can concurrently output predictions of both depth and optical flow, and performs on par with or outperforms the state-of-the-art depth estimation methods and optical flow methods.

A deep learning-based surface defect inspection system using multi-scale and channel-compressed features

In machine vision-based surface inspection tasks, defects are typically considered as local anomalies in homogeneous background. However, industrial workpieces commonly contain complex structures, including hallow regions, welding joints, or rivet holes. Such obvious structural interference will inevitably cause a cluttered background and mislead the classification results. Moreover, the sizes of various surface defects might change significantly. Last but not least, it is extremely time-consuming and not scalable to capture large-scale defect data sets to train deep CNN models. To address the challenges mentioned earlier, we first proposed to incorporate multiple convolutional layers with different kernel sizes to increase the receptive field and to generate multiscale features. As a result, the proposed model can better handle the cluttered background and defects of various sizes. Also, we purposely compress the size of parameters in the newly added convolutional layers for better learning of defect-related features using a limited number of training samples. Evaluated in a newly constructed surface defect data set (images contain complex structures and defects of various sizes), our proposed model achieves more accurate recognition results compared with the state-of-the-art surface defect classifiers. Moreover, it is a lightweight model and can deliver real-time processing speed (>100 frames/s) on a computer equipped with a single NVIDIA TITAN X Graphics Processing Unit (12-GB memory).

Accelerating the Generalized Born with Molecular Volume and Solvent Accessible Surface Area Implicit Solvent Model Using Graphics Processing Units.

The generalized Born with molecular volume and solvent accessible surface area (GBMV2/SA) implicit solvent model provides an accurate description of molecular volume and has the potential to accurately describe the conformational equilibria of structured and disordered proteins. However, its broader application has been limited by the computational cost and poor scaling in parallel computing. Here, we report an efficient implementation of both the electrostatic and nonpolar components of GBMV2/SA on graphics processing unit (GPU) within the CHARMM/OpenMM module. The GPU-GBMV2/SA is numerically equivalent to the original CPU-GBMV2/SA. The GPU acceleration offers ~60- to 70-fold speedup on a single NVIDIA TITAN X (Pascal) graphics card for molecular dynamic simulations of both folded and unstructured proteins of various sizes. The current implementation can be further optimized to achieve even greater acceleration with minimal reduction on the numerical accuracy. The successful development of GPU-GBMV2/SA greatly facilitates its application to biomolecular simulations and paves the way for further development of the implicit solvent methodology. © 2019 Wiley Periodicals, Inc.

GPET: a GPU-based, accurate and efficient Monte Carlo simulation tool for PET

Monte Carlo (MC) simulation method plays an essential role in the refinement and development of positron emission tomography (PET) systems. However, most existing MC simulation packages suffer from long execution time for practical PET simulations. To fully address this issue, we developed and validated gPET, a graphics processing unit (GPU)-based MC simulation tool for PET. gPET was built on the NVidia CUDA platform. The simulation process was modularized into three functional parts and carried out by the GPU parallel threads: (1) source management, including positron decay, transport and annihilation; (2) gamma transport inside the phantom; and (3) signal detection and processing inside the detector. A hybrid of voxelized (for patient phantoms) and parametrized (for detectors) geometries were employed to sufficiently support particle navigations. Multiple inputs and outputs were available. Hence, a user can flexibly examine different aspects of a PET simulation. We evaluated the performance of gPET in three test cases with benchmark work from GATE8.0, in terms of the testing of the functional modules, the physics models used for gamma transport inside the detector, and the geometric configuration of an irregularly shaped PET detector. Both accuracy and efficiency were quantified. In all test cases, the differences between gPET and GATE for the coincidences with respect to the energy and crystal index distributions are below 3.18% and 2.54%, respectively. The speedup factor is 500 for gPET on a single Titan Xp GPU (1.58 GHz) over GATE8.0 on a single core of Intel i7-6850K CPU (3.6 GHz) for all test cases. In summary, gPET is an accurate and efficient MC simulation tool for PET.

A deep-learning-based approach for fast and robust steel surface defects classification

Automatic visual recognition of steel surface defects provides critical functionality to facilitate quality control of steel strip production. In this paper, we present a compact yet effective convolutional neural network (CNN) model, which emphasizes the training of low-level features and incorporates multiple receptive fields, to achieve fast and accurate steel surface defect classification. Our proposed method adopts the pre-trained SqueezeNet as the backbone architecture. It only requires a small amount of defect-specific training samples to achieve high-accuracy recognition on a diversity-enhanced testing dataset of steel surface defects which contains severe non-uniform illumination, camera noise, and motion blur. Moreover, our proposed light-weight CNN model can meet the requirement of real-time online inspection, running over 100 fps on a computer equipped with a single NVIDIA TITAN X Graphics Processing Unit (12G memory). Codes and a diversity-enhanced testing dataset will be made publicly available.

Dense RGB-D Semantic Mapping with Pixel-Voxel Neural Network.

In this paper, a novel Pixel-Voxel network is proposed for dense 3D semantic mapping, which can perform dense 3D mapping while simultaneously recognizing and labelling the semantic category each point in the 3D map. In our approach, we fully leverage the advantages of different modalities. That is, the PixelNet can learn the high-level contextual information from 2D RGB images, and the VoxelNet can learn 3D geometrical shapes from the 3D point cloud. Unlike the existing architecture that fuses score maps from different modalities with equal weights, we propose a softmax weighted fusion stack that adaptively learns the varying contributions of PixelNet and VoxelNet and fuses the score maps according to their respective confidence levels. Our approach achieved competitive results on both the SUN RGB-D and NYU V2 benchmarks, while the runtime of the proposed system is boosted to around 13 Hz, enabling near-real-time performance using an i7 eight-cores PC with a single Titan X GPU.

Deep Semantic Role Labeling With Self-Attention

Semantic Role Labeling (SRL) is believed to be a crucial step towards natural language understanding and has been widely studied. Recent years, end-to-end SRL with recurrent neural networks (RNN) has gained increasing attention. However, it remains a major challenge for RNNs to handle structural information and long range dependencies. In this paper, we present a simple and effective architecture for SRL which aims to address these problems. Our model is based on self-attention which can directly capture the relationships between two tokens regardless of their distance. Our single model achieves F1=83.4 on the CoNLL-2005 shared task dataset and F1=82.7 on the CoNLL-2012 shared task dataset, which outperforms the previous state-of-the-art results by 1.8 and 1.0 F1 score respectively. Besides, our model is computationally efficient, and the parsing speed is 50K tokens per second on a single Titan X GPU.

ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation

Semantic segmentation is a challenging task that addresses most of the perception needs of intelligent vehicles (IVs) in an unified way. Deep neural networks excel at this task, as they can be trained end-to-end to accurately classify multiple object categories in an image at pixel level. However, a good tradeoff between high quality and computational resources is yet not present in the state-of-the-art semantic segmentation approaches, limiting their application in real vehicles. In this paper, we propose a deep architecture that is able to run in real time while providing accurate semantic segmentation. The core of our architecture is a novel layer that uses residual connections and factorized convolutions in order to remain efficient while retaining remarkable accuracy. Our approach is able to run at over 83 FPS in a single Titan X, and 7 FPS in a Jetson TX1 (embedded device). A comprehensive set of experiments on the publicly available Cityscapes data set demonstrates that our system achieves an accuracy that is similar to the state of the art, while being orders of magnitude faster to compute than other architectures that achieve top precision. The resulting tradeoff makes our model an ideal approach for scene understanding in IV applications. The code is publicly available at: https://github.com/Eromera/erfnet

Multi-GPU Acceleration of Branchless Distance Driven Projection and Backprojection for Clinical Helical CT

Abstract Model-based image reconstruction (MBIR) techniques have the potential to generate high quality images from noisy measurements and a small number of projections which can reduce the x-ray dose in patients. These MBIR techniques rely on projection and backprojection to refine an image estimate. One of the widely used projectors for these modern MBIR based technique is called branchless distance driven (DD) projection and backprojection. While this method produces superior quality images, the computational cost of iterative updates keeps it from being ubiquitous in clinical applications. In this paper, we provide several new parallelization ideas for concurrent execution of the DD projectors in multi-GPU systems using CUDA programming tools. We have introduced some novel schemes for dividing the projection data and image voxels over multiple GPUs to avoid runtime overhead and inter-device synchronization issues. We have also reduced the complexity of overlap calculation of the algorithm by eliminating the common projection plane and directly projecting the detector boundaries onto image voxel boundaries. To reduce the time required for calculating the overlap between the detector edges and image voxel boundaries, we have proposed a pre-accumulation technique to accumulate image intensities in perpendicular 2D image slabs (from a 3D image) before projection and after backprojection to ensure our DD kernels run faster in parallel GPU threads. For the implementation of our iterative MBIR technique we use a parallel multi-GPU version of the alternating minimization (AM) algorithm with penalized likelihood update. The time performance using our proposed reconstruction method with Siemens Sensation 16 patient scan data shows an average of 24 times speedup using a single TITAN X GPU and 74 times speedup using 3 TITAN X GPUs in parallel for combined projection and backprojection.