Convolutional Filters Research Articles

Logistics Optimization Strategy Based on Deep Neural Framework.

We propose a logistics optimization method based on improved graph convolutional networks to address the current problem of low product delivery rate and untimely product delivery during the peak period of e-commerce activities. Our method can learn excellent planning strategies from previous data and can give the best logistics strategy in time during the peak logistics period, which improves the product delivery rate and delivery time of logistics and greatly enhances the return on investment. First, we add a tensor rotation module to the graph convolution layer to better capture the global features of logistics nodes. Then we add inception structures in the temporal convolution layer to build multiscale temporal convolution filters to obtain temporal information of logistics nodes in different time-aware domains and reduce arithmetic power. Finally, we cooperate with e-commerce platforms to adopt logistics data as the experimental database. The experimental results show that our method greatly accelerates the logistics planning speed, improves the product delivery rate, ensures the timely delivery of products, and increases the return on investment.

A novel intelligent fault diagnosis method of rotating machinery based on signal-to-image mapping and deep Gabor convolutional adaptive pooling network

To address the limitations of insufficient feature representation and easy to be overwhelmed by strong noise in the grayscale images of vibration signals, the random generation and single structure of convolutional kernels of the convolutional neural network leading to insufficient extracted features, and the max pooling and average pooling leading to model overfitting and suppression of critical fault features, this paper proposes a novel diagnosis method based on signal-to-image mapping and deep Gabor convolutional adaptive pooling network. Firstly, this paper designs a vibration signal-to-image mapping strategy to highlight the fault information of vibration signal. Then Gabor convolutional filter is proposed instead of convolutional kernels to guide the model to extract multi-scale and multi-directional fault features. Next, the dynamic adaptive pooling is designed to facilitate the retention of local features and suppress the decay of critical features. Finally, a deep Gabor convolutional adaptive pooling network model is constructed to improve the robustness of the fault feature extraction process and the generalization of the model. The results of the bearing and gear datasets indicate that the proposed method enhances the feature extraction ability, improves the robustness and generalization of the model, and realizes highly accurate fault diagnosis.

SparNet: A Convolutional Neural Network for EEG Space-Frequency Feature Learning and Depression Discrimination.

Depression affects many people around the world today and is considered a global problem. Electroencephalogram (EEG) measurement is an appropriate way to understand the underlying mechanisms of major depressive disorder (MDD) to distinguish depression from normal control. With the development of deep learning methods, many researchers have adopted deep learning models to improve the classification accuracy of depression recognition. However, there are few studies on designing convolution filters for spatial and frequency domain feature learning in different brain regions. In this study, SparNet, a convolutional neural network composed of five parallel convolutional filters and the SENet, is proposed to learn EEG space-frequency domain characteristics and distinguish between depressive and normal control. The model is trained and tested by the cross-validation method of subject division. The results show that SparNet achieves a sensitivity of 95.07%, a specificity of 93.66%, and an accuracy of 94.37% in classification. Therefore, our results can conclude that the proposed SparNet model is effective in detecting depression using EEG signals. It also indicates that the combination of spatial information and frequency domain information is an effective way to identify patients with depression.

A fault diagnosis method for power grid based on image feature extraction

In order to solve the problem that the traditional power grid fault diagnosis along the railway requires a lot of manpower and material resources, this paper proposes a fault diagnosis method on the power grid along the railway based on image feature extraction. A variable function family series method is designed to extract features of faults. For the fault image denoising, a convolution filter method is used, and the number of 2 denoising layers is selected. At the same time, extracting the decomposition coefficient features and energy features of power grid image faults. Moreover, the matching diagnosis method is given. The fault is finally identified based on the similarity. To compare with the existing fault diagnosis methods, the correct diagnosis rate of the fault diagnosis method proposed is more than 92%. From experimental comparison results, this diagnosis effect not only has higher precision, stronger anti-interference ability, but also has faster diagnosis speed.

A Robust Deep Learning Network for Low-Speed Machinery Fault Diagnosis Based on Multikernel and RPCA

Machinery fault diagnosis is an attractive but challenging task, especially for low-speed conditions. Therefore, a new discriminative approach that introduces robust principal component analysis (RPCA) and multikernel to deep neural networks is proposed to perform intelligent fault diagnosis. First, RPCA is applied to extract fault signals from extreme background noise based on its sensitivity to grossly corrupted data. Second, two cascaded multikernel principal component analysis stages with additional robustness to distortions in feature extraction are used to enhance the energy of spectrum symptom and overcome the tricky issues of low-speed machinery. Especially, the multikernel is introduced into the basic PCA filters to learn the data-adapting convolution filter and gain additional robustness to nonlinearity in the signal. Finally, the proposed method is demonstrated on signals from laboratory tests (with a slightly damaged defect in a bearing) and structural fault data, outperforming those of traditional machine learning and classical deep learning methods. Moreover, hidden information of the network is visualized to analyze the reasons for its high performance.

MIMO-OFDM Modulation Classification Using Three-Dimensional Convolutional Network

Automatic modulation classification (AMC) plays a vital role in cognitive radio to improve spectrum utilization efficiency, however, most of the existing works have focused on single-carrier communications in single-input single-output systems. In this paper, we propose an efficient AMC method for multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) communication systems with the assumption of unknown frequency-selective fading channels and signal-to-noise ratio. At the receiver, the complex envelope samples of a burst signal acquired by multiple antennas are decomposed into in-phase and quadrature samples, which are then structured into a high-dimensional data array. To learn the modulation patterns from received signals, we develop a deep network, namely three-dimensional MIMO-OFDM convolutional neural network (MONet). With cuboidal convolution filters, the proposed MONet allows the network to capture underlying features as intra- and inter-antenna correlations at multi-scale signal representations. Relying on simulations, MONet achieves the classification accuracy of over 95% at 0 dB SNR under various channel impairments and shows the robustness with different MIMO antenna configurations.

Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function

Magnetic resonance imaging (MRI) images synthesized from computed tomography (CT) data can provide more detailed information on pathological structures than that of CT data alone; thus, the synthesis of MRI has received increased attention especially in medical scenarios where only CT images are available. A novel convolutional neural network (CNN) combined with a contextual loss function was proposed for synthesis of T1- and T2-weighted images (T1WI and T2WI) from CT data. A total of 5,053 and 5,081 slices of T1WI and T2WI, respectively were selected for the dataset of CT and MRI image pairs. Affine registration, image denoising, and contrast enhancement were done on the aforementioned multi-modality medical image dataset comprising T1WI, T2WI, and CT images of the brain. A deep CNN was then proposed by modifying the ResNet structure to constitute the encoder and decoder of U-Net, called double ResNet-U-Net (DRUNet). Three different loss functions were utilized to optimize the parameters of the proposed models: mean squared error (MSE) loss, binary crossentropy (BCE) loss, and contextual loss. Statistical analysis of the independent-sample t-test was conducted by comparing DRUNets with different loss functions and different network layers. DRUNet-101 with contextual loss yielded higher values of peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and Tenengrad function (i.e., 34.25±2.06, 0.97±0.03, and 17.03±2.75 for T1WI and 33.50±1.08, 0.98±0.05, and 19.76±3.54 for T2WI respectively). The results were statistically significant at P<0.001 with a narrow confidence interval of difference, indicating the superiority of DRUNet-101 with contextual loss. In addition, both image zooming and difference maps presented for the final synthetic MR images visually reflected the robustness of DRUNet-101 with contextual loss. The visualization of convolution filters and feature maps showed that the proposed model can generate synthetic MR images with high-frequency information. The results demonstrated that DRUNet-101 with contextual loss function provided better high-frequency information in synthetic MR images compared with the other two functions. The proposed DRUNet model has a distinct advantage over previous models in terms of PSNR, SSIM, and Tenengrad score. Overall, DRUNet-101 with contextual loss is recommended for synthesizing MR images from CT scans.

Automated Apple Recognition System Using Semantic Segmentation Networks with Group and Shuffle Operators

Apples are one of the most consumed fruits, and they require efficient harvesting procedures to remains in optimal states for a longer period, especially during transportation. Therefore, automation has been adopted by many orchard operators to help in the harvesting process, which includes apple localization on the trees. The de facto sensor that is currently used for this task is the standard camera, which can capture wide view information of various apple trees from a reasonable distance. Therefore, this paper aims to produce the output mask of the apple locations on the tree automatically by using a deep semantic segmentation network. The network must be robust enough to overcome all challenges of shadow, surrounding illumination, size variations, and occlusion to produce accurate pixel-wise localization of the apples. A high-resolution deep architecture is embedded with an optimized design of group and shuffle operators (GSO) to produce the best apple segmentation network. GSO allows the network to reduce the dependency on a few sets of dominant convolutional filters by forcing each smaller group to contribute effectively to the task of extracting optimal apple features. The experimental results show that the proposed network, GSHR-Net, with two sets of group convolution applied to all layers produced the best mean intersection over union of 0.8045. The performance has been benchmarked with 11 other state-of-the-art deep semantic segmentation networks. For future work, the network performance can be increased by integrating synthetic augmented data to further optimize the training phase. Moreover, spatial and channel-based attention mechanisms can also be explored by emphasizing some strategic locations of the apples, which makes the recognition more accurate.

Automatic Segmentation and Quantification of Upper Airway Anatomic Risk Factors for Obstructive Sleep Apnea on Unprocessed Magnetic Resonance Images

Accurate segmentation of the upper airway lumen and surrounding soft tissue anatomy, especially tongue fat, using magnetic resonance images is crucial for evaluating the role of anatomic risk factors in the pathogenesis of obstructive sleep apnea (OSA). We present a convolutional neural network to automatically segment and quantify upper airway structures that are known OSA risk factors from unprocessed magnetic resonance images. Four datasets (n=[31, 35, 64, 76]) with T1-weighted scans and manually delineated labels of 10 regions of interest were used for model training and validations. We investigated a modified U-Net architecture that uses multiple convolution filter sizes to achieve multi-scale feature extraction. Validations included four-fold cross-validation and leave-study-out validations to measure generalization ability of the trained models. Automatic segmentations were also used to calculate the tongue fat ratio, a biomarker of OSA. Dice coefficient, Pearson's correlation, agreement analyses, and expert-derived clinical parameters were used to evaluate segmentations and tongue fat ratio values. Cross-validated mean Dice coefficient across all regions of interests and scans was 0.70 ± 0.10 with highest mean Dice coefficient in the tongue (0.89) and mandible (0.81). The accuracy was consistent across all four folds. Also, leave-study-out validations obtained comparable accuracy across uniquely acquired datasets. Segmented volumes and the derived tongue fat ratio values showed high correlation with manual measurements, with differences that were not statistically significant (p < 0.05). High accuracy of automated segmentations indicate translational potential of the proposed method to replace time consuming manual segmentation tasks in clinical settings and large-scale research studies.

Structural MRI-Based Schizophrenia Classification Using Autoencoders and 3D Convolutional Neural Networks in Combination with Various Pre-Processing Techniques.

Schizophrenia is a severe neuropsychiatric disease whose diagnosis, unfortunately, lacks an objective diagnostic tool supporting a thorough psychiatric examination of the patient. We took advantage of today’s computational abilities, structural magnetic resonance imaging, and modern machine learning methods, such as stacked autoencoders (SAE) and 3D convolutional neural networks (3D CNN), to teach them to classify 52 patients with schizophrenia and 52 healthy controls. The main aim of this study was to explore whether complex feature extraction methods can help improve the accuracy of deep learning-based classifiers compared to minimally preprocessed data. Our experiments employed three commonly used preprocessing steps to extract three different feature types. They included voxel-based morphometry, deformation-based morphometry, and simple spatial normalization of brain tissue. In addition to classifier models, features and their combination, other model parameters such as network depth, number of neurons, number of convolutional filters, and input data size were also investigated. Autoencoders were trained on feature pools of 1000 and 5000 voxels selected by Mann-Whitney tests, and 3D CNNs were trained on whole images. The most successful model architecture (autoencoders) achieved the highest average accuracy of 69.62% (sensitivity 68.85%, specificity 70.38%). The results of all experiments were statistically compared (the Mann-Whitney test). In conclusion, SAE outperformed 3D CNN, while preprocessing using VBM helped SAE improve the results.

Improved GCN Framework for Human Motion Recognition

Human recognition models based on spatial-temporal graph convolutional neural networks have been gradually developed, and we present an improved spatial-temporal graph convolutional neural network to solve the problems of the high number of parameters and low accuracy of this type of model. The method mainly draws on the inception structure. First, the tensor rotation is added to the graph convolution layer to realize the conversion between graph node dimension and channel dimension and enhance the model’s ability to capture global information for small-scale tasks. Then the inception temporal convolution layer is added to build a multiscale temporal convolution filter to perceive temporal information under different time domains hierarchically from 4-time dimensions. It overcomes the shortcomings of temporal graph convolutional networks in the field of joint relevance of hidden layers and compensates for the information omission of small-scale graph tasks. It also limits the volume of parameters, decreases the arithmetic power, and speeds up the computation. In our experiments, we verify our model on the public dataset NTU RGB + D. Our method reduces the number of the model parameters by 50% and achieves an accuracy of 90% in the CS evaluation system and 94% in the CV evaluation system. The results show that our method not only has high recognition accuracy and good robustness in human behavior recognition applications but also has a small number of model parameters, which can effectively reduce the computational cost.

Star topology convolution for graph representation learning

We present a novel graph convolutional method called star topology convolution (STC). This method makes graph convolution more similar to conventional convolutional neural networks (CNNs) in Euclidean feature spaces. STC learns subgraphs which have a star topology rather than learning a fixed graph like most spectral methods. Due to the properties of a star topology, STC is graph-scale free (without a fixed graph size constraint). It has fewer parameters in its convolutional filter and is inductive, so it is more flexible and can be applied to large and evolving graphs. The convolutional filter is learnable and localized, similar to CNNs in Euclidean feature spaces, and can share weights across graphs. To test the method, STC was compared with the state-of-the-art graph convolutional methods in a supervised learning setting on nine node properties prediction benchmark datasets: Cora, Citeseer, Pubmed, PPI, Arxiv, MAG, ACM, DBLP, and IMDB. The experimental results showed that STC achieved the state-of-the-art performance on all these datasets and maintained good robustness. In an essential protein identification task, STC outperformed the state-of-the-art essential protein identification methods. An application of using pretrained STC as the embedding for feature extraction of some downstream classification tasks was introduced. The experimental results showed that STC can share weights across different graphs and be used as the embedding to improve the performance of downstream tasks.

An FPGA Overlay for CNN Inference with Fine-grained Flexible Parallelism

Increasingly, pre-trained convolutional neural networks (CNNs) are being deployed for inference in various computer vision applications, both on the server-side in the data centers and at the edge. CNN inference is a very compute-intensive task. It is a challenge to meet performance metrics such as latency and throughput while optimizing power. Special-purpose ASICs and FPGAs are suitable candidates to meet these power and performance budgets simultaneously. Rapidly evolving CNN architectures involve novel convolution operations such as point convolutions, depth separable convolutions, and so on. This leads to substantial variation in the computational structure across CNNs and layers within a CNN. Because of this, FPGA reconfigurability provides an attractive tradeoff compared to ASICs. FPGA-based hardware designers address the structural variability issue by generating a network-specific accelerator for a single network or a class of networks. However, homogeneous accelerators are network agnostic and often sacrifice throughput and FPGA LUTs for flexibility. In this article, we propose an FPGA overlay for efficient processing of CNNs that can be scaled based on the available compute and memory resources of the FPGA. The overlay is configured on the fly through control words sent by the host on a per-layer basis. Unlike current overlays, our architecture exploits all forms of parallelism inside a convolution operation. A constraint system is employed at the host end to find out the per-layer configuration of the overlay that uses all forms of parallelism in the processing of the layer, resulting in the highest throughput for that layer. We studied the effectiveness of our overlay by using it to process AlexNet, VGG16, YOLO, MobileNet, and ResNet-50 CNNs targeting a Virtex7 and a bigger Ultrascale+VU9P FPGAs. The chosen CNNs have a mix of different types of convolution layers and filter sizes, presenting a good variation in model size and structure. Our accelerator reported a maximum throughput of 1,200 GOps/second on the Virtex7, an improvement of 1.2 \( \times \) to 5 \( \times \) over the recent designs. Also, the reported performance density, measured in giga operations per second per KLUT, is 1.3 \( \times \) to 4 \( \times \) improvement over existing works. Similar speed-up and performance density is also observed for the Ultrascale+VU9P FPGA.

A Novel Machine Learning–Based Gap-Filling of Fine-Resolution Remotely Sensed Snow Cover Fraction Data by Combining Downscaling and Regression

Abstract Satellite-based remotely sensed observations of snow cover fraction (SCF) can have data gaps in spatially distributed coverage from sensor and orbital limitations. We mitigate these limitations in the example fine-resolution Moderate Resolution Imaging Spectroradiometer (MODIS) data by gap-filling using auxiliary 1-km datasets that either aid in downscaling from coarser-resolution (5 km) MODIS SCF wherever not fully covered by clouds, or else by themselves via regression wherever fully cloud covered. This study’s prototype predicts a 1-km version of the 500-m MOD10A1 SCF target. Due to noncollocatedness of spatial gaps even across input and auxiliary datasets, we consider a recent gap-agnostic advancement of partial convolution in computer vision for both training and predictive gap-filling. Partial convolution accommodates spatially consistent gaps across the input images, effectively implementing a two-dimensional masking. To overcome reduced usable data from noncollocated spatial gaps across inputs, we innovate a fully generalized three-dimensional masking in this partial convolution. This enables a valid output value at a pixel even if only a single valid input variable and its value exist in the neighborhood covered by the convolutional filter zone centered around that pixel. Thus, our gap-agnostic technique can use significantly more examples for training (∼67%) and prediction (∼100%), instead of only less than 10% for the previous partial convolution. We train an example simple three-layer legacy super-resolution convolutional neural network (SRCNN) to obtain downscaling and regression component performances that are better than baseline values of either climatology or MOD10C1 SCF as relevant. Our generalized partial convolution can enable multiple Earth science applications like downscaling, regression, classification, and segmentation that were hindered by data gaps.

Activity Classification Based on Feature Fusion of FMCW Radar Human Motion Micro-Doppler Signatures

Fall is a challenging task that poses a great danger to the elderly person’s health as they carry out their daily routines and activities and could lead to serious injuries, long hospitalization, or even death. One of the key solutions to this important problem is the prompt and automatic detection of their fall motion, among other activities, so that immediate help can be rendered to avoid further complications. Cameras and other wearable sensors are the conventional means employed to monitor the daily activities performed by elderly people. However, recently radar sensing technology has been favored largely due to its short-range and velocity estimation. And most importantly, their ability to remotely capture the human activity motion electromagnetic wave signal without infringing on the user’s privacy. To this end, we present an elderly people fall detection system that integrates the radar signal micro-Doppler features extracted with the convolutional neural networks (CNNs). The features were extracted using Alex-Net, VGG-16-Net, and VGG-19-Net pre-trained models. We adapted the canonical correlation analysis (CCA) algorithm by proposing a channel attention network to fuse the extracted features effectively. The channel attention module employed a series of convolutional filters to determine the most discriminative features to focus on and discard the redundant ones. The fused features are classified using an SVM classifier. Our proposed method achieved the best performance against the state-of-the-art approaches. Specifically, our approach attained 99.77% test accuracy, which is about a 2% to 4 % increase compared to the recent state-of-the-art approaches.

Visual sensitivity filtering based local maximum variation prior for blind image deblurring

Blind image deblurring aims to recover the latent sharp image from a given blurred image when the blur kernel is unknown. To solve this ill-posed problem, many researchers have studied various priors to guide the deblurring process. However, whether the image is sharp or not is mostly based on the subjective judgment of human eyes, and there is no unified objective standard. Since the response of blur to visual sensitivity can be modeled by a filtering method, we use human visual system (HVS) convolution filter to filter the target image to simulate the corresponding visual sensitivity. At the same time, we find that the local maximum variation values of the filtered image will be reduced after blurring process. Therefore, we propose a local maximum variation prior (LMV) based on visual sensitivity filtering and construct a new energy function according to LMV. Our method performs well in various scenarios. Compared with the most advanced algorithms, our model achieves good results on both benchmark datasets and natural images.

Retinopathy grading with deep learning and wavelet hyper-analytic activations.

Recent developments reveal the prominence of Diabetic Retinopathy (DR) grading. In the past few decades, Wavelet-based DR classification has shown successful impacts and the Deep Learning models, like Convolutional Neural Networks (CNN’s), have evolved in offering the highest prediction accuracy. In this work, the features of the input image are enhanced with the integration of Multi-Resolution Analysis (MRA) and a CNN framework without costing more convolution filters. The bottleneck with conventional activation functions, used in CNN’s, is the nullification of the feature maps that are negative in value. In this work, a novel Hyper-analytic Wavelet (HW) phase activation function is formulated with unique characteristics for the wavelet sub-bands. Instead of dismissal, the function transforms these negative coefficients that correspond to significant edge feature maps. The hyper-analytic wavelet phase forms the imaginary part of the complex activation. And the hyper-parameter of the activation function is selected such that the corresponding magnitude spectrum produces monotonic and effective activations. The performance of 3 CNN models (1 custom, shallow CNN, ResNet with Soft attention, Alex Net for DR) with spatial–Wavelet quilts is better. With the spatial–Wavelet quilts, the Alex Net for DR has an improvement with an 11% of accuracy level (from 87 to 98%). The highest accuracy level of 98% and the highest Sensitivity of 99% are attained through Modified Alex Net for DR. The proposal also illustrates the visualization of the negative edge preservation with assumed image patches. From this study, the researcher infers that models with spatial–Wavelet quilts, with the hyper-analytic activations, have better generalization ability. And the visualization of heat maps provides evidence of better learning of the feature maps from the wavelet sub-bands.

Automatic Evaluation of Motor Rehabilitation Exercises Based on Deep Mixture Density Neural Networks

An automatic assessment system for physical telerehabilitation could reduce the time and cost of treatments. But such assessment involves stochastic uncertainties, nonlinearities, and complexities of human movement. Probabilistic models and deep structures are two categories that could, respectively, address the stochastic uncertainty and complexity of motion data. In the proposed Deep Mixture Density Network (DMDN), probabilistic models were concurrently processed along with deep neural networks. More specifically, a multi-branch convolutional layer extracted the deep features of motion data, a Long Short Term Memory (LSTM) learned its temporal dependencies, and a Gaussian Mixture Model (GMM) handled the stochastic interaction of its preceding layers in reaching a more valid assessment and improved generalization to new movements. Finally, the weighted mean of the GMM components was used as the performance score for exercises. Input data were the time series related to the joints' position and orientation extracted from the Kinect v2 sensor video. A clinical reference score for each movement was also included for training the DMDN. In addition to comparisons with the state-of-the-art algorithms, an ablation study of the various elements comprising the DMDN was performed. Three configurations of convolutional filter window transitions across input data were also investigated. Results indicate that the proposed DMDN with one-dimensional parallel window transitions outperforms the other competing strategies in the ablation study. It also achieves higher reliability in terms of a lower RMSE standard deviation against a DMDN without GMM strategy while ranking competitively according to the Spearman correlation coefficient and Root Mean Square Error.

LightEyes: A Lightweight Fundus Segmentation Network for Mobile Edge Computing.

Fundus is the only structure that can be observed without trauma to the human body. By analyzing color fundus images, the diagnosis basis for various diseases can be obtained. Recently, fundus image segmentation has witnessed vast progress with the development of deep learning. However, the improvement of segmentation accuracy comes with the complexity of deep models. As a result, these models show low inference speeds and high memory usages when deploying to mobile edges. To promote the deployment of deep fundus segmentation models to mobile devices, we aim to design a lightweight fundus segmentation network. Our observation comes from the fact that high-resolution representations could boost the segmentation of tiny fundus structures, and the classification of small fundus structures depends more on local features. To this end, we propose a lightweight segmentation model called LightEyes. We first design a high-resolution backbone network to learn high-resolution representations, so that the spatial relationship between feature maps can be always retained. Meanwhile, considering high-resolution features means high memory usage; for each layer, we use at most 16 convolutional filters to reduce memory usage and decrease training difficulty. LightEyes has been verified on three kinds of fundus segmentation tasks, including the hard exudate, the microaneurysm, and the vessel, on five publicly available datasets. Experimental results show that LightEyes achieves highly competitive segmentation accuracy and segmentation speed compared with state-of-the-art fundus segmentation models, while running at 1.6 images/s Cambricon-1A speed and 51.3 images/s GPU speed with only 36k parameters.

A deep grouping fusion neural network for multimedia content understanding

How Deep Neural Networks (DNNs) best cope with the understanding of multimedia contents still remains an open problem, mainly due to two factors. First, conventional DNNs cannot effectively learn the representations of the images with sparse visual information. For example, the images describing knowledge concepts in textbooks. Second, existing DNNs cannot effectively capture the fine-grained interactions between the images and text descriptions. To address these issues, we propose a deep Cross-Media Grouping Fusion Network (CMGFN), which mainly has two distinctive properties: 1) CMGFN can effectively learn visual features from the images with sparse visual information. This is achieved by first progressively adjusting the attention of convolution filters to valuable visual regions, and then enhancing the use of key visual information in feature construction. 2) By a cross-media grouping co-attention mechanism, CMGFN can effectively use the interactions between visual features of different semantics and textual descriptions, to learn cross-media features representing different fine-grained semantics in different groups. Empirical studies demonstrate that CMGFN not only achieves state-of-the-art performance on the multimedia documents containing sparse visual information, but also shows superior general applicability on other multimedia data, e.g., the multimedia fake news.