Enormous Parameters Research Articles

DSMT-1DCNN: Densely supervised multitask 1DCNN for fault diagnosis

Deep learning has greatly succeeded in fault diagnosis (FD). However, most existing methods require enormous parameters to train the model, limiting its usage on lightweight terminals. Besides, they ignore the correlations between FD and other tasks that reduce performance. Therefore, this article presents a lightweight, end-to-end framework, densely supervised multitask one-dimensional convolution neural network (DSMT-1DCNN) for FD using raw signals. The DSMT-1DCNN designs three blocks, including the main network, densely supervised network, and multitask learning blocks, to extract rich, distinguishable, and less-noise hidden features for FD more accurately. Firstly, the main network block (MNB) applies 1DCNN to extract common features shared by different tasks from raw signals. Secondly, a novel supervised learning scheme (SLS) is designed in the densely supervised block (DSB) to supervise each 1-D convolution layer to learn hidden features thoroughly in a lightweight mode. In DSB, each convolution layer learned features are fused as the fusion sharing features. Finally, the fusion-sharing features are fed into the multitask learning block (MTLB) for FD. It utilizes two auxiliary tasks, including the speed identification branch (SIB) and load identification branch (LIB), to assist the FD branch (FDB) in extracting rich distinguishable features that consider their correlations. Therefore, the comprehensive feature utilized for FD is a fusion of densely CNN features, speed, and load information, which enables FD to be more accurate. The comparative experimental results and analysis on three public subsets confirmed its state-of-the-art in terms of accuracy and model size. Also, we proved that SLS has a good generality based on three 1DCNN structures.

COVID-19 detection from chest X-ray images using CLAHE-YCrCb, LBP, and machine learning algorithms

BackgroundCOVID-19 is a disease that caused a contagious respiratory ailment that killed and infected hundreds of millions. It is necessary to develop a computer-based tool that is fast, precise, and inexpensive to detect COVID-19 efficiently. Recent studies revealed that machine learning and deep learning models accurately detect COVID-19 using chest X-ray (CXR) images. However, they exhibit notable limitations, such as a large amount of data to train, larger feature vector sizes, enormous trainable parameters, expensive computational resources (GPUs), and longer run-time.ResultsIn this study, we proposed a new approach to address some of the above-mentioned limitations. The proposed model involves the following steps: First, we use contrast limited adaptive histogram equalization (CLAHE) to enhance the contrast of CXR images. The resulting images are converted from CLAHE to YCrCb color space. We estimate reflectance from chrominance using the Illumination–Reflectance model. Finally, we use a normalized local binary patterns histogram generated from reflectance (Cr) and YCb as the classification feature vector. Decision tree, Naive Bayes, support vector machine, K-nearest neighbor, and logistic regression were used as the classification algorithms. The performance evaluation on the test set indicates that the proposed approach is superior, with accuracy rates of 99.01%, 100%, and 98.46% across three different datasets, respectively. Naive Bayes, a probabilistic machine learning algorithm, emerged as the most resilient.ConclusionOur proposed method uses fewer handcrafted features, affordable computational resources, and less runtime than existing state-of-the-art approaches. Emerging nations where radiologists are in short supply can adopt this prototype. We made both coding materials and datasets accessible to the general public for further improvement. Check the manuscript’s availability of the data and materials under the declaration section for access.

Gibbon: An Efficient Co-Exploration Framework of NN Model and Processing-In-Memory Architecture

The memristor-based Processing-In-Memory (PIM) architectures have been proven to be a potential architecture to store enormous parameters and execute the complicated computations of Deep Neural Networks (DNNs) efficiently. Existing PIM studies focus on designing high energy-efficient hardware architecture and algorithm-hardware co-optimization for better performance. However, the impacts of the algorithms and hardware architectures on the performance intersect with each other. Only optimizing the algorithms or the hardware architectures can not realize the optimal design. Therefore, the co-exploration of NN models and PIM architecture is necessary. However, for one thing, the co-exploration space size of NN models and PIM architectures is extremely huge, and is challenging to search. For another, during the co-exploration process, time-consuming PIM simulators are needed to evaluate various design candidates and pose a heavy time burden. To tackle these problems, we propose an efficient co-exploration framework of NN models and PIM architectures, named . In, the co-exploration space is carefully designed to adapt both NN models and PIM architectures. Besides, in order to improve search efficiency, we propose an evolutionary search algorithm with adaptive parameter priority (ESAPP). In addition, introduces a multi-level joint simulator to alleviate the problem of time-consuming evaluation. The experimental results show that the proposed co-exploration framework can find better NN models and PIM architectures than existing studies in only six GPU hours (9.8 48.2× speedup). At the same time, can improve the accuracy of co-design results by 15.3% and reduce the energy-delay-product (EDP) by 5.96× compared with existing work.

Fast and Accurate NoC Latency Estimation for Application-Specific Traffics via Machine Learning

Latency is one of the critical performance metrics for Networks-on-Chips (NoCs). When designing an NoC, the designers have to explore enormous design parameters and various traffic patterns, thus a fast and accurate latency estimator is essential to explore the large design space. In this paper, we present an ideal neural network-based methodology for latency estimation in NoCs, especially for application-specific traffics. By inputting the sequence of extracted traffic features and NoC parameters, the neural network model will infer the corresponding average network latency in a fast while accurate way. Instead of training one neural network model for each benchmark from scratch, we adopt transfer learning to train the network model for a new benchmark from another trained one. Experimental results on a set of widely used application-specific NoC benchmarks show that, our method can achieve an average estimation accuracy of 95%, and a 17.1X speedup for large NoCs compared to BookSim2 simulations. Our method can also achieve 20% to 70% improvement in accuracy over the other state-of-art machine learning-based works.

A CNN Compression Method via Dynamic Channel Ranking Strategy

In recent years, the rapid development of mobile devices and embedded system raises a demand for intelligent models to address increasingly complicated problems. However, the complexity of the structure and extensive parameters press significantly on efficiency, storage space, and energy consumption. Additionally, the explosive growth of tasks with enormous model structures and parameters makes it impossible to compress models manually. Thus, a standardized and effective model compression solution achieving lightweight neural networks is established as an urgent demand by the industry. Accordingly, Dynamic Channel Ranking Strategy (DCRS) method is proposed to compress deep convolutional neural networks. DCRS selects channels with high contribution of each prunable layer according to compression ratio searched by reinforcement learning agent. Compared with current model compression methods, DCRS efficaciously applies various channel ranking strategies on prunable layers. Experiments indicate with a 50% compression ratio, compressed MobileNet achieved 70.62% top1 and 88.2% top5 accuracy on ImageNet, and compressed ResNet achieved 92.03% accuracy on CIFAR-10. DCRS reduces more FLOPS in these neural networks. The compressed model achieves the best Top-1 and Top-5 accuracy on ResNet50, the best Top-1 accuracy on MobilNetV1.

MLNet: A Multi-Domain Lightweight Network for Multi-Focus Image Fusion

Existing multi-focus image fusion (MFIF) methods are difficult to achieve satisfactory results in both fusion performance and rate simultaneously. The spatial domain methods are hard to determine the focus/defocus boundary (FDB), and the transform domain methods are likely to damage the content information of the source images. Moreover, the deep learning-based MFIF methods are usually confronted with low rate due to complex models and enormous learnable parameters. To address these issues, we propose a multi-domain lightweight network (MLNet) for MFIF, which can achieve competitive results in both performance and rate. The proposed MLNet mainly includes three modules, namely focus extraction (FE), focus measure (FM) and image fusion (IF). In the interpretable FE module, the image features extracted by discrete cosine transform-based convolution (DCTConv) and local binary pattern-based convolution (LBPConv) are concatenated and fed into the FM module. DCTConv based on transform domain takes DCT coefficients to construct a fixed convolution kernel without parameter learning, which can effectively capture the high/low frequency content of the image. LBPConv based on spatial domain can achieve structure features and gradient information from source images. In the FM module, a 3-layer 1×1 convolution with a few learnable parameters is employed to generate the initial decision map, which has the properties of flexible input. The fused image is obtained by the IF module according to the final decision map. In terms of quantitative and qualitative evaluations, extensive experiments validate that the proposed method outperforms existing state-of-the-art methods on three public datasets. In addition, the proposed MLNet contains only 0.01M parameters, which is 0.2% of the first CNN-based MFIF method [25].

Video‐based action recognition using spurious‐3D residual attention networks

Recently, 3D Convolutional Neural Networks (3D CNNs) have attracted extensive attention in extracting spatial and temporal features in videos for their efficient feature extraction ability. However, it also brings enormous model parameters by training very deep 3D CNNs. Here, a novel network named spurious-3D Residual Attention Networks (S3D RANs) is proposed for video-based action recognition, which has the powerful capacity to learn collaborative spatiotemporal features. In particular, by leveraging the merits from 2D Convolutional Neural Networks (2D CNNs) and 3D CNNs, 2D CNNs are applied rather than 3D CNNs on frames of the single view of volumetric videos data to learn temporal motion features directly. Furthermore, view and channel-wise attention mechanism submodules are employed in the residual unit to learn the importance of each view for action recognition and guide the network to pay more attention to the more useful information for action recognition. Experimental results on UCF-101, HMDB-51 datasets demonstrate that our S3D RANs have higher accuracy and lower model complexity than existing works.

Domain Adaptation for Object Classification in Point Clouds via Asymmetrical Siamese and Conditional Adversarial Network

Nowadays, researchers have developed various deep neural networks for processing point clouds effectively. Due to the enormous parameters in deep learning-based models, a lot of manual efforts have to be invested into annotating sufficient training samples. To mitigate such manual efforts of annotating samples for a new scanning device, this letter focuses on proposing a new neural network to achieve domain adaptation in 3D object classification. Specifically, to minimize the data discrepancy of intra-class objects in different domains, an Asymmetrical Siamese module is designed to align the intra-class features. To preserve the discriminative information for distinguishing inter-class objects in different domains, a Conditional Adversarial module is leveraged to consider the classification information conveyed from the classifier. To verify the effectiveness of the proposed method on object classification in heterogeneous point clouds, evaluations are conducted on three point cloud datasets, which are collected in different scenarios by different laser scanning devices. Furthermore, the comparative experiments also demonstrate the superior performance of the proposed method on the classification accuracy.

Optimization of green sand process for quality improvement in castings by using combination of Taguchi Techniques-GRA-PCA

The castings produced by a sand casting process have multiple quality features. The quality of a casting always depends upon proper controlling of enormous parameters involved in the process as well as proper combination among these parameters, while some of the defects are found persisting and difficult to avoid. Controlling the process to avoid the rejections of cast components due to the presence of such persisting defects mainly depends on the right understanding of the parameters associated with it. In the work presented in this paper, the primary focus is on the optimization of essential parameters of the green sand process to minimize the occurrences of these persisting defects in gray iron automotive components through experimental investigation using effective approach. The study applies the Taguchi’s ‘Design of Experiments’ approach in combination with grey relational analysis (GRA) and principal component analysis (PCA) for determining the optimal level of parameters to minimize the defects Shrinkage porosity (SP), blowholes (BH), sand inclusion (SI), scabbing (SC) and low hardness (LH), which are persisting in the foundry industry producing cylinder heads and other castings required in the automotive, factory situated in central India. Optimal levels of parameters were determined by the single grey relational grade (GRG) obtained from GRA for overall improvement in multi-quality characteristics. PCA is used to determine the corresponding weighting values of each performance. The outcomes of this study assure that the approach used in this work, is useful to foundry industries to minimize the persisting casting defects.

Retracted] Research on Efficient Deep Learning Algorithm Based on ShuffleGhost in the Field of Virtual Reality

Deep learning algorithms are facing the limitation in virtual reality application due to the cost of memory, computation, and real‐time computation problem. Models with rigorous performance might suffer from enormous parameters and large‐scale structure, and it would be hard to replant them onto embedded devices. In this paper, with the inspiration of GhostNet, we proposed an efficient structure ShuffleGhost to make use of the redundancy in feature maps to alleviate the cost of computations, as well as tackling some drawbacks of GhostNet. Since GhostNet suffers from high computation of convolution in Ghost module and shortcut, the restriction of downsampling would make it more difficult to apply Ghost module and Ghost bottleneck to other backbone. This paper proposes three new kinds of ShuffleGhost structure to tackle the drawbacks of GhostNet. The ShuffleGhost module and ShuffleGhost bottlenecks are utilized by the shuffle layer and group convolution from ShuffleNet, and they are designed to redistribute the feature maps concatenated from Ghost Feature Map and Primary Feature Map. Besides, they eliminate the gap of them and extract the features. Then, SENet layer is adopted to reduce the computation cost of group convolution, as well as evaluating the importance of the feature maps which concatenated from Ghost Feature Maps and Primary Feature Maps and giving proper weights for the feature maps. This paper conducted some experiments and proved that the ShuffleGhostV3 has smaller trainable parameters and FLOPs with the ensurance of accuracy. And with proper design, it could be more efficient in both GPU and CPU side.

Learning-Free Unsupervised Extractive Summarization Model

Text summarization is an information condensation technique that abbreviates a source document to a few representative sentences with the intention to create a coherent summary containing relevant information of source corpora. This promising subject has been rapidly developed since the advent of deep learning. However, summarization models based on deep neural network have several critical shortcomings. First, a large amount of labeled training data is necessary. This problem is standard for low-resource languages in which publicly available labeled data do not exist. In addition, a significant amount of computational ability is required to train neural models with enormous network parameters. In this study, we propose a model called Learning Free Integer Programming Summarizer (LFIP-SUM), which is an unsupervised extractive summarization model. The advantage of our approach is that parameter training is unnecessary because the model does not require any labeled training data. To achieve this, we formulate an integer programming problem based on pre-trained sentence embedding vectors. We also use principal component analysis to automatically determine the number of sentences to be extracted and to evaluate the importance of each sentence. Experimental results demonstrate that the proposed model exhibits generally acceptable performance compared with deep learning summarization models although it does not learn any parameters during the model construction process.

Small Network for Lightweight Task in Computer Vision: A Pruning Method Based on Feature Representation.

Many current convolutional neural networks are hard to meet the practical application requirement because of the enormous network parameters. For accelerating the inference speed of networks, more and more attention has been paid to network compression. Network pruning is one of the most efficient and simplest ways to compress and speed up the networks. In this paper, a pruning algorithm for the lightweight task is proposed, and a pruning strategy based on feature representation is investigated. Different from other pruning approaches, the proposed strategy is guided by the practical task and eliminates the irrelevant filters in the network. After pruning, the network is compacted to a smaller size and is easy to recover accuracy with fine-tuning. The performance of the proposed pruning algorithm is validated on the acknowledged image datasets, and the experimental results prove that the proposed algorithm is more suitable to prune the irrelevant filters for the fine-tuning dataset.

Gaussian Processes Proxy Model with Latent Variable Models and Variogram-Based Sensitivity Analysis for Assisted History Matching

Reservoir history matching is a well-known inverse problem for production prediction where enormous uncertain reservoir parameters of a reservoir numerical model are optimized by minimizing the misfit between the simulated and history production data. Gaussian Process (GP) has shown promising performance for assisted history matching due to the efficient nonparametric and nonlinear model with few model parameters to be tuned automatically. Recently introduced Gaussian Processes proxy models and Variogram Analysis of Response Surface-based sensitivity analysis (GP-VARS) uses forward and inverse Gaussian Processes (GP) based proxy models with the VARS-based sensitivity analysis to optimize the high-dimensional reservoir parameters. However, the inverse GP solution (GPIS) in GP-VARS are unsatisfactory especially for enormous reservoir parameters where the mapping from low-dimensional misfits to high-dimensional uncertain reservoir parameters could be poorly modeled by GP. To improve the performance of GP-VARS, in this paper we propose the Gaussian Processes proxy models with Latent Variable Models and VARS-based sensitivity analysis (GPLVM-VARS) where Gaussian Processes Latent Variable Model (GPLVM)-based inverse solution (GPLVMIS) instead of GP-based GPIS is provided with the inputs and outputs of GPIS reversed. The experimental results demonstrate the effectiveness of the proposed GPLVM-VARS in terms of accuracy and complexity. The source code of the proposed GPLVM-VARS is available at https://github.com/XinweiJiang/GPLVM-VARS.

UL-CNN: An Ultra-Lightweight Convolutional Neural Network Aiming at Flash-Based Computing-In-Memory Architecture for Pedestrian Recognition

Pedestrian recognition has achieved the state-of-the-art performance due to the progress of recent convolutional neural network (CNN). However, mainstream CNN models are too complicated to emerging Computing-In-Memory (CIM) architectures for hardware implementation, because enormous parameters and massive intermediate processing results may incur severe “memory bottleneck”. This paper proposed a design methodology of Parameter Substitution with Nodes Compensation (PSNC) to significantly reduce parameters of CNN model without inference accuracy degradation. Based on the PSNC methodology, an ultra-lightweight convolutional neural network (UL-CNN) was designed. The UL-CNN model is a specially optimized convolutional neural network aiming at a flash-based CIM architecture (Conv-Flash) and to apply for recognizing person. The implementation result of running UL-CNN on Conv-Flash shows that the inference accuracy is up to 94.7%. Compared to LeNet-5, on the premise of the similar operations and accuracy, the amounts of UL-CNN’s parameters are less than 37% of LeNet-5 at the same dataset benchmark. Such parameter reduction can dramatically speed up the training process and economize on-chip storage overhead, as well as save the power consumption of the memory access. With the aid of UL-CNN, the Conv-Flash architecture can provide the best energy efficiency compared to other platforms (CPU, GPU, FPGA, etc.), which consumes only 2.2 × 105J to complete pedestrian recognition for one frame.

Combining weighted category-aware contextual information in convolutional neural networks for text classification

Convolutional neural networks (CNNs) are widely used in many natural language processing tasks, which employ some convolutional filters to capture useful semantic features of a text. However, a small window size convolutional filter is short of the ability to capture contextual information, simply increasing the window size may bring the problems of data sparsity and enormous parameters. To capture the contextual information, we propose to use the weighted sum operation to obtain contextual word representation. We present one implicit weighting method and two explicit category-aware weighting methods to assign the weights of the contextual information. Experimental results on five text classification datasets show the effectiveness of our proposed methods.

Entropy Generation and Dual Solutions in Mixed Convection Stagnation Point Flow of Micropolar Ti6Al4V Nanoparticle along a Riga Surface

Entropy generation and dual solutions are rarely studied in the literature. An analysis is attempted here. More exactly, the present paper looks at the impact of radiation of a micropolar fluid on mixed convective flow containing the titanium alloy Ti6Al4V nanoparticle along with a Riga plate. The study of dual-nature solution for the entropy generation along a Riga surface was not being explored in the literature; therefore, the current model focuses on the dual solutions of this complex nature model. Riga surface is identified as an actuator of electromagnetic in which electrodes are accumulated alternatively. This array produces the behavior of electromagnetic hydrodynamic in the flow field. The transmuted leading equations were worked out through the formula of 3-stage Lobatto IIIA. Influences of exercising enormous parameters on temperature distribution, velocity, and micro rotation fields are portrayed and argued. More than one solution is achieved in opposing flow, while in the phenomenon of assisting flow result is unique. Moreover, due to the micropolar parameter, the separation of the boundary layer is decelerating. It is determined that the entire structure produces the dual-nature solution of the phenomenon of stagnation point flow, and the temperature profile behavior shows the significant enhancement in the thermal conductivity due to the addition of the nanoparticle. The results exposed that liquid velocity is enhanced, and micro rotation is decelerated, by improving the values of Hartmann numbers in both solutions, whereas the temperature field is decelerated in the first solution and accelerated in the second solution.

A Novel Residual Dense Pyramid Network for Image Dehazing

Recently, convolutional neural network (CNN) based on the encoder-decoder structure have been successfully applied to image dehazing. However, these CNN based dehazing methods have two limitations: First, these dehazing models are large in size with enormous parameters, which not only consumes much GPU memory, but also is hard to train from scratch. Second, these models, which ignore the structural information at different resolutions of intermediate layers, cannot capture informative texture and edge information for dehazing by stacking more layers. In this paper, we propose a light-weight end-to-end network named the residual dense pyramid network (RDPN) to address the above problems. To exploit the structural information at different resolutions of intermediate layers fully, a new residual dense pyramid (RDP) is proposed as a building block. By introducing a dense information fusion layer and the residual learning module, the RDP can maximize the information flow and extract local features. Furthermore, the RDP further learns the structural information from intermediate layers via a multiscale pyramid fusion mechanism. To reduce the number of network parameters and to ease the training process, we use one RDP in the encoder and two RDPs in the decoder, following a multilevel pyramid pooling layer for incorporating global context features before estimating the final result. The extensive experimental results on a synthetic dataset and real-world images demonstrate that the new RDPN achieves favourable performance compared with some state-of-the-art methods, e.g., the recent densely connected pyramid dehazing network, the all-in-one dehazing network, the enhanced pix2pix dehazing network, pixel-based alpha blending, artificial multi-exposure image fusions and the genetic programming estimator, in terms of accuracy, run time and number of parameters. To be specific, RDPN outperforms all of the above methods in terms of PSNR by at least 4.25 dB. The run time of the proposed method is 0.021 s, and the number of parameters is 1,534,799, only 6% of that used by the densely connected pyramid dehazing network.

SCRSR: An efficient recursive convolutional neural network for fast and accurate image super-resolution

Convolutional neural networks have recently demonstrated high-quality reconstruction for single image super-resolution (SR). These CNN networks effectively recover a high-resolution (HR) image from a low-resolution (LR) image, at the cost of enormous parameters and heavy computational burden. In this work, we propose a recursive efficient deep convolutional network for fast and accurate single-image SR with only 0.28M parameters. A Split-Concatenate-Residual (SCR) block is proposed to reduce computation and parameters. With downsampling block and upsampling block, we significantly reduce computational complexity and enlarge the size of the receptive field. Specifically, two-level recursive learning is proposed which can improve accuracy by increasing depth without adding any weight parameters. We also employ local, semi-global and global residual techniques to train our very deep network steadily and improve its performance. Extensive experiments indicate that our proposed method Split-Concatenate-Residual Super Resolution (SCRSR) yields promising SR performance while maintaining shorter running time and fewer parameters.

Bidirectional Convolutional LSTM Neural Network for Remote Sensing Image Super-Resolution

Single-image super-resolution (SR) is an effective approach to enhance spatial resolution for numerous applications such as object detection and classification when the resolution of sensors is limited. Although deep convolutional neural networks (CNNs) proposed for this purpose in recent years have outperformed relatively shallow models, enormous parameters bring the risk of overfitting. In addition, due to the different scale of objects in images, the hierarchical features of deep CNN contain additional information for SR tasks, while most CNN models have not fully utilized these features. In this paper, we proposed a deep yet concise network to address these problems. Our network consists of two main structures: (1) recursive inference block based on dense connection reuse of local low-level features, and recursive learning is applied to control the model parameters while increasing the receptive fields; (2) a bidirectional convolutional LSTM (BiConvLSTM) layer is introduced to learn the correlations of features from each recursion and adaptively select the complementary information for the reconstruction layer. Experiments on multispectral satellite images, panchromatic satellite images, and nature high-resolution remote-sensing images showed that our proposed model outperformed state-of-the-art methods while utilizing fewer parameters, and ablation studies demonstrated the effectiveness of a BiConvLSTM layer for an image SR task.

Lightweight Feature Fusion Network for Single Image Super-Resolution

Single image super-resolution(SISR) has witnessed great progress as convolutional neural network(CNN) gets deeper and wider. However, enormous parameters hinder its application to real world problems. In this letter, We propose a lightweight feature fusion network (LFFN) that can fully explore multi-scale contextual information and greatly reduce network parameters while maximizing SISR results. LFFN is built on spindle blocks and a softmax feature fusion module (SFFM). Specifically, a spindle block is composed of a dimension extension unit, a feature exploration unit and a feature refinement unit. The dimension extension layer expands low dimension to high dimension and implicitly learns the feature maps which is suitable for the next unit. The feature exploration unit performs linear and nonlinear feature exploration aimed at different feature maps. The feature refinement layer is used to fuse and refine features. SFFM fuses the features from different modules in a self-adaptive learning manner with softmax function, making full use of hierarchical information with a small amount of parameter cost. Both qualitative and quantitative experiments on benchmark datasets show that LFFN achieves favorable performance against state-of-the-art methods with similar parameters.