Multi-column Convolutional Neural Network Research Articles

LettuceNet: A Novel Deep Learning Approach for Efficient Lettuce Localization and Counting

Traditional lettuce counting relies heavily on manual labor, which is laborious and time-consuming. In this study, a simple and efficient method for localization and counting lettuce is proposed, based only on lettuce field images acquired by an unmanned aerial vehicle (UAV) equipped with an RGB camera. In this method, a new lettuce counting model based on the weak supervised deep learning (DL) approach is developed, called LettuceNet. The LettuceNet network adopts a more lightweight design that relies only on point-level labeled images to train and accurately predict the number and location information of high-density lettuce (i.e., clusters of lettuce with small planting spacing, high leaf overlap, and unclear boundaries between adjacent plants). The proposed LettuceNet is thoroughly assessed in terms of localization and counting accuracy, model efficiency, and generalizability using the Shanghai Academy of Agricultural Sciences-Lettuce (SAAS-L) and the Global Wheat Head Detection (GWHD) datasets. The results demonstrate that LettuceNet achieves superior counting accuracy, localization, and efficiency when employing the enhanced MobileNetV2 as the backbone network. Specifically, the counting accuracy metrics, including mean absolute error (MAE), root mean square error (RMSE), normalized root mean square error (nRMSE), and coefficient of determination (R2), reach 2.4486, 4.0247, 0.0276, and 0.9933, respectively, and the F-Score for localization accuracy is an impressive 0.9791. Moreover, the LettuceNet is compared with other existing widely used plant counting methods including Multi-Column Convolutional Neural Network (MCNN), Dilated Convolutional Neural Networks (CSRNets), Scale Aggregation Network (SANet), TasselNet Version 2 (TasselNetV2), and Focal Inverse Distance Transform Maps (FIDTM). The results indicate that our proposed LettuceNet performs the best among all evaluated merits, with 13.27% higher R2 and 72.83% lower nRMSE compared to the second most accurate SANet in terms of counting accuracy. In summary, the proposed LettuceNet has demonstrated great performance in the tasks of localization and counting of high-density lettuce, showing great potential for field application.

An intelligent skin cancer detection system using two-level multi-column convolutional neural network architecture

An intelligent skin cancer detection system using two-level multi-column convolutional neural network architecture

Multi Column Convolutional Neural Network for Accurate Crowd Counting and Analysis in Highly Congested Urban Scenes

Multi Column Convolutional Neural Network for Accurate Crowd Counting and Analysis in Highly Congested Urban Scenes

Image Object and Scene Recognition Based on Improved Convolutional Neural Network

In recent years, due to the continuous optimization of network structure and the emergence of large-scale data, Convolutional neural network has made breakthroughs in a series of applications of computer vision. Based on this, the Convolutional neural network is improved and optimized. The improved convolutional neural network is introduced into image Object detection and scene recognition, and image object detection is carried out by combining sliding window fusion and Convolutional neural network. The image scene recognition model is constructed by using potential object area recognition and Convolutional neural network Transfer learning. Using different data sets to verify the algorithm, the research results show that in Group1 and Group2, the error rate of the multi column convolutional neural network fused by sliding window is reduced by about 25% compared with the single column convolutional neural network. As the group with the smallest decrease in error rate, Group3 also achieved a 9% decrease in error rate. The fitness rate of object detection algorithm is gradually stable after 7 runs, reaching about 9.8%, and its operation effect is obviously better than other algorithms. The multi column convolutional neural network fused by sliding window is more adaptive to the training data set, and gets better recognition effect in the algorithm operation. However, the image scene recognition model based on potential object area recognition algorithm and Convolutional neural network has good convergence. The average recognition time for image scenes is 1.5356s. The recognition speed is fast and stable, which can effectively solve the problem of multi-scale image scene recognition.

Label-free quantification of gold nanoparticles at the single-cell level using a multi-column convolutional neural network (MC-CNN).

Gold nanoparticles (AuNPs) are extensively used in cellular imaging, single-particle tracking, disease diagnosis, studying membrane protein interaction, and drug delivery. Understanding the dynamics of AuNP uptake in live cells is crucial for optimizing their efficacy and safety. Traditional manual methods for quantifying AuNP uptake are time-consuming and subjective, limiting their scalability and accuracy. The available fluorescence-based techniques are limited to photobleaching and photoblinking. Optical microscopy techniques are limited by diffraction limits. Electron microscopy-based imaging techniques are destructive and unsuitable for live cell imaging. Furthermore, the resulting images may contain hundreds of particles with varied intensities, blurring, and substantial occlusion, making it difficult to manually quantify AuNP uptake. To overcome this issue and measure AuNP uptake by live cells, we annotated a dataset of dark-field images of 50 nanometer-radius AuNPs at different incubation durations. Then, to count the number of particles present in a cell, we created a customized multi-column convolutional neural network (MC-CNN). The customized MC-CNN outperformed typical particle counting architectures when compared to spectroscopy-based counting. This will allow researchers to gain a better understanding of AuNP behavior and interactions with cells, paving the way for advancements in nanomedicine, drug delivery, and biomedical research. The code for this paper is available at the following link: https://github.com/Namerlight/LabelFree_AuNP_Quantification.

Planar grasp detection using generative multi-column convolutional neural networks

This paper presents an accurate, real-time generative multi-column convolutional neural network for two-dimension planner grasp detection. A multi-column structure is used to improve the ability of the network to extract features of different scales. Three parallel channels with three different reception fields can make the network learn different scales of features, which result in the network is more adaptable to a complex environment. This network overcomes some shortcomings like long computing period and by using a generative method rather than sampling grasping candidate. Our network has short computing time because of its light structure, which can be deployed in some close-loop situations. While training the network, we find that some labels in Cornell database is not suitable for the planner grasping detection training, for some specific labels represent different angel of grasping. By comparison with other models, our models accuracy on Cornell grasping dataset is higher, reaching 94%, and our model runs at 13 frames per second.

Machine Learning-Based Crowd behavior Analysis and Forecasting

In many places today, the world's overcrowding causes crowded conditions. Analysis of crowd activity is a developing field of study. It is common knowledge that mob activity can forecast what might happen during an event. Crowd management could be very effective if situations like riots, mass lynchings, traffic jams, accidents, stampedes, etc. could be predicted beforehand. In this paper, we propose a new multicolumn convolutional neural network (MCNN) based technique for predicting mob behavior. The features of the incoming image are first analyzed and extracted. The approximated number of the gathering is then established, and image cropping is completed. For each area of the image, low level characteristics are retrieved. The objects in the picture are then created as density images. Using our method, the gathered characteristics and their object density maps are then linearly mapped. At last, we forecast and quantify the population using the MCNN algorithm. For the ShanghaiTech dataset, we have evaluated our method using actual data.

Three-Dimensional Animation Generation and Image Enhancement Technology Based on Multi-Column Convolutional Neural Network Model

Three-Dimensional Animation Generation and Image Enhancement Technology Based on Multi-Column Convolutional Neural Network Model

Enhancing Solar Energy Forecast Using Multi-Column Convolutional Neural Network and Multipoint Time Series Approach

The rapid expansion of solar industries presents unknown technological challenges. A dedicated and suitable energy forecast is an effective solution for the daily dispatching and production of the electricity grid. The traditional forecast technique uses weather and plant parameters as the model information. Nevertheless, these are insufficient to consider problematic weather variability and the various plant characteristics in the actual field. Considering the above facts and inspired by the excellent implementation of the multi-column convolutional neural network (MCNN) in image processing, we developed a novel approach for forecasting solar energy by transforming multipoint time series (MT) into images for the MCNN to examine. We first processed the data to convert the time series solar energy into image matrices. We observed that the MCNN showed a preeminent response under a ground-based high-resolution spatial–temporal image matrix with a 0.2826% and 0.5826% RMSE for 15 min-ahead forecast under clear (CR) and cloudy (CD) conditions, respectively. Our process was performed on the MATLAB deep learning platform and tested on CR and CD solar energy conditions. The excellent execution of the suggested technique was compared with state-of-the-art deep neural network solar forecasting techniques.

The Application of Pattern Recognition System in Design Field Based on Aesthetic Principles.

The design system based on aesthetic principles is the most representative in the field of design and has a certain significance for the research and construction of design aesthetics and the development of design education. Therefore, this paper studies the application of pattern recognition system in the field of design based on aesthetic principles and designs a new type of aesthetic principle design system based on pattern recognition in computer vision. This paper proposes pattern similarity measurement and image preprocessing technology to improve the traditional aesthetic principle design system through pattern recognition and then further refine the research of the whole system through histogram equalization and gamma correction. Finally, the MNIST dataset experiment is used to verify the effect of multicolumn convolutional neural network pattern recognition on the aesthetic principle design system. The questionnaire survey experiment in this article and the traditional comparative experiment show that 76% of the public are very satisfied with this design system based on the aesthetic principles of pattern recognition in computer vision. Also, the improved aesthetic principle system scores as high as 90–95 points.

Automatic Prediction of Conductive Hearing Loss Using Video Pneumatic Otoscopy and Deep Learning Algorithm.

Diseases of the middle ear can interfere with normal sound transmission, which results in conductive hearing loss. Since video pneumatic otoscopy (VPO) findings reveal not only the presence of middle ear effusions but also dynamic movements of the tympanic membrane and part of the ossicles, analyzing VPO images was expected to be useful in predicting the presence of middle ear transmission problems. Using a convolutional neural network (CNN), a deep neural network implementing computer vision, this preliminary study aimed to create a deep learning model that detects the presence of an air-bone gap, conductive component of hearing loss, by analyzing VPO findings. The medical records of adult patients who underwent VPO tests and pure-tone audiometry (PTA) on the same day were reviewed for enrollment. Conductive hearing loss was defined as an average air-bone gap of more than 10 dB at 0.5, 1, 2, and 4 kHz on PTA. Two significant images from the original VPO videos, at the most medial position on positive pressure and the most laterally displaced position on negative pressure, were used for the analysis. Applying multi-column CNN architectures with individual backbones of pretrained CNN versions, the performance of each model was evaluated and compared for Inception-v3, VGG-16 or ResNet-50. The diagnostic accuracy predicting the presence of conductive component of hearing loss of the selected deep learning algorithm used was compared with experienced otologists. The conductive hearing loss group consisted of 57 cases (mean air-bone gap = 25 ± 8 dB): 21 ears with effusion, 14 ears with malleus-incus fixation, 15 ears with stapes fixation including otosclerosis, one ear with a loose incus-stapes joint, 3 cases with adhesive otitis media, and 3 ears with middle ear masses including congenital cholesteatoma. The control group consisted of 76 cases with normal hearing thresholds without air-bone gaps. A total of 1130 original images including repeated measurements were obtained for the analysis. Of the various network architectures designed, the best was to feed each of the images into the individual backbones of Inception-v3 (three-column architecture) and concatenate the feature maps after the last convolutional layer from each column. In the selected model, the average performance of 10-fold cross-validation in predicting conductive hearing loss was 0.972 mean areas under the curve (mAUC), 91.6% sensitivity, 96.0% specificity, 94.4% positive predictive value, 93.9% negative predictive value, and 94.1% accuracy, which was superior to that of experienced otologists, whose performance had 0.773 mAUC and 79.0% accuracy on average. The algorithm detected over 85% of cases with stapes fixations or ossicular chain problems other than malleus-incus fixations. Visualization of the region of interest in the deep learning model revealed that the algorithm made decisions generally based on findings in the malleus and nearby tympanic membrane. In this preliminary study, the deep learning algorithm created to analyze VPO images successfully detected the presence of conductive hearing losses caused by middle ear effusion, ossicular fixation, otosclerosis, and adhesive otitis media. Interpretation of VPO using the deep learning algorithm showed promise as a diagnostic tool to differentiate conductive hearing loss from sensorineural hearing loss, which would be especially useful for patients with poor cooperation.

Estimating litchi flower number using a multicolumn convolutional neural network based on a density map

Estimating litchi flower number using a multicolumn convolutional neural network based on a density map

Comparison of various deep learning inpainting methods in smart colposcopy images

The specular reflection appears as white area on the cervical images that covers certain region of the images. It affects the quality of the cervical images causing difficulty for the physician to analyse the smart colposcopy images. In this paper, specular reflection is detected and removed from the cervical images, and these removed pixels are refilled using the inpainting methods. To fill the removed pixels, the deep learning inpainting algorithms like partial convolutional neural network, generative multicolumn convolutional neural network, and the dilated convolutional neural network to were used to get the complete and enhanced cervical images. The enhanced images are considered for lesion identification using the Bayes classifier. Based on the analysis, the partial convolution inpainting method gives higher quality with the PSNR value of 48.25 dB and SSIM value of 0.984. The enhanced images using the partial inpainting method identify the neoplasm with an accuracy of 98%.

EEG Emotion Recognition via Graph-based Spatio-Temporal Attention Neural Networks.

Emotion recognition based on electroencephalography (EEG) signals has been receiving significant attention in the domains of affective computing and brain-computer interfaces (BCI). Although several deep learning methods have been proposed dealing with the emotion recognition task, developing methods that effectively extract and use discriminative features is still a challenge. In this work, we propose the novel spatio-temporal attention neural network (STANN) to extract discriminative spatial and temporal features of EEG signals by a parallel structure of multi-column convolutional neural network and attention-based bidirectional long-short term memory. Moreover, we explore the inter-channel relationships of EEG signals via graph signal processing (GSP) tools. Our experimental analysis demonstrates that the proposed network improves the state-of-the-art results in subject-wise, binary classification of valence and arousal levels as well as four-class classification in the valence-arousal emotion space when raw EEG signals or their graph representations, in an architecture coined as GFT-STANN, are used as model inputs.

Research on a Rice Counting Algorithm Based on an Improved MCNN and a Density Map.

The thousand grain weight is an index of size, fullness and quality in crop seed detection and is an important basis for field yield prediction. To detect the thousand grain weight of rice requires the accurate counting of rice. We collected a total of 5670 images of three different types of rice seeds with different qualities to construct a model. Considering the different shapes of different types of rice, this study used an adaptive Gaussian kernel to convolve with the rice coordinate function to obtain a more accurate density map, which was used as an important basis for determining the results of subsequent experiments. A Multi-Column Convolutional Neural Network was used to extract the features of different sizes of rice, and the features were fused by the fusion network to learn the mapping relationship from the original map features to the density map features. An advanced prior step was added to the original algorithm to estimate the density level of the image, which weakened the effect of the rice adhesion condition on the counting results. Extensive comparison experiments show that the proposed method is more accurate than the original MCNN algorithm.

Image-Based Crowd Stability Analysis Using Improved Multi-Column Convolutional Neural Network

Crowd stability analysis is an important yet challenging task, as it is difficult to obtain the quantitative information regarding a crowd in motion, for instance, the crowd count and crowd density in a pedestrian merging area. This paper proposes a novel model that can be used to accurately analyze the crowd stability based on images obtained from a real-time video surveillance system (VSS) in dense crowd scenarios. To enhance the accuracy of the human head recognition for the crowd counting and crowd density estimation, we improve the conventional convolution-neural-network (CNN) model with more columns and merged features, obtaining a four-column convolutional neural network (4C-CNN). Using more columns with receptive fields of deferent sizes, more merged features can be learned, to be adaptive to variations in pedestrian head size due to image resolution. Furthermore, the crowd density of different areas is calculated with image rectification against the perspective distortion. By utilizing the stability criterion based on crowd density, we propose a crowd stability analysis model (CSAM) with the capability of quantitative computation dynamically. The results of extensive experiments performed on public datasets indicate that this improved CNN model exhibits a better performance for crowd counting than the typical multi-column CNN models. In addition, the experiment results pertaining to Shanghai Hongqiao Railway Station demonstrate the effectiveness of the crowd stability analysis model. Thus, this integrated approach can simultaneously conduct the processes of crowd counting, image rectification, density map calculation and crowd stability analysis by using dense crowd images from a VSS.

Crowd density detection method based on crowd gathering mode and multi-column convolutional neural network

Crowds and stampedes often occur in crowd gathering places, resulting in a large number of casualties and causing great negative social impacts. Traditional research on the dynamic assessment of crowd gathering safety mainly relies on real-time video monitoring, but lacks reliable methods for processing a large amount of video data from different sources, different perspectives and different granularities. Based on Edward Hall's personal space theory, this article considers crowd psychology and other factors, and establishes static basic model of crowd gathering patterns. In order to fuse real-time multi-granularity surveillance videos with different perspectives, a multi-column convolutional neural network (M-CNN) was used to extract the local density characteristics of the crowd in a low-altitude perspective, thereby establishing a holographic model of the temporal and spatial evolution of the crowd situation, and a new crowd gathering safety assessment method. This method was actually applied to the safety assessment of crowd gathering in Suzhou landmark-Urban Living Fountain Square, and achieved good results, providing theoretical support for the safety management of crowd gathering places.

Multi-scale and multi-column convolutional neural network for crowd density estimation

In order to accurately identify objects of different sizes, we propose an efficient Multi-Scale and Multi-Column Convolutional Neural Network (MSMC) to estimate the crowd density. On the one hand, the ground truth is generated based on the existed label information. On the other hand, the image is fed into our model to find the relationship between the ground truth and the predicted density map. The network is composed of three components: feature extraction, feature fusion and feature regression. First, VGG16 is utilized for faster feature extraction. Second, different sizes layers from VGG16 are fused, which helps the detection of objects with different sizes. Third, we apply multi-channel convolution to further solve the issue of multi-sizes. After the fusion block, the dilated convolution is employed to strengthen the receptive field without increasing the amount of parameters. In the crowd density estimation, the combination of multiple sizes and multiple channels enhances the ability of receiving information, improves the mapping ability of the original image and the density map, and promotes the accuracy of crowd density estimation. In this paper, the test results of the ShanghaiTech Dataset and UCF_CC_50 Dataset are provided in the Experiment section, which shows that the proposed method makes an excellent performance in both accuracy and robustness.

A novel convolutional neural network method for crowd counting

Crowd density estimation, in general, is a challenging task due to the large variation of head sizes in the crowds. Existing methods always use a multi-column convolutional neural network (MCNN) to adapt to this variation, which results in an average effect in areas with different densities and brings a lot of noise to the density map. To address this problem, we propose a new method called the segmentation-aware prior network (SAPNet), which generates a high-quality density map without noise based on a coarse head-segmentation map. SAPNet is composed of two networks, i.e., a foreground-segmentation convolutional neural network (FS-CNN) as the front end and a crowd-regression convolutional neural network (CR-CNN) as the back end. With only the single dot annotation, we generate the ground truth of segmentation masks in heads. Then, based on the ground truth, FS-CNN outputs a coarse head-segmentation map, which helps eliminate the noise in regions without people in the density map. By inputting the head-segmentation map generated by the front end, CR-CNN performs accurate crowd counting estimation and generates a high-quality density map. We demonstrate SAPNet on four datasets (i.e., ShanghaiTech, UCF-CC-50, WorldExpo’10, and UCSD), and show the state-of-the-art performances on ShanghaiTech part B and UCF-CC-50 datasets.

Crowd Counting via Multi-Column Deconvolution Networks

The difference in distance between the crowd and the camera results in different scales of the population in the image. In the study of crowd counting, this paper first proposed the method of adding deconvolution to multi-column network for the problem that the population size in the image does not affect the accuracy of counting, namely multi-column deconvolution neural network (Mutil-Column Deconvolutinal Neural Network, MCDNN). Compared with the results of the Multi-Column Convolutional Neural Network (MCNN) on the ShanghaiTech dataset, the MSE is reduced by 42.8 and the MAE is reduced by 26.6. On the UCF_CC_50 dataset, the MSE is reduced by 109.4 and the MAE is reduced. 86.3. Compared with other advanced methods, the proposed MCDNN network achieves better performance on the data set, effectively solves the problem of inaccurate population counting accuracy due to scale changes, and can generate a better quality density map.