Deep Learning-based Solutions Research Articles

HardGBM: A Framework for Accurate and Hardware-Efficient Gradient Boosting Machines

Gradient boosting machine (GBM) is a powerful and widely used type of ensemble machine learning methods, among which the most famous one is XGBoost. However, the cost of running large GBMs on hardware could become prohibitive given stringent resources. Ensemble reduction on boosting ensemble is intrinsically hard because member models are constructed in a sequential order, where the training targets for latter ones depend on the performance of the former ones. In this work, a GBM reduction framework is proposed for the first time to tackle the problem. For the first time, the framework supports automatic hardware implementation of regression tree ensembles. Experiments on 24 datasets from various applications demonstrate that our method reduces overall area utilization by 81.60% (80.64%) and power consumption by 21.15% (19.06%), while exceeding or successfully maintaining the performance level comparing with the original XGBoost (LightGBM) ensembles. In comparative experiments, to attain approximately the same accuracy level as our framework or XGBoost, deep learning-based solutions require >52.7× footprints, 6.0× power consumption, and 1.4× training time. Equipped with tunable parameters, the framework is expected to seek a Pareto optimal front considering hardware resource limitation, accuracy and stability, and computation (training) efficiency.

Exploiting multi-granularity visual features for retinal layer segmentation in human eyes.

Accurate segmentation of retinal layer boundaries can facilitate the detection of patients with early ophthalmic disease. Typical segmentation algorithms operate at low resolutions without fully exploiting multi-granularity visual features. Moreover, several related studies do not release their datasets that are key for the research on deep learning-based solutions. We propose a novel end-to-end retinal layer segmentation network based on ConvNeXt, which can retain more feature map details by using a new depth-efficient attention module and multi-scale structures. In addition, we provide a semantic segmentation dataset containing 206 retinal images of healthy human eyes (named NR206 dataset), which is easy to use as it does not require any additional transcoding processing. We experimentally show that our segmentation approach outperforms state-of-the-art approaches on this new dataset, achieving, on average, a Dice score of 91.3% and mIoU of 84.4%. Moreover, our approach achieves state-of-the-art performance on a glaucoma dataset and a diabetic macular edema (DME) dataset, showing that our model is also suitable for other applications. We will make our source code and the NR206 dataset publicly available at (https://github.com/Medical-Image-Analysis/Retinal-layer-segmentation).

AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting

Sensors in cyber-physical systems often capture interconnected processes and thus emit correlated time series (CTS), the forecasting of which enables important applications. The key to successful CTS forecasting is to uncover the temporal dynamics of time series and the spatial correlations among time series. Deep learning-based solutions exhibit impressive performance at discerning these aspects. In particular, automated CTS forecasting, where the design of an optimal deep learning architecture is automated, enables forecasting accuracy that surpasses what has been achieved by manual approaches. However, automated CTS solutions remain in their infancy and are only able to find optimal architectures for predefined hyperparameters and scale poorly to large-scale CTS. To overcome these limitations, we propose AutoCTS+, a joint, scalable framework, to automatically devise effective CTS forecasting models. Specifically, we encode each candidate architecture and accompanying hyperparameters into a joint graph representation. We introduce an efficient Architecture-Hyperparameter Comparator (AHC) to rank all architecture-hyperparameter pairs, and we then further evaluate the top-ranked pairs to select an architecture-hyperparameter pair as the final model. Extensive experiments on six benchmark datasets demonstrate that AutoCTS+ not only eliminates manual efforts but also is capable of better performance than manually designed and existing automatically designed CTS models. In addition, it shows excellent scalability to large CTS.

Strength of Deep Learning-based Solutions to Secure Healthcare IoT: A Critical Review

Healthcare applications of IoT systems have gained huge popularity across the globe. From personal monitoring to expert clinical diagnosis, healthcare IoT systems have shown their importance to all possible extents. The ease of use and precise results add to the wide acceptance of such systems. However, this has also led to a magnificent increase in the number of attacks aimed at stealing or manipulating data as well as operations of HIoT-based healthcare assistance. Among the various modes of attacks, network-based attacks are found in the majority. In this work, we perform a critical review of these attacks, the existing countermeasures, and their limitations to understand and proclaim the importance of securing healthcare networks in the best possible manner. We also emphasize the necessity of deep learning-based smart solutions for securing healthcare systems, understanding the potential of deep learning in the security aspects being deployed in other genres of IoT applications. A comparative analysis of deep learning and machine learning-based security solutions is performed to examine their performances.

Performance analysis of AI-based solutions for crop disease identification, detection, and classification

Agriculture is an integral part of human civilization. Apart from providing Food, Agriculture also contributes to the economy. During crop production, crops are susceptible to insects. The identification and treatment of these insects have become a severe issue. The optimal way to reduce this cost is by early detection of the insects and taking appropriate measures to minimize the damage to crop plants. However, the traditional method lacks to examine the diseases and insects' presence in real-time. This study provides deep learning-based solutions for real-time identification and detection of insects in the Soybean crop. Various Transfer-learning models’ performances are examined to explore the proposed solution's feasibility and reliability to find the insect's identification and detection accuracy. The accuracy achieved with the proposed solution is 98.75%, 97%, and 97% using YoloV5, InceptionV3, and CNN, respectively. Among them, the YoloV5 algorithm's performance in the solution is very fast and can run at 53fps, making them fit for real-time detection. Moreover, a dataset of crop insects was collected and labeled by mixing the images collected using different devices. The proposed study helps reduce the producer's workload and is much simpler, and provides better results.

VulANalyzeR: Explainable Binary Vulnerability Detection with Multi-task Learning and Attentional Graph Convolution

Software vulnerabilities have been posing tremendous reliability threats to the general public as well as critical infrastructures, and there have been many studies aiming to detect and mitigate software defects at the binary level. Most of the standard practices leverage both static and dynamic analysis, which have several drawbacks like heavy manual workload and high complexity. Existing deep learning-based solutions not only suffer to capture the complex relationships among different variables from raw binary code but also lack the explainability required for humans to verify, evaluate, and patch the detected bugs.We propose VulANalyzeR, a deep learning-based model, for automated binary vulnerability detection, Common Weakness Enumeration-type classification, and root cause analysis to enhance safety and security. VulANalyzeR features sequential and topological learning through recurrent units and graph convolution to simulate how a program is executed. The attention mechanism is integrated throughout the model, which shows how different instructions and the corresponding states contribute to the final classification. It also classifies the specific vulnerability type through multi-task learning as this not only provides further explanation but also allows faster patching for zero-day vulnerabilities. We show that VulANalyzeR achieves better performance for vulnerability detection over the state-of-the-art baselines. Additionally, a Common Vulnerability Exposure dataset is used to evaluate real complex vulnerabilities. We conduct case studies to show that VulANalyzeR is able to accurately identify the instructions and basic blocks that cause the vulnerability even without given any prior knowledge related to the locations during the training phase.

Improved Detection and Analysis of Wheat Spikes U sing Multi-Stage Convolutional Neural Network

High throughput plant phenotyping is the advanced scientific approach for rapid phenotyping of plant traits, especially high consumable grains or crops, which is designed to process a high volume of data in a short time for plant breeders and cultivars to utilize. Detection and counting of crop traits such as plants, fruits, wheat or rice spikes, sorghum head, and plant diseases is more advanced research in this field, where real-world data are collected using aerial and land-based imaging platforms equipped with a variety of geospatial sensors, and their statistical analysis is conducted using Artificial Intelligence (AI) and Deep Learning-based solutions. In this paper, we contributed to solving such a challenge of phenotyping by detecting and counting wheat spikes from land-based imaging by applying a Region-based Convolutional Neural Network (CNN) model. Our method employs the use of CNN to extract features from the imaging platform and the learning model is trained to detect and count wheat spikes in field images based on these extracted features. Using the publicly available SPIKE dataset to train and test our model, our proposed method achieved 98% average precision and 91% average F1 score on the test set. Our results show a significant improvement of 2.9% and 11.2% in detection accuracy as well as 1% and 3% in average precision metric over state-of-the-art Faster Regionbased Convolutional Neural Network (Faster-RCNN), and RetinaNet, respectively, and have the potential to significantly benefit plant breeders by facilitating the selection of wheat varieties with high yields. DUJASE Vol. 7 (2) 21-30, 2022 (July)

A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part II.

Deep learning has the potential to be one of the most transformative technologies to impact orthopedic surgery. Substantial innovation in this area has occurred over the past 5years, but clinically meaningful advancements remain limited by a disconnect between clinical and technical experts. That is, it is likely that few orthopedic surgeons possess both the clinical knowledge necessary to identify orthopedic problems, and the technical knowledge needed to implement deep learning-based solutions. To maximize the utilization of rapidly advancing technologies derived from deep learning models, orthopedic surgeons should understand the steps needed to design, organize, implement, and evaluate a deep learning project and its workflow. Equipping surgeons with this knowledge is the objective of this three-part editorial review. Part I described the processes involved in defining the problem, team building, data acquisition, curation, labeling, and establishing the ground truth. Building on that, this review (Part II) provides guidance on pre-processing and augmenting the data, making use of open-source libraries/toolkits, and selecting the required hardware to implement the pipeline. Special considerations regarding model training and evaluation unique to deep learning models relative to "shallow" machine learning models are also reviewed. Finally, guidance pertaining to the clinical deployment of deep learning models in the real world is provided. As in Part I, the focus is on applications of deep learning for computer vision and imaging.

Fingerprint Recognition Using Deep Learning - A Review

There have been efforts to address the problems with fingerprint identification systems that require physical contact by creating contactless fingerprint identification systems. Numerous studies on various aspects of contactless fingerprint processing, including the use of deep learning in various algorithmic frameworks, classical image processing, and the machine-learning pipeline, have been published. It was demonstrated that the deep learning-based solutions were more accurate than the alternatives. This effort was driven by a desire to provide a thorough assessment of these successes and their identified limitations. This study examined three approaches to contactless fingerprint recognition: (i) methods for capturing images of the fingerprint, (ii) traditional preprocessing techniques for enhancing fingerprint images for recognition tasks, and (iii) deep learning. (i) taking a picture of your finger, and (ii) using conventional image processing to get the picture ready for recognition. In total, eight research papers were found to meet both the inclusion and exclusion criteria. Based on this review's findings, we discussed the potential benefits of deep learning methods for biometrics and the challenges that still need to be overcome before these methods can be used in practical biometric settings.

Towards asteroid detection in microlensing surveys with deep learning

Asteroids are an indelible part of most astronomical surveys though only a few surveys are dedicated to their detection. Over the years, high cadence microlensing surveys have amassed several terabytes of data while scanning primarily the Galactic Bulge and Magellanic Clouds for microlensing events and thus provide a treasure trove of opportunities for scientific data mining. In particular, numerous asteroids have been observed by visual inspection of selected images. This paper presents novel deep learning-based solutions for the recovery and discovery of asteroids in the microlensing data gathered by the MOA project. Asteroid tracklets can be clearly seen by combining all the observations on a given night and these tracklets inform the structure of the dataset. Known asteroids were identified within these composite images and used for creating the labelled datasets required for supervised learning. Several custom CNN models were developed to identify images with asteroid tracklets. Model ensembling was then employed to reduce the variance in the predictions as well as to improve the generalization error, achieving a recall of 97.67%. Furthermore, the YOLOv4 object detector was trained to localize asteroid tracklets, achieving a mean Average Precision (mAP) of 90.97%. These trained networks will be applied to 16 years of MOA archival data to find both known and unknown asteroids that have been observed by the survey over the years. The methodologies developed can be adapted for use by other surveys for asteroid recovery and discovery.

Generalized Tensor Summation Compressive Sensing Network (GTSNET): An Easy to Learn Compressive Sensing Operation.

The efforts in compressive sensing (CS) literature can be divided into two groups: finding a measurement matrix that preserves the compressed information at its maximum level, and finding a robust reconstruction algorithm. In the traditional CS setup, the measurement matrices are selected as random matrices, and optimization-based iterative solutions are used to recover the signals. Using random matrices when handling large or multi-dimensional signals is cumbersome especially when it comes to iterative optimizations. Recent deep learning-based solutions increase reconstruction accuracy while speeding up recovery, but jointly learning the whole measurement matrix remains challenging. For this reason, state-of-the-art deep learning CS solutions such as convolutional compressive sensing network (CSNET) use block-wise CS schemes to facilitate learning. In this work, we introduce a separable multi-linear learning of the CS matrix by representing the measurement signal as the summation of the arbitrary number of tensors. As compared to block-wise CS, tensorial learning eases blocking artifacts and improves performance, especially at low measurement rates (MRs), such as [Formula: see text]. The software implementation of the proposed network is publicly shared at https://github.com/mehmetyamac/GTSNET.

Multiscale recurrent-guided denoising network for distributed acoustic sensing-vertical seismic profile background noise attenuation

In recent years, distributed optical fiber acoustic sensing (DAS) has emerged as a novel seismic acquisition technique. Compared with conventional hydrophones and geophones of microelectromechanical systems, DAS has an advantage in terms of acquisition geometry, such as low-cost and high-density observations. However, the collected DAS records always suffer from various types of noise, which poses challenges for subsequent processing. Thus, deep learning-based solutions have addressed the attenuation of seismic background noise. Denoising networks can provide excellent denoising results. Nonetheless, the architectures of these networks are relatively simple, which may result in degeneration when confronted with the complex DAS background noise. To effectively attenuate complex noise, a novel multiscale network called recurrent-guided self-enhanced attention network (RGSA-Net) is developed for complex seismic data processing. Specifically, the backbone of RGSA-Net uses a conventional feedforward neural network to preliminarily extract the potential features. Meanwhile, multiscale modules, inspired by the recurrent-guided scheme, are used to enhance the contour information. On this basis, a self-enhanced attention module is applied to fuse the multiscale features and further reinforce the effective information, thereby improving the noise attenuation capability. Synthetic and field experiments demonstrate that RGSA-Net indicates promise in complex noise attenuation and weak upgoing signal recovery.

A strain gauge-based Bridge Weigh-In-Motion system using deep learning

Several breakthroughs have appeared in different applications due to deep learning which can be applied in Bridge Weigh-In-Motion (BWIM) systems as well. Therefore, deep learning applications should be examined in BWIM systems. However, some deep learning-based solutions have already been proposed in the international literature, they cannot be compared to static methods because available datasets are not appropriate to train and test artificial neural networks. In addition, numerous other aspects have been already considered during tests. In the current paper, a numerically simulated and validated comprehensive dataset is provided. It is designed to meet requirements of incremental development. This dataset makes it possible to benchmark static algorithms and deep learning-based methods on the same dataset. In the second part of the paper, a deep learning-based solution is proposed. The developed solution shows promising results on the provided dataset and also demonstrates the applicability of the dataset.

Comparative Transfer Learning Models for End-to-End Self-Driving Car

Self-driving automobiles are prominent in science and technology, which affect social and economic development. Deep learning (DL) is the most common area of study in artificial intelligence (AI). In recent years, deep learning-based solutions have been presented in the field of self-driving cars and have achieved outstanding results. Different studies investigated a variety of significant technologies for autonomous vehicles, including car navigation systems, path planning, environmental perception, as well as car control. End-to-end learning control directly converts sensory data into control commands in autonomous driving. This research aims to identify the most accurate pre-trained Deep Neural Network (DNN) for predicting the steering angle of a self-driving vehicle that is suitable to be applied to embedded automotive technologies with limited performance. Three well-known pre-trained models were compared in this study: AlexNet, ResNet18, and DenseNet121. Transfer learning was utilized by modifying the final layer of pre-trained models in order to predict the steering angle of the vehicle. Experiments were conducted on the dataset collected from two different tracks. According to the study's findings, ResNet18 and DenseNet121 have the lowest error percentage for steering angle values. Furthermore, the performance of the modified models was evaluated on predetermined tracks. ResNet18 outperformed DenseNet121 in terms of accuracy, with less deviation from the center of the track, while DenseNet121 demonstrated greater adaptability across multiple tracks, resulting in better performance consistency.

Low-Light Image and Video Enhancement Using Deep Learning: A Survey.

Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination. Recent advances in this area are dominated by deep learning-based solutions, where many learning strategies, network structures, loss functions, training data, etc. have been employed. In this paper, we provide a comprehensive survey to cover various aspects ranging from algorithm taxonomy to unsolved open issues. To examine the generalization of existing methods, we propose a low-light image and video dataset, in which the images and videos are taken by different mobile phones' cameras under diverse illumination conditions. Besides, for the first time, we provide a unified online platform that covers many popular LLIE methods, of which the results can be produced through a user-friendly web interface. In addition to qualitative and quantitative evaluation of existing methods on publicly available and our proposed datasets, we also validate their performance in face detection in the dark. This survey together with the proposed dataset and online platform could serve as a reference source for future study and promote the development of this research field. The proposed platform and dataset as well as the collected methods, datasets, and evaluation metrics are publicly available and will be regularly updated. Project page: https://www.mmlab-ntu.com/project/lliv_survey/index.html.

Sensor-to-Image Based Neural Networks: A Reliable Reconstruction Method for Diffuse Optical Imaging of High-Scattering Media

Imaging tasks today are being increasingly shifted toward deep learning-based solutions. Biomedical imaging problems are no exception toward this tendency. It is appealing to consider deep learning as an alternative to such a complex imaging task. Although research of deep learning-based solutions continues to thrive, challenges still remain that limits the availability of these solutions in clinical practice. Diffuse optical tomography is a particularly challenging field since the problem is both ill-posed and ill-conditioned. To get a reconstructed image, various regularization-based models and procedures have been developed in the last three decades. In this study, a sensor-to-image based neural network for diffuse optical imaging has been developed as an alternative to the existing Tikhonov regularization (TR) method. It also provides a different structure compared to previous neural network approaches. We focus on realizing a complete image reconstruction function approximation (from sensor to image) by combining multiple deep learning architectures known in imaging fields that gives more capability to learn than the fully connected neural networks (FCNN) and/or convolutional neural networks (CNN) architectures. We use the idea of transformation from sensor- to image-domain similarly with AUTOMAP, and use the concept of an encoder, which is to learn a compressed representation of the inputs. Further, a U-net with skip connections to extract features and obtain the contrast image, is proposed and implemented. We designed a branching-like structure of the network that fully supports the ring-scanning measurement system, which means it can deal with various types of experimental data. The output images are obtained by multiplying the contrast images with the background coefficients. Our network is capable of producing attainable performance in both simulation and experiment cases, and is proven to be reliable to reconstruct non-synthesized data. Its apparent superior performance was compared with the results of the TR method and FCNN models. The proposed and implemented model is feasible to localize the inclusions with various conditions. The strategy created in this paper can be a promising alternative solution for clinical breast tumor imaging applications.

Asking the right questions to solve algebraic word problems

Word algebra problems are among challenging AI tasks as they combine natural language understanding with a formal equation system. Traditional approaches to the problem work with equation templates and frame the task as a template selection and number assignment to the selected template. The recent deep learning-based solutions exploit contextual language models like BERT and encode the natural language text to decode the corresponding equation system. The proposed approach is similar to the template-based methods as it works with a template and fills in the number slots. Nevertheless, it has contextual understanding because it adopts a question generation and answering pipeline to create tuples of numbers, to finally perform the number assignment task by custom sets of rules. The inspiring idea is that by asking the right questions and answering them using a state-of-the-art language model-based system, one can learn the correct values for the number slots in an equation system. The empirical results show that the proposed approach outperforms the other methods significantly on the word algebra benchmark dataset alg514 and performs the second best on the AI2 corpus for arithmetic word problems. It also has superior performance on the challenging SVAMP dataset. Though it is a rule-based system, simple rule sets and relatively slight differences between rules for different templates indicate that it is highly probable to develop a system that can learn the patterns for the collection of all possible templates, and produce the correct equations for an example instance.

GeoDLS: A Deep Learning-Based Corn Disease Tracking and Location System Using RTK Geolocated UAS Imagery

Deep learning-based solutions for precision agriculture have recently achieved promising results. Deep learning has been used to identify crop diseases at the initial stages of disease development in an effort to create effective disease management systems. However, the use of deep learning and unmanned aerial system (UAS) imagery to track the spread of diseases, identify diseased regions within cornfields, and notify users with actionable information remains a research gap. Therefore, in this study, high-resolution, UAS-acquired, real-time kinematic (RTK) geotagged, RGB imagery at an altitude of 12 m above ground level (AGL) was used to develop the Geo Disease Location System (GeoDLS), a deep learning-based system for tracking diseased regions in corn fields. UAS images (resolution 8192 × 5460 pixels) were acquired in cornfields located at Purdue University’s Agronomy Center for Research and Education (ACRE), using a DJI Matrice 300 RTK UAS mounted with a 45-megapixel DJI Zenmuse P1 camera during corn stages V14 to R4. A dataset of 5076 images was created by splitting the UAS-acquired images using tile and simple linear iterative clustering (SLIC) segmentation. For tile segmentation, the images were split into tiles of sizes 250 × 250 pixels, 500 × 500 pixels, and 1000 × 1000 pixels, resulting in 1804, 1112, and 570 image tiles, respectively. For SLIC segmentation, 865 and 725 superpixel images were obtained using compactness (m) values of 5 and 10, respectively. Five deep neural network architectures, VGG16, ResNet50, InceptionV3, DenseNet169, and Xception, were trained to identify diseased, healthy, and background regions in corn fields. DenseNet169 identified diseased, healthy, and background regions with the highest testing accuracy of 100.00% when trained on images of tile size 1000 × 1000 pixels. Using a sliding window approach, the trained DenseNet169 model was then used to calculate the percentage of diseased regions present within each UAS image. Finally, the RTK geolocation information for each image was used to update users with the location of diseased regions with an accuracy of within 2 cm through a web application, a smartphone application, and email notifications. The GeoDLS could be a potential tool for an automated disease management system to track the spread of crop diseases, identify diseased regions, and provide actionable information to the users.

A streamlined platform for analyzing tera-scale DDA and DIA mass spectrometry data enables highly sensitive immunopeptidomics

Integrating data-dependent acquisition (DDA) and data-independent acquisition (DIA) approaches can enable highly sensitive mass spectrometry, especially for imunnopeptidomics applications. Here we report a streamlined platform for both DDA and DIA data analysis. The platform integrates deep learning-based solutions of spectral library search, database search, and de novo sequencing under a unified framework, which not only boosts the sensitivity but also accurately controls the specificity of peptide identification. Our platform identifies 5-30% more peptide precursors than other state-of-the-art systems on multiple benchmark datasets. When evaluated on immunopeptidomics datasets, we identify 1.7-4.1 and 1.4-2.2 times more peptides from DDA and DIA data, respectively, than previously reported results. We also discover six T-cell epitopes from SARS-CoV-2 immunopeptidome that might represent potential targets for COVID-19 vaccine development. The platform supports data formats from all major instruments and is implemented with the distributed high-performance computing technology, allowing analysis of tera-scale datasets of thousands of samples for clinical applications.

Hybrid Spatiotemporal Contrastive Representation Learning for Content-Based Surgical Video Retrieval

In the medical field, due to their economic and clinical benefits, there is a growing interest in minimally invasive surgeries and microscopic surgeries. These types of surgeries are often recorded during operations, and these recordings have become a key resource for education, patient disease analysis, surgical error analysis, and surgical skill assessment. However, manual searching in this collection of long-term surgical videos is an extremely labor-intensive and long-term task, requiring an effective content-based video analysis system. In this regard, previous methods for surgical video retrieval are based on handcrafted features which do not represent the video effectively. On the other hand, deep learning-based solutions were found to be effective in both surgical image and video analysis, where CNN-, LSTM- and CNN-LSTM-based methods were proposed in most surgical video analysis tasks. In this paper, we propose a hybrid spatiotemporal embedding method to enhance spatiotemporal representations using an adaptive fusion layer on top of the LSTM and temporal causal convolutional modules. To learn surgical video representations, we propose exploring the supervised contrastive learning approach to leverage label information in addition to augmented versions. By validating our approach to a video retrieval task on two datasets, Surgical Actions 160 and Cataract-101, we significantly improve on previous results in terms of mean average precision, 30.012 ± 1.778 vs. 22.54 ± 1.557 for Surgical Actions 160 and 81.134 ± 1.28 vs. 33.18 ± 1.311 for Cataract-101. We also validate the proposed method’s suitability for surgical phase recognition task using the benchmark Cholec80 surgical dataset, where our approach outperforms (with 90.2% accuracy) the state of the art.