Deep Learning Research Research Articles

A finite element-inspired hypergraph neural network: Application to fluid dynamics simulations

An emerging trend in deep learning research focuses on the applications of graph neural networks (GNNs) for mesh-based continuum mechanics simulations. Most of these learning frameworks operate on graphs wherein each edge connects two nodes. Inspired by the data connectivity in the finite element method, we present a method to construct a hypergraph by connecting the nodes by elements rather than edges. A hypergraph message-passing network is defined on such a node-element hypergraph that mimics the calculation process of local stiffness matrices. We term this method a finite element-inspired hypergraph neural network, in short FEIH(ϕ)-GNN. We further equip the proposed network with rotation equivariance, and explore its capability for modeling unsteady fluid flow systems. The effectiveness of the network is demonstrated on two common benchmark problems, namely the fluid flow around a circular cylinder and airfoil configurations. Stabilized and accurate temporal roll-out predictions can be obtained using the ϕ-GNN framework within the interpolation Reynolds number range. The network is also able to extrapolate moderately towards higher Reynolds number domain out of the training range.

A Curriculum Study: Accounting Analytics Using Python

The purpose of this study is to highlight the necessity of incorporating AI technology into the accounting sector and to provide a curriculum that allows university students to practice using data science in accounting. As the accounting work environment evolves along with technological advancements, employers, including big accounting firms, are looking for people with technical skills in addition to accounting knowledge. Accounting major students should be prepared for a changing business environment by learning technical skills in combination with accounting knowledge. This necessitates a modification in the accounting curriculum to reflect the dynamic accounting environment driven by technological innovation. However, it doesn't appear that the accounting curriculum has changed much to keep up with these modern changes, and there don't appear to be many case studies that expressly combine accounting and data science. As a part of this endeavor, this paper offers a few concrete examples for the integration of accounting and Python. Python via Anaconda is utilized for the cases in this study, and a creative but beginner-friendly programming is applied to each case. As a result, in this study, a few Python-integrated accounting challenges comprising fundamental financial accounting concepts are addressed. These are only a few examples, but they might aid in introducing data science fundamentals, demonstrating how data science is used in accounting, and encouraging further research and development of deep learning.

Automating hyperparameter optimization in geophysics with Optuna: A comparative study

AbstractDeep learning has gained attraction amongst geophysicists for solving complex longstanding problems. Nevertheless, proper hyperparameter optimization methodologies remain critically underexplored in geophysical deep learning research. This paper attempts to first highlight the importance of hyperparameter optimization and then showcase two geophysics‐related deep learning examples where a grid search and Optuna framework (an automated optimization approach) were used for hyperparameter optimization. We consider two geophysical problems related to denoising seismic traces and the inversion of seismic traces for velocity information. In both cases, models created based on Optuna hyperparameter optimization were able to perform better than those created through grid search. The most significant advantage of Optuna, however, is having quantifiable results to justify the choice of a neural network architecture, depth and other hyperparameters rather than relying on inefficient methods of exploring the hyperparameter space such as a trial‐and‐error or grid search. This study aims to stimulate further exploration and adoption of these frameworks, pushing the boundaries of current deep learning based geophysical problem‐solving methodologies towards full automation.

Translating color fundus photography to indocyanine green angiography using deep-learning for age-related macular degeneration screening

Age-related macular degeneration (AMD) is the leading cause of central vision impairment among the elderly. Effective and accurate AMD screening tools are urgently needed. Indocyanine green angiography (ICGA) is a well-established technique for detecting chorioretinal diseases, but its invasive nature and potential risks impede its routine clinical application. Here, we innovatively developed a deep-learning model capable of generating realistic ICGA images from color fundus photography (CF) using generative adversarial networks (GANs) and evaluated its performance in AMD classification. The model was developed with 99,002 CF-ICGA pairs from a tertiary center. The quality of the generated ICGA images underwent objective evaluation using mean absolute error (MAE), peak signal-to-noise ratio (PSNR), structural similarity measures (SSIM), etc., and subjective evaluation by two experienced ophthalmologists. The model generated realistic early, mid and late-phase ICGA images, with SSIM spanned from 0.57 to 0.65. The subjective quality scores ranged from 1.46 to 2.74 on the five-point scale (1 refers to the real ICGA image quality, Kappa 0.79–0.84). Moreover, we assessed the application of translated ICGA images in AMD screening on an external dataset (n = 13887) by calculating area under the ROC curve (AUC) in classifying AMD. Combining generated ICGA with real CF images improved the accuracy of AMD classification with AUC increased from 0.93 to 0.97 (P < 0.001). These results suggested that CF-to-ICGA translation can serve as a cross-modal data augmentation method to address the data hunger often encountered in deep-learning research, and as a promising add-on for population-based AMD screening. Real-world validation is warranted before clinical usage.

CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation

Multi-sequence magnetic resonance imaging (MRI) has found wide applications in both modern clinical studies and deep learning research. However, in clinical practice, it frequently occurs that one or more of the MRI sequences are missing due to different image acquisition protocols or contrast agent contraindications of patients, limiting the utilization of deep learning models trained on multi-sequence data. One promising approach is to leverage generative models to synthesize the missing sequences, which can serve as a surrogate acquisition. State-of-the-art methods tackling this problem are based on convolutional neural networks (CNN) which usually suffer from spectral biases, resulting in poor reconstruction of high-frequency fine details. In this paper, we propose Conditional Neural fields with Shift modulation (CoNeS), a model that takes voxel coordinates as input and learns a representation of the target images for multi-sequence MRI translation. The proposed model uses a multi-layer perceptron (MLP) instead of a CNN as the decoder for pixel-to-pixel mapping. Hence, each target image is represented as a neural field that is conditioned on the source image via shift modulation with a learned latent code. Experiments on BraTS 2018 and an in-house clinical dataset of vestibular schwannoma patients showed that the proposed method outperformed state-of-the-art methods for multi-sequence MRI translation both visually and quantitatively. Moreover, we conducted spectral analysis, showing that CoNeS was able to overcome the spectral bias issue common in conventional CNN models. To further evaluate the usage of synthesized images in clinical downstream tasks, we tested a segmentation network using the synthesized images at inference. The results showed that CoNeS improved the segmentation performance when some MRI sequences were missing and outperformed other synthesis models. We concluded that neural fields are a promising technique for multi-sequence MRI translation.

A Framework for Reconstructing Super-Resolution Magnetic Resonance Images from Sparse Raw Data Using Multilevel Generative Methods

Super-resolution magnetic resonance (MR) scans give anatomical data for quantitative analysis and treatment. The use of convolutional neural networks (CNNs) in image processing and deep learning research have led to super-resolution reconstruction methods based on deep learning. The study offers a G-guided generative multilevel network for training 3D neural networks with poorly sampled MR input data. The author suggest using super-resolution reconstruction (SRR) and modified sparse sampling to address these issues. Image-based Wasserstein GANs retain k-space data sparsity. Wasserstein Generative Adversarial Networks (WGANs) store and represent picture space knowledge. The method obtains null-valued k-space data and repairs fill gaps in the dataset to preserve data integrity. The proposed reconstruction method processes raw data samples and is able to perform subspace synchronization, deblurring, denoising, motion estimation, and super-resolution image production. The suggested algorithm uses different preprocessing methods to deblur and denoise datasets. Preliminary trials contextualize and speed up assessments. Results indicate that reconstructed pictures have better high-frequency features than sophisticated multi-frame techniques. This is supported by rising PSNR, MAE, and IEM measurements. A k-space correction block improves GAN network refinement learning in the suggested method. This block improves the network’s ability to avoid unnecessary data, speeding reconstruction. A k-space correction module can limit the generator’s output to critical lines, allowing the reconstruction of only missing lines. This improves convergence and speeds rebuilding. This study shows that this strategy reduces aliasing artifacts better than contemporaneous and noniterative methods.

A review of motion generation technology

Nowadays, deep learning and neural network-related research play a very important role in the widely use of artificial intelligence -related technologies, Among them, the hot development in the direction of generative adversarial networks (GAN) has given birth to many generation-related techniques. For example, MoCoGAN is based on the implementation principle of GAN, which enables video generation of different actions of the same character or the same action of different characters, through the method that decompose video into actions and content. This paper introduces the history and principles of MoCoGAN, starting from the prospect of using MoCoGAN in artificial intelligence (AI) industry and the technical challenges that need to be overcome in the future application of action generation. Besides, this paper also discusses the two main issues of how to improve the quality of video generation using MoCoGAN and the input conditions that are the most central problem to GAN networks. By summarizing the optimization solutions of other researchers in these two areas in recent years, this paper searches the core problems need to be solved and propose a broad prospect for future video generation techniques that can be implemented by using MoCoGAN in human-computer interaction (HCI) area.

Guest Editorial: Foreword to the Special Issue on Advanced Research and Applications of Deep Learning and Neural Network in Image Recognition

Over the last two decades, the realm of image recognition has undergone a remarkable transformation, characterized by an astonishing pace of advancement [...]

Protocol for metadata and image collection at diabetic foot ulcer clinics: enabling research in wound analytics and deep learning

BackgroundThe escalating impact of diabetes and its complications, including diabetic foot ulcers (DFUs), presents global challenges in quality of life, economics, and resources, affecting around half a billion people. DFU healing is hindered by hyperglycemia-related issues and diverse diabetes-related physiological changes, necessitating ongoing personalized care. Artificial intelligence and clinical research strive to address these challenges by facilitating early detection and efficient treatments despite resource constraints. This study establishes a standardized framework for DFU data collection, introducing a dedicated case report form, a comprehensive dataset named Zivot with patient population clinical feature breakdowns and a baseline for DFU detection using this dataset and a UNet architecture.ResultsFollowing this protocol, we created the Zivot dataset consisting of 269 patients with active DFUs, and about 3700 RGB images and corresponding thermal and depth maps for the DFUs. The effectiveness of collecting a consistent and clean dataset was demonstrated using a bounding box prediction deep learning network that was constructed with EfficientNet as the feature extractor and UNet architecture. The network was trained on the Zivot dataset, and the evaluation metrics showed promising values of 0.79 and 0.86 for F1-score and mAP segmentation metrics.ConclusionsThis work and the Zivot database offer a foundation for further exploration of holistic and multimodal approaches to DFU research.

MSCAN: multi-scale self- and cross-attention network for RNA methylation site prediction

BackgroundEpi-transcriptome regulation through post-transcriptional RNA modifications is essential for all RNA types. Precise recognition of RNA modifications is critical for understanding their functions and regulatory mechanisms. However, wet experimental methods are often costly and time-consuming, limiting their wide range of applications. Therefore, recent research has focused on developing computational methods, particularly deep learning (DL). Bidirectional long short-term memory (BiLSTM), convolutional neural network (CNN), and the transformer have demonstrated achievements in modification site prediction. However, BiLSTM cannot achieve parallel computation, leading to a long training time, CNN cannot learn the dependencies of the long distance of the sequence, and the Transformer lacks information interaction with sequences at different scales. This insight underscores the necessity for continued research and development in natural language processing (NLP) and DL to devise an enhanced prediction framework that can effectively address the challenges presented.ResultsThis study presents a multi-scale self- and cross-attention network (MSCAN) to identify the RNA methylation site using an NLP and DL way. Experiment results on twelve RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) reveal that the area under the receiver operating characteristic of MSCAN obtains respectively 98.34%, 85.41%, 97.29%, 96.74%, 99.04%, 79.94%, 76.22%, 65.69%, 92.92%, 92.03%, 95.77%, 89.66%, which is better than the state-of-the-art prediction model. This indicates that the model has strong generalization capabilities. Furthermore, MSCAN reveals a strong association among different types of RNA modifications from an experimental perspective. A user-friendly web server for predicting twelve widely occurring human RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) is available at http://47.242.23.141/MSCAN/index.php.ConclusionsA predictor framework has been developed through binary classification to predict RNA methylation sites.

Reducing Q-Value Estimation Bias via Mutual Estimation and Softmax Operation in MADRL

With the development of electronic game technology, the content of electronic games presents a larger number of units, richer unit attributes, more complex game mechanisms, and more diverse team strategies. Multi-agent deep reinforcement learning shines brightly in this type of team electronic game, achieving results that surpass professional human players. Reinforcement learning algorithms based on Q-value estimation often suffer from Q-value overestimation, which may seriously affect the performance of AI in multi-agent scenarios. We propose a multi-agent mutual evaluation method and a multi-agent softmax method to reduce the estimation bias of Q values in multi-agent scenarios, and have tested them in both the particle multi-agent environment and the multi-agent tank environment we constructed. The multi-agent tank environment we have built has achieved a good balance between experimental verification efficiency and multi-agent game task simulation. It can be easily extended for different multi-agent cooperation or competition tasks. We hope that it can be promoted in the research of multi-agent deep reinforcement learning.

Can Neural Networks Do Arithmetic? A Survey on the Elementary Numerical Skills of State-of-the-Art Deep Learning Models

Creating learning models that can exhibit sophisticated reasoning abilities is one of the greatest challenges in deep learning research, and mathematics is rapidly becoming one of the target domains for assessing scientific progress in this direction. In the past few years there has been an explosion of neural network architectures, datasets, and benchmarks specifically designed to tackle mathematical problems, reporting impressive achievements in disparate fields such as automated theorem proving, numerical integration, and the discovery of new conjectures or matrix multiplication algorithms. However, despite this notable success it is still unclear whether deep learning models possess an elementary understanding of quantities and numbers. This survey critically examines the recent literature, concluding that even state-of-the-art architectures and large language models often fall short when probed with relatively simple tasks designed to test basic numerical and arithmetic knowledge.

A Systematic Review on Deep Learning in Education: Concepts, Factors, Models and Measurements

Deep learning, a cognitive process involving goal alignment, advanced problem-solving, and rigorous knowledge application, lacks a unified framework despite its wide application in education. This gap poses challenges for researchers, educators, and policymakers. A meticulous review of ten pertinent articles published between 2019 and 2023, aimed to define the concept, identify influencing factors, examine research models, and scrutinize measurement methods. Results show that while "deep learning" is commonly used, a clear, unified definition is absent. Influencing factors are proposed based on diverse theoretical frameworks, including individual and environmental elements. Commonly referenced models are Biggs' 3P model and Bloom's taxonomy. Data collection predominantly employs surveys, interviews, and experiments, with a bias towards quantitative analysis, supplemented by qualitative methods. It's worth mentioning that there is a growing trend in the use of mixed methods. This study provides valuable insights for the effective implementation of empirical deep learning research. It provides practical references bridging theory and practice for education researchers.

MMV_Im2Im: an open-source microscopy machine vision toolbox for image-to-image transformation.

Over the past decade, deep learning (DL) research in computer vision has been growing rapidly, with many advances in DL-based image analysis methods for biomedical problems. In this work, we introduce MMV_Im2Im, a new open-source Python package for image-to-image transformation in bioimaging applications. MMV_Im2Im is designed with a generic image-to-image transformation framework that can be used for a wide range of tasks, including semantic segmentation, instance segmentation, image restoration, image generation, and so on. Our implementation takes advantage of state-of-the-art machine learning engineering techniques, allowing researchers to focus on their research without worrying about engineering details. We demonstrate the effectiveness of MMV_Im2Im on more than 10 different biomedical problems, showcasing its general potentials and applicabilities. For computational biomedical researchers, MMV_Im2Im provides a starting point for developing new biomedical image analysis or machine learning algorithms, where they can either reuse the code in this package or fork and extend this package to facilitate the development of new methods. Experimental biomedical researchers can benefit from this work by gaining a comprehensive view of the image-to-image transformation concept through diversified examples and use cases. We hope this work can give the community inspirations on how DL-based image-to-image transformation can be integrated into the assay development process, enabling new biomedical studies that cannot be done only with traditional experimental assays. To help researchers get started, we have provided source code, documentation, and tutorials for MMV_Im2Im at [https://github.com/MMV-Lab/mmv_im2im] under MIT license.

Crafting Expressive Faces: A Comparative Exploration of Facial Animation Techniques in VToonify and AnimeGan

Abstract: This review paper presents an overview and comparative analysis of two cutting-edge technologies: VToonify and AnimeGAN, designed to enhance portrait video style transfer and photo animation, respectively. VToonify introduces a novel high-resolution portrait video style transfer approach, offering users enhanced control over the transformation process. On the other hand, AnimenGAN is a lightweight Generative Adversarial Network (GAN) designed specifically for photo animation with a focus on generating anime-style outputs. This review delves into the underlying methodologies and technical principles employed by both VToonify and AnimeGAN. We discuss the crucial features that set these approaches apart from traditional methods and their respective strengths in terms of controllability, high-resolution output, and efficiency. Furthermore, the paper investigates each technology's key challenges and potential areas for future improvements. The review highlights the practical applications of VToonify and AnimeGAN in the realm of creative content generation, multimedia, and visual storytelling. Moreover, we explore real-world use cases and evaluate the impact of these technologies on various industries, including entertainment, advertising, and social media, Through a comprehensive analysis, this paper aims to provide readers with an informed understanding of the state-of-the-art in portrait video style transfer and photo animation. By combining insights from VToonify and AnimeGAN, this review contributes to advancing research in computer vision, deep learning, and artistic content creation.

A survey of multimodal hybrid deep learning for computer vision: Architectures, applications, trends, and challenges

In recent years, deep learning algorithms have rapidly revolutionized artificial intelligence, particularly machine learning, enabling researchers and practitioners to extend previously hand-crafted feature extraction procedures. In particular, deep learning uses adaptive learning processes to learn more complex and informative patterns from datasets of varying sizes. With the increasing availability of multimodal data streams and recent advances in deep learning algorithms, multimodal deep learning is on the rise. This requires the development of complex models that can process and analyze multimodal information in a consistent manner. However, unstructured data can come in many different forms (also known as modalities). Extracting relevant features from this data remains an ambitious goal for deep learning researchers. According to the literature, most deep learning systems consist of a single architecture (i.e., standalone deep learning). When two or more deep learning architectures are combined over multiple sensory modalities, the result is called a multimodal hybrid deep learning model. Since this research direction has received much attention in the field of deep learning, the purpose of this survey is to provide a broader overview of the topic. In this paper, we provide a comprehensive review of recent advances in multimodal hybrid deep learning, including a thorough analysis of the most commonly developed hybrid architectures. In particular, one of the main challenges in multimodal hybrid analysis is the ability of these architectures to systematically integrate cross-modal features in hybrid designs. Therefore, we propose a generic framework for multimodal hybrid learning that focuses mainly on fusion methods. We also identify trends and challenges in multimodal hybrid learning and provide insights and directions for future research. Our findings show that multimodal hybrid learning can perform well in a variety of challenging computer vision applications and tasks.

BRAIN TUMOR DETECTION

Brain tumor detection is a significant problem in medical diagnostics since early and accurate detection improves patient outcomes. Traditional tumor identification techniques often depend on manual interpretation of medical examination, which can be time-consuming and prone to humanerror. Algorithms based on deep learning have emerged in recent years as a viable way to automateand enhance brain tumor identification using medical imaging data. This paper conveys an extensive look on the use of deep learning for brain tumor identification. A Convolutional NeuralNetwork(CNN) architecture is put forward to reach minimum accuracy of 97% and maximum of 100%, using its abilities to automatically learn hierarchical attributes from medical imagery that involve Magnetic Resonance Imaging(MRI) scans. To learn discriminative features suggestive oftumor presence, the suggested CNN framework is trained an extensive collection of labeled brainMRI images. The findings from experiments show that the proposed deep learning approach works. The trained CNN is quite good at differentiating between tumor and non-tumor regions in brain scans. Furthermore, cross-validation and unbiased evaluation are used to assess the model’scapacity to generalize to data that was previously unavailable. Deep learning in brain tumor identification has the potential to greatly enhance diagnostic accuracy, reduce human error, and speed up decision-making. As deep learning research advances, future studies may look at the amalgamation of multi-modal imaging data, transfer learning, and ensemble techniques in order to boost the robustness and generalizability of brain tumor diagnosis. The proposed deep learning-based brain tumor detection system offers the potential for improving medical professionals’ capacity to properly and instantly diagnose brain tumors, ultimately leading to improvements in patient care and outcomes. Keywords: Brain Tumor detection, Diagnosis, Deep Learning, Convolutional NeuralNetworks, Pooling, MRI Dataset

Fairness in Deep Learning: A Survey on Vision and Language Research

Despite being responsible for state-of-the-art results in several computer vision and natural language processing tasks, neural networks have faced harsh criticism due to some of their current shortcomings. One of them is that neural networks are correlation machines prone to model biases within the data instead of focusing on actual useful causal relationships. This problem is particularly serious in application domains affected by aspects such as race, gender, and age. To prevent models from incurring unfair decision-making, the AI community has concentrated efforts on correcting algorithmic biases, giving rise to the research area now widely known as fairness in AI . In this survey paper, we provide an in-depth overview of the main debiasing methods for fairness-aware neural networks in the context of vision and language research. We propose a novel taxonomy that builds upon previous proposals but is tailored for deep learning research to better organize the literature on debiasing methods for fairness. We review all important neural-based methods and evaluation metrics while discussing the current challenges, trends, and important future work directions for the interested researcher and practitioner.

Deep Learning-Inspired IoT-IDS Mechanism for Edge Computing Environments.

The Internet of Things (IoT) technology has seen substantial research in Deep Learning (DL) techniques to detect cyberattacks. Critical Infrastructures (CIs) must be able to quickly detect cyberattacks close to edge devices in order to prevent service interruptions. DL approaches outperform shallow machine learning techniques in attack detection, giving them a viable alternative for use in intrusion detection. However, because of the massive amount of IoT data and the computational requirements for DL models, transmission overheads prevent the successful implementation of DL models closer to the devices. As they were not trained on pertinent IoT, current Intrusion Detection Systems (IDS) either use conventional techniques or are not intended for scattered edge-cloud deployment. A new edge-cloud-based IoT IDS is suggested to address these issues. It uses distributed processing to separate the dataset into subsets appropriate to different attack classes and performs attribute selection on time-series IoT data. Next, DL is used to train an attack detection Recurrent Neural Network, which consists of a Recurrent Neural Network (RNN) and Bidirectional Long Short-Term Memory (LSTM). The high-dimensional BoT-IoT dataset, which replicates massive amounts of genuine IoT attack traffic, is used to test the proposed model. Despite an 85 percent reduction in dataset size made achievable by attribute selection approaches, the attack detection capability was kept intact. The models built utilizing the smaller dataset demonstrated a higher recall rate (98.25%), F1-measure (99.12%), accuracy (99.56%), and precision (99.45%) with no loss in class discrimination performance compared to models trained on the entire attribute set. With the smaller attribute space, neither the RNN nor the Bi-LSTM models experienced underfitting or overfitting. The proposed DL-based IoT intrusion detection solution has the capability to scale efficiently in the face of large volumes of IoT data, thus making it an ideal candidate for edge-cloud deployment.

Federated Learning for Intrusion Detection Systems in Internet of Vehicles: A General Taxonomy, Applications, and Future Directions

In recent years, the Internet of Vehicles (IoV) has garnered significant attention from researchers and automotive industry professionals due to its expanding range of applications and services aimed at enhancing road safety and driver/passenger comfort. However, the massive amount of data spread across this network makes securing it challenging. The IoV network generates, collects, and processes vast amounts of valuable and sensitive data that intruders can manipulate. An intrusion detection system (IDS) is the most typical method to protect such networks. An IDS monitors activity on the road to detect any sign of a security threat and generates an alert if a security anomaly is detected. Applying machine learning methods to large datasets helps detect anomalies, which can be utilized to discover potential intrusions. However, traditional centralized learning algorithms require gathering data from end devices and centralizing it for training on a single device. Vehicle makers and owners may not readily share the sensitive data necessary for training the models. Granting a single device access to enormous volumes of personal information raises significant privacy concerns, as any system-related problems could result in massive data leaks. To alleviate these problems, more secure options, such as Federated Learning (FL), must be explored. A decentralized machine learning technique, FL allows model training on client devices while maintaining user data privacy. Although FL for IDS has made significant progress, to our knowledge, there has been no comprehensive survey specifically dedicated to exploring the applications of FL for IDS in the IoV environment, similar to successful systems research in deep learning. To address this gap, we undertake a well-organized literature review on IDSs based on FL in an IoV environment. We introduce a general taxonomy to describe the FL systems to ensure a coherent structure and guide future research. Additionally, we identify the relevant state of the art in FL-based intrusion detection within the IoV domain, covering the years from FL’s inception in 2016 through 2023. Finally, we identify challenges and future research directions based on the existing literature.