CNN-based Model Research Articles

A Comparative Study of Swin Transformer and CNN Models for Crowd Counting

Abstract. Crowd counting, a critical component in the management and safety planning of large gatherings and public spaces, is essential for ensuring smooth event operations and preventing potential overcrowding issues. While the standard convolutional neural network (CNN) based model performs well in head counting tasks, it has certain drawbacks when applied to complex scenarios. With the rapid development of artificial intelligence, Transformer models that rely on self-attention mechanisms, as Swin Transformer, have demonstrated exceptional performance in visual tasks, such as image classification, and segmentation in recent times. This study examines the experimental findings of Swin Transformer's head counting tasks and contrasts them with the CNN-based model. Mean Absolute Error (MAE) and Mean Square Error (MSE) evaluation indicators show that the Transformer model outperforms the classic CNN model in terms of generalization abilities when dealing with complicated scenarios. Future research work will increase the diversity of data sets and focus on optimizing model structure and improving training efficiency.

Evaluation of Electrical Properties and Uniformity of Single Wall Carbon Nanotube Dip-Coated Conductive Fabrics Using Convolutional Neural Network-Based Image Analysis

This study proposes a convolutional neural network (CNN)-based image analysis method to evaluate the electrical properties and uniformity of conductive fabrics treated with single-walled carbon nanotube (SWCNT) dip-coating. The conductive fabric was produced by dip-coating cotton-blended spandex with SWCNT, and the surface images were scanned and preprocessed to obtain image data, while resistance measurements were conducted to obtain labels and build the dataset. SEM analysis revealed that as the number of dip-coating cycles increased, particle density and path formation improved. The CNN model learned the relationship between surface images and resistance values, achieving a high predictive performance, with an R-squared (R²) value of 0.9422. The model demonstrated prediction accuracies of 99.1792% for the coefficient of variation (CV) of uniformly coated fabrics and 96.8877% for non-uniformly coated fabrics. Additionally, p-value analysis of all fabric samples yielded a result of 0.96044, indicating no statistically significant difference between the predicted and actual values. The proposed CNN-based model can accurately evaluate the electrical uniformity of conductive fabrics, showing potential for contributing to quality control and process optimization in production.

Bubble defect detection for tire shearography images with transfer learning based deep CNN models

Abstract Safety of drivers and passengers is an important quality assurance aim of the tire manufacturers. Tires that have air bubbles in them are a frequent cause of accidents. Recently, manufacturers have placed a high value on employing laborers with tire diagnostic experience. Along with the expenditure of time and money, this process also necessitates a sizable workforce of experienced laborers. The detection of tire defects can be automated utilizing a number of different ways, greatly reducing the margin for human error. One such method is digital shearography, which can be used to detect air bubbles in images. In this study, transfer learning-based models are proposed to classify air bubble defects in tire shearography images. Proposed method employed in transfer learning with several deep learning CNN-based models, including VGG16, VGG19, ResNet50, Xception, MobileNetV2 and DenseNet201. Owing to the assistance of the experts at the Pirelli Automobile Tires Izmit factory, we were able to collect and label the dataset used in this study. The dataset contains 1392 tire shearography images. 811 of these images belong to the “good” class, while the remaining 581 images belong to the “bad” class. The proposed method in this study achieves remarkable bubble detection results over 95% in terms of accuracy and other metrics with the DenseNet201 model. The results indicate that the proposed model has comparable accuracy and recall with the existing studies. This study is valuable for the tire industry, as the existing literature offers limited research on the classification of tire bubble defects, the differentiation between defective and defect-free tires, and the categorization of various defect types. In addition, the proposed model offers several advantages over existing studies, including higher robustness against overfitting, suitability for real-time industrial applications, and improved efficiency in processing, making a practical solution for defect detection in industrial settings.

Exploring protein natural diversity in environmental microbiomes with DeepMetagenome.

Protein natural diversity offers a vast sequence space for protein engineering, and deep learning enables its detection from metagenomes/proteomes without prior assumptions. DeepMetagenome, a Python-based method, explores protein diversity through modules for training and analyzing sequence datasets. The deep learning model includes Embedding, Conv1D, LSTM, and Dense layers, with sequence feature analysis for data cleaning. Applied to metallothioneins from a database of over 146 million coding features, DeepMetagenome identified over 500 high-confidence metallothionein sequences, outperforming DIAMOND and CNN-based models. It showed stable performance compared to a Transformer-based model over 25 epochs. Among 23 synthesized sequences, 20 exhibited metal resistance. The tool also successfully explored the diversity of three additional protein families and is freely available on GitHub with detailed instructions.

Detection of tuberculosis using cough audio analysis: a deep learning approach with capsule networks

PurposeTuberculosis (TB) is a widespread infectious disease that requires early detection for effective treatment and control. This study aims to improve TB detection using cough audio analysis, comparing the performance of capsule networks to other deep learning models.MethodsWe used cough audio recordings from 1105 individuals with a new or worsening cough for at least two weeks, totaling 9772 recordings. These recordings were processed into spectral images, and HOG features were extracted. Various models, including Capsule Networks + FCNN, CNN, VGG16, and ResNet50 were trained and evaluated.ResultsCapsule Networks + FCNN achieved the best performance with an accuracy of 0.97, sensitivity of 0.98, specificity of 0.96, F1 score of 0.97, and precision of 0.97, outperforming other models. This attribute is due to the model’s ability to learn complex features from spectral images.ConclusionsThis study concludes that Capsule Networks are more effective than typical CNN-based models in diagnosing TB from cough audio. This suggests that advanced deep learning frameworks could significantly enhance TB screening accuracy, especially in resource-limited areas.

Convolutional neural networks-based surrogate model for fast computational fluid dynamics simulations of indoor airflow distribution

Computational fluid dynamics (CFD) is a powerful but time-consuming simulation tool for building indoor environment analysis. Artificial intelligence (AI)-based surrogate models, especially artificial neural networks (ANN)-based models which are the dominated ones, have demonstrated a great potential in accelerating CFD simulations. However, the published AI-based models are not good at capturing local spatial features in indoor airflow distribution, leading to poor local simulation accuracy. To overcome this challenge, this study proposes a convolutional neural networks (CNN)-based surrogate model for fast CFD simulations of indoor airflow distribution. Compared with other published AI-based models, this model can capture local spatial features in indoor airflow distribution datasets simulated by CFD, leading to higher accuracy. To enable this model to process room geometry information, a geometry representation strategy is proposed to convert room geometry information into inputs suitable for CNN models. Simulation data of 2000 indoor airflow velocity fields with various boundary conditions are generated using COMSOL Multiphysics. Five cases with various model training strategies are designed based on these simulation data to verify the performance of the CNN-based model by comparing this model with ANN-based and GNN-based models. The results show that the CNN-based model outperforms other models in all cases. The CNN-based and GNN-based models have significantly smaller local simulation errors than the ANN-based model. The simulation accuracy of the CNN-based model is improved by an average of 45.55 % and 32.90 % compared with the ANN-based and GNN-based models, respectively. Moreover, the computational time of the CNN-based model is reduced to about 0.05 % of the computational time of CFD simulations.

Advancements and Applications of Convolutional Neural Network Models in Cardiovascular Disease: A Comprehensive Review

In recent years, artificial intelligence (AI) has demonstrated remarkable advancements in the field of cardiovascular disease (CVD), particularly in the analysis of electrocardiograms (ECGs). Due to its widespread use, low cost, and high efficiency, the ECG has long been regarded as a cornerstone of cardiological examinations and remains the most widely utilized diagnostic tool in cardiology. The integration of AI, especially deep learning (DL) technologies based on convolutional neural networks (CNNs), into ECG analysis, has shown immense potential across several cardiological subfields. Deep learning methods have provided robust support for the rapid interpretation of ECGs, enabling the fine-grained analysis of ECG waveform changes with diagnostic accuracy comparable to that of expert cardiologists. Additionally, CNN-based models have proven capable of capturing subtle ECG changes that are often undetectable by traditional methods, accurately predicting complex conditions such as atrial fibrillation, left and right ventricular dysfunction, hypertrophic cardiomyopathy, acute coronary syndrome, and aortic stenosis. This highlights the broad application potential of AI in the diagnosis of cardiovascular diseases. However, despite their extensive applications, CNN models also face significant limitations, primarily related to the reliability of the acquired data, the opacity of the “black box” processes, and the associated medical, legal, and ethical challenges. Addressing these limitations and seeking viable solutions remain critical challenges in modern medicine.

Integrating genomic and molecular data to predict antimicrobial minimum inhibitory concentration in Klebsiella pneumoniae

Minimum inhibitory concentration (MIC) denotes the in vitro benchmark indicating the quantity of antibiotic required to inhibit proliferation of specific bacterial strains. Determining MIC values corresponding to the infecting bacterial strain is paramount for tailoring appropriate antibiotic therapy. In the interim between specimen collection and laboratory-derived MIC outcomes, clinicians frequently resort to empirical therapy informed by retrospective analyses. Here introduces two deep learning approaches, a Convolutional Neural Network (CNN)-based model and an Enformer-based model, integrating genomic data of Klebsiella Pneumoniae and molecular structural data of 20 antibiotics to anticipate the MIC value of the bacterium for each antibiotic under consideration. These models demonstrate enhanced raw accuracy over the existing state-of-the-art model, which rely exclusively on genomic data. The CNN-based model achieves a notable 20% increase in raw accuracy while further mirroring the 1-tier accuracy of the state-of-the-art model. Although the Enformer-based model does not quite reach the performance levels of the CNN-based model, it offers an advantage by eliminating the need for arbitrary data processing steps. This streamlining of the data processing pipeline facilitates fast updates and improves the model interpretability. It is expected that these deep learning paradigms can significantly inform and bolster clinician decision-making during the empirical treatment phase.

Artificial intelligence derived electrocardiograms: a comparative study of network architectures predicting sex and left ventricular dysfunction

Abstract Background Within cardiovascular medicine, AI-driven electrocardiogram (ECG) analysis has emerged as a powerful tool, capable of predicting clinically significant abnormalities. The increasing growth of wearable technologies for ECG monitoring underscores the importance of advancing AI methodologies. While Convolutional Neural Networks (CNNs) have traditionally been the go-to choice for ECG analysis, the rise of Vision Transformers (ViTs), a novel computer vision paradigm, raises the question of their potential superiority. Purpose To compare the performance of CNNs and ViTs in ECG analysis, and to assess the efficiency of lean models using truncated ECG recordings. Methods We trained a model using 12-lead ECGs as input for the binary classification of two outputs: Sex (Male or Female) and Left ventricular dysfunction (LVD, defined as below 35%), determined by echocardiography within 2 weeks of the ECG. Models were developed using full standard ECGs, single lead and truncated recordings (1,2 and 4 10 seconds). We compared CNNs (PyTorch) and ViTs (HuggingFace) performance. We analyzed the explainability of our findings, representing the AI model's focus of interest on ECGs from normal cases versus those with reduced ejection fraction, thereby validating its diagnostic discernment (Fig 1) Results We identified 150,691 and 29,422 patients with valid ECG tests for the Sex and LVD prediction models, respectively. For the Sex outcome, the AUROCs were 0.911 (95% CI: 0.908-0.914) and 0.898 (95% CI: 0.894-0.901) for the CNN-based model and the ViT-based models, respectively, with the difference statistically significant (P-Value < 0.001). For the LVD outcome, the AUROCs were comparable at 0.878 (95% CI: 0.864-0.892) and 0.866 (95% CI: 0.853-0.879) for the CNN-based model and the ViT-based models, respectively, (P-Value = 0.056). Furthermore, we found that 98.8% for the maximal predictive power of the models was achieved within a 2 second time frame of a 12 lead ECG. Similarly, a single lead based model achieved a relatively high AUC for the prediction of LVD. Conclusions AI-ECG models developed with ViT architecture did not surpass CNN-based models in sex classification and LVD identification. Our study highlights the significance of lean models allowing for rapid prediction within 2-second tracings or using a single lead, demonstrates the potential for streamlined AI-ECG applications in clinical practice.

Transfer Learning for Turkish Cuisine Classification

Thanks to developments in data-oriented domains like deep learning and big data, the integration of artificial intelligence with food category recognition has been a topic of interest for decades. The capacity of image classification to produce more precise outcomes in less time has made it a popular topic in computer vision. For the purpose of food categorization, three well-known CNN-based models—EfficientNetV2M, ResNet101, and VGG16—were fine-tuned in this research. Moreover, the pre-trained Vision Transformer (ViT) was used for feature extraction, followed by classification using a Random Forest (RF) algorithm. All the models were assessed on the TurkishFoods-15 dataset. It was found that the ViT and RF models were most effective in accurately capturing food images, with precision, recall, and F1-score values of 0.91, 0.86, and 0.88 respectively.

Evaluation of root canal filling length on periapical radiograph using artificial intelligence.

This work proposes a novel method to evaluate root canal filling (RCF) success using artificial intelligence (AI) and image analysis techniques. 1121 teeth with root canal treatment in 597 periapical radiographs (PARs) were anonymized and manually labeled. First, RCFs were segmented using 5 different state-of-the-art deep learning models based on convolutional neural networks. Their performances were compared based on the intersection over union (IoU), dice score and accuracy. Additionally, fivefold cross validation was applied for the best-performing model and their outputs were later used for further analysis. Secondly, images were processed via a graphical user interface (GUI) that allows dental clinicians to mark the apex of the tooth, which was used to find the distance between the apex of the tooth and the nearest RCF prediction of the deep learning model towards it. The distance can show whether the RCF is normal, short or long. Model performances were evaluated by well-known evaluation metrics for segmentation such as IoU, Dice score and accuracy. CNN-based models can achieve an accuracy of 88%, an IoU of 79% and Dice score of 88% in segmenting root canal fillings. Our study demonstrates that AI-based solutions present accurate and reliable performance for root canal filling evaluation.

Breast Lesion Detection for Ultrasound Images Using MaskFormer.

This study evaluates the performance of the MaskFormer model for segmenting and classifying breast lesions using ultrasound images, addressing ultrasound's limitations. Ultrasound used for breast cancer detection faces challenges like low image contrast and difficulty in the detection of small or multiple lesions, further complicated by variability based on operator skill. Initial experiments with U-Net and other CNN-based models revealed constraints, such as early plateauing in model loss, which indicated suboptimal learning and performance. In contrast, MaskFormer demonstrated continuous improvement, achieving higher precision in breast lesion segmentation and significantly reducing both false positives and false negatives. Comparative analysis showed MaskFormer's superior performance, with the highest precision and recall rates for malignant lesions and an overall mean average precision (mAP) of 0.943. The model's ability to detect a diverse range of breast lesions, including those potentially missed by the human eye, especially by less experienced practitioners, underscores its potential. These findings suggest that integrating AI models like MaskFormer could greatly enhance ultrasound performance for breast cancer detection, providing reliable, operator-independent image analysis and potentially improving patient outcomes on a global scale.

Auto encoder-based defense mechanism against popular adversarial attacks in deep learning.

Convolutional Neural Network (CNN)-based models are prone to adversarial attacks, which present a significant hurdle to their reliability and robustness. The vulnerability of CNN-based models may be exploited by attackers to launch cyber-attacks. An attacker typically adds small, carefully crafted perturbations to original medical images. When a CNN-based model receives the perturbed medical image as input, it misclassifies the image, even though the added perturbation is often imperceptible to the human eye. The emergence of such attacks has raised security concerns regarding the implementation of deep learning-based medical image classification systems within clinical environments. To address this issue, a reliable defense mechanism is required to detect adversarial attacks on medical images. This study will focus on the robust detection of pneumonia in chest X-ray images through CNN-based models. Various adversarial attacks and defense strategies will be evaluated and analyzed in the context of CNN-based pneumonia detection. From earlier studies, it has been observed that a single defense mechanism is usually not effective against more than one type of adversarial attack. Therefore, this study will propose a defense mechanism that is effective against multiple attack types. A reliable defense framework for pneumonia detection models will ensure secure clinical deployment, facilitating radiologists and doctors in their diagnosis and treatment planning. It can also save time and money by automating routine tasks. The proposed defense mechanism includes a convolutional autoencoder to denoise perturbed Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) adversarial images, two state- of-the-art attacks carried out at five magnitudes, i.e., ε (epsilon) values. Two pre-trained models, VGG19 and VGG16, and our hybrid model of MobileNetV2 and DenseNet169, called Stack Model, have been used to compare their results. This study shows that the proposed defense mechanism outperforms state-of-the-art studies. The PGD attack using the VGG16 model shows a better attack success rate by reducing overall accuracy by up to 67%. The autoencoder improves accuracy by up to 16% against PGD attacks in both the VGG16 and VGG19 models.

Predicting physicochemical properties of papayas (Carica papaya L.) using a convolutional neural networks model approach.

The current state of quality assessment methods for agricultural produce, particularly fruits, heavily relies on manual inspection techniques, which could be subjective, time-consuming, and prone to human errors. Consequently, there have been emerging trends and needs for non-destructive methods to evaluate fruit quality accurately and practically. This research aimed to develop a novel approach for predicting the physicochemical properties of papayas using a convolutional neural network (CNN) model that combines image analysis and weight assessment. This study involved capturing images of papayas at different ripening stages, measuring papaya weights, and determining various physicochemical properties such as texture, pH, total soluble solids, and seed weight. A total of 532 images were obtained from 132 papayas, and an additional 1064 images were generated through image augmentation. The dataset was divided into three sets with an 8:1:1 ratio for training, validation, and testing. The CNN model was trained using papaya images and weights as input values to predict and estimate the physicochemical property values. Model performance was evaluated using mean squared error (MSE) and the coefficient of determination (R2) as metrics. The CNN model, integrated with image processing, could predict the diverse physicochemical properties of papayas with high accuracy. The MSE values estimated for the training and validation sets were 0.0284 and 0.1729, respectively. The R2 values for the test dataset ranged from 0.71 to 0.94. These findings demonstrate that CNN-based models could provide detailed and quantitative insights, facilitating improved understanding and management of papaya quality while enhancing predictive modeling accuracy in agriculture. PRACTICAL APPLICATION: This research introduces a new method for accurately predicting the quality of papayas using a computer model. Instead of relying on manual inspection, which can be slow and prone to errors, this model uses images of papayas and their weights to predict properties, including texture, pH, total soluble solids, and seed weight. This can help manage papaya quality better while also improving agricultural production and transportation processes.

Comparative Study of GPT-4Vision and Convolutional Neural Networks in Histopathological Image Analysis

Abstract Introduction/Objective Recent advancements in artificial intelligence (AI), particularly with Large Language Models like ChatGPT-4Vision (GPT-4V), have expanded the potential for automated interpretation of medical images. This study evaluates the diagnostic accuracy of GPT-4V in histopathological analysis of neurodegenerative diseases and compares its performance with traditional Convolutional Neural Networks (CNNs). Methods/Case Report We utilized 1515 histopathological images, including hematoxylin and eosin (H&E) staining and tau immunohistochemistry, from patients with various neurodegenerative diseases, such as Alzheimer’s disease, progressive supranuclear palsy, and corticobasal degeneration. GPT-4V’s performance was assessed through multi- step prompts to assess how textual context influences the image interpretation. In the quantitative analysis, the diagnostic accuracy of GPT-4V and the CNN-based model YOLOv8 was assessed for classifying three tau lesions—astrocytic plaques, neuritic plaques, and tufted astrocytes. The analysis used both zero-shot and few-shot learning methods, where GPT-4V and YOLOv8 were evaluated on their ability to distinguish these lesions using ten hold-out test images and 490 training images per lesion. Results (if a Case Study enter NA) GPT-4V accurately recognized Lewy bodies in all cases, but it showed limited accuracy in identifying neurofibrillary tangles (20%) and Pick bodies (0%) on H&E slides. Notably, GPT-4V often suggested Alzheimer’s disease as a potential diagnosis, relying more on contextual clues, such as the prevalence of certain diseases, rather than direct image analysis. In the tau immunohistochemical analysis, GPT-4V’s diagnoses were strongly influenced by additional information about staining and brain region. However, few-shot learning markedly improved GPT-4V’s diagnostic capabilities from 40% accuracy with zero-shot learning to 90% with few-shot learning, reaching best accuracy with 20-shot learning. This performance compared favorably against YOLOv8, which also achieved 90% accuracy but required 100-shot learning. Conclusion While GPT-4V faces challenges in independently interpreting histopathological images, few-shot learning significantly improves its accuracy. This approach is particularly promising for neuropathology, where acquiring extensive labeled datasets is challenging. The findings highlight the need for ongoing refinement of AI applications in pathology, emphasizing balanced integration of textual and visual data to optimize diagnostic efficacy and reliability.

Enhanced COVID-19 Detection from X-ray Images with Convolutional Neural Network and Transfer Learning.

The global spread of Coronavirus (COVID-19) has prompted imperative research into scalable and effective detection methods to curb its outbreak. The early diagnosis of COVID-19 patients has emerged as a pivotal strategy in mitigating the spread of the disease. Automated COVID-19 detection using Chest X-ray (CXR) imaging has significant potential for facilitating large-scale screening and epidemic control efforts. This paper introduces a novel approach that employs state-of-the-art Convolutional Neural Network models (CNNs) for accurate COVID-19 detection. The employed datasets each comprised 15,000 X-ray images. We addressed both binary (Normal vs. Abnormal) and multi-class (Normal, COVID-19, Pneumonia) classification tasks. Comprehensive evaluations were performed by utilizing six distinct CNN-based models (Xception, Inception-V3, ResNet50, VGG19, DenseNet201, and InceptionResNet-V2) for both tasks. As a result, the Xception model demonstrated exceptional performance, achieving 98.13% accuracy, 98.14% precision, 97.65% recall, and a 97.89% F1-score in binary classification, while in multi-classification it yielded 87.73% accuracy, 90.20% precision, 87.73% recall, and an 87.49% F1-score. Moreover, the other utilized models, such as ResNet50, demonstrated competitive performance compared with many recent works.

Reduction of Misclassifications in Wildfire Detection: A Weighted Ensemble Deep Learning Approach

Governments worldwide are increasingly prioritizing early wildfire detection to safeguard lives, property, and the environment. Although CNN-based models have demonstrated exceptional performance in various computer vision applications, the evolving nature of wildfire images poses significant challenges for a single CNN-based model in wildfire detection. In this study, we addressed this issue by integrating and weighting the differential learning capabilities of three individual transfer learning models: InceptionV3, ResNet50, and VGG16. Experimental results show that the ensemble deep learning models significantly outperformed all single classifiers across all performance metrics. Both the ensemble and weighted ensemble deep learning models achieved 99.7% accuracy, 99.5% precision, 100% recall, 99.8% F1-score, 0.5%false positive rate, 0.0% false negative rate and 0.3% error rate. Additionally, these models reduced the error rate by 98%, 91%, and 40% compared to the error rates of ResNet50, InceptionV3, and VGG16 respectively. A false negative rate of 0% indicates that our proposed ensemble deep learning models identified and predicted all the wildfire instances present in the test set correctly without a single misclassification. This positions our proposed ensemble deep learning models as superior choices for reducing misclassifications in wildfire detection.

Machine/deep learning-assisted hemoglobin level prediction using palpebral conjunctival images.

Palpebral conjunctival hue alteration is used in non-invasive screening for anaemia, whereas it is a qualitative measure. This study constructed machine/deep learning models for predicting haemoglobin values using 150 palpebral conjunctival images taken by a smartphone. The median haemoglobin value was 13.1 g/dL, including 10 patients with <11 g/dL. A segmentation model using U-net was successfully constructed. The segmented images were subjected to non-convolutional neural network (CNN)-based and CNN-based regression models for predicting haemoglobin values. The correlation coefficients between the actual and predicted haemoglobin values were 0.38 and 0.44 in the non-CNN-based and CNN-based models, respectively. The sensitivity and specificity for anaemia detection were 13% and 98% for the non-CNN-based model and 20% and 99% for the CNN-based model. The performance of the CNN-based model did not improve with a mask layer guiding the model's attention towards the conjunctival regions, however, slightly improved with correction by the aspect ratio and exposure time of input images. The gradient-weighted class activation mapping heatmap indicated that the lower half area of the conjunctiva was crucial for haemoglobin value prediction. In conclusion, the CNN-based model had better results than the non-CNN-based model. The prediction accuracy would improve by using more input data with anaemia.

Improved Urdu-English Neural Machine Translation with a fully Convolutional Neural Network Encoder

Neural machine translation (NMT) approaches driven by artificial intelligence (AI) has gained more and more attention in recent years, mainly due to their simplicity yet state-of-the-art performance. Despite NMT models with attention mechanism relying heavily on the accessibility of substantial parallel corpora, they have demonstrated efficacy even for languages with limited linguistic resources. The convolutional neural network (CNN) is frequently employed in tasks involving visual and speech recognition. Implementing CNN for MT is still challenging compared to the predominant approaches. Recent research has shown that the CNN-based NMT model cannot capture long-term dependencies present in the source sentence. The CNN-based model can only capture the word dependencies within the width of its filters. This unnatural character often causes a worse performance for CNN-based NMT than the RNN-based NMT models. This study introduces a simple method to improve neural translation of a low-resource language, specifically Urdu-English (UR-EN). In this paper, we use a Fully Convolutional Neural Network (FConv-NN) based NMT architecture to create a powerful MT encoder for UR-EN translation that can capture the long dependency of words in a sentence. Although the model is quite simple, it yields strong empirical results. Experimental results show that the FConv-NN model consistently outperforms the traditional CNN-based model with filters. On the Urdu-English Dataset, the FConv-NN model produces translation with a gain of 18.42 BLEU points. Moreover, the quantitative and comparative analysis shows that in a low-resource setting, FConv-NN-based NMT outperforms conventional CNN-based NMT models.

Siamese InternImage for Change Detection

For some time, CNN was the de facto state-of-the-art method in remote sensing image change detection. Although transformer-based models have surpassed CNN-based models due to their larger receptive fields, CNNs still retain their value for their efficiency and ability to extract precise local features. To overcome the limitations of the restricted receptive fields in standard CNNs, deformable convolution allows for dynamic adjustment of sampling locations in convolutional kernels, improving the network’s ability to model global contexts. InternImage is an architecture built upon deformable convolution as its foundational operation. Motivated by InternImage, in this paper, a CNN-based change detection vision foundation model is proposed. By introducing deformable convolution into Siamese InternImage architecture, the proposed CNN-based change detection vision foundation model is capable of capturing long-range dependencies and global information. A refinement block is utilized to merge local detail, where channel attention is incorporated. The proposed approach achieved excellent performance on the LEVIR-CD and WHU-CD datasets.