Innovative Network Architecture Research Articles

Water Demand Forecasting in Multiple District Metered Areas Based on a Multi-Scale Correction Module Neural Network Architecture

Short-term water demand forecasting (STWDF) for multiple spatially and temporally correlated District Metering Areas (DMAs) is an essential foundation for achieving more refined management of urban water supply networks. However, due to the greater uncertainty associated with specific DMA demand compared to overall water usage, accurately predicting STWDF poses significant challenges. This study introduces an innovative network architecture—the multi-scale correction module neural network, built upon Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNN) enhanced with Attention mechanisms—for simultaneous STWDF with a temporal resolution of one hour over a week for 10 DMAs located in a single city in northern Italy. This framework utilizes multivariate corrections to refine and enhance the output accuracy. The results reveal that, in comparison to traditional Gated Recurrent Unit or LSTM models, the proposed model with integrated correction modules, particularly those that leverage inter-DMA correlations, improves performance across all evaluation metrics by an average of 5%-20% per DMA. Additionally, it consistently delivers superior accuracy across three scenarios: single DMA forecasting, total water demand, and extreme conditions, while maintaining stable performance throughout. Furthermore, the interpretability analysis underscores the feasibility of this innovative structure and highlights the contribution of meteorological features to the predictive model in some DMA-level STWDF. The unified input-output framework elegantly simplifies the STWDF process across multiple DMAs, providing new insights and methodologies for future research in this domain.

MARes-Net: multi-scale attention residual network for jaw cyst image segmentation.

Jaw cyst is a fluid-containing cystic lesion that can occur in any part of the jaw and cause facial swelling, dental lesions, jaw fractures, and other associated issues. Due to the diversity and complexity of jaw images, existing deep-learning methods still have challenges in segmentation. To this end, we propose MARes-Net, an innovative multi-scale attentional residual network architecture. Firstly, the residual connection is used to optimize the encoder-decoder process, which effectively solves the gradient disappearance problem and improves the training efficiency and optimization ability. Secondly, the scale-aware feature extraction module (SFEM) significantly enhances the network's perceptual abilities by extending its receptive field across various scales, spaces, and channel dimensions. Thirdly, the multi-scale compression excitation module (MCEM) compresses and excites the feature map, and combines it with contextual information to obtain better model performance capabilities. Furthermore, the introduction of the attention gate module marks a significant advancement in refining the feature map output. Finally, rigorous experimentation conducted on the original jaw cyst dataset provided by Quzhou People's Hospital to verify the validity of MARes-Net architecture. The experimental data showed that precision, recall, IoU and F1-score of MARes-Net reached 93.84%, 93.70%, 86.17%, and 93.21%, respectively. Compared with existing models, our MARes-Net shows its unparalleled capabilities in accurately delineating and localizing anatomical structures in the jaw cyst image segmentation.

Using Machine Learning to Identify Diseases and Perform Sorting in Apple Fruit

Fruit diseases play a major role in global agriculture, leading to substantial crop losses and influencing food production and economic stability. In this age of Industry 4.0 the fruit sorting is an important part in the food processing wherein this work plays a vital role. In this study, a solution for the detection and classification of apple fruit diseases is proposed and experimentally validated. Deep learning models offer promise for automating disease identification using fruit images, but encounter obstacles such as therequirement for extensive training data, computational complexity, and the risk of overfitting. This study introduces an innovative convolutional neural network (CNN) architecture aimed at addressing these challenges by incorporating a reduced number of layers, thus alleviating computational burdens while maintaining performance. Additionally, augmentation techniques such as shift, shear, scaling, zoom, and flipping are employed to diversify the training set without additional image acquisition. Our CNN model is specifically trained to identify common apple crop diseases like Scab, Rot, and Blotch. Rigorous experimental evaluation demonstrates the effectiveness ofour model, achieving a remarkable classification accuracy of 95.37%. Significantly, our model demonstrates reduced storage requirements and faster execution times compared to existing deep CNN architectures, enabling deployment on handheld devices and resource-limited environments. While other CNN models may offer similar accuracy levels, our approach emphasizes efficiency and resource optimization, rendering it practical for real-world applications in agriculture. Furthermore, our CNN model exhibits resilience to environmental variations and imaging parameters, enhancing its applicability across diverse agricultural settings. By leveraging advanced machine learning techniques, the approach developed in this experimental work contributes to modernizing fruits and vegetables sorting operations in food processing, crop management practices thus promoting agricultural sustainability. The scalability and portability of our model make it suitable for deployment in both small-scale farms and large-scale agricultural operations.

A novel CNN architecture for robust structural damage identification via strain measurements and its validation via full-scale experiments

In this study, an innovative two-dimensional Convolutional Neural Network (2D CNN) architecture is proposed and investigated for the classification of bridge damage. Employing unique strain time-history data transformed into grayscale images, the approach seamlessly combines feature extraction and classification, allowing for the precise identification and categorization of structural damage. The method’s effectiveness was validated through field experiments on a full-scale bridge mock-up sample subjected to several controlled damage states under nonstationary, commercial vehicle loads. A wide range of realistic damage conditions, from minor to severe structural damage states, was included in the experimental scenarios together with inherent operational uncertainties. The robustness of the 2D CNN model was rigorously tested against fluctuating loads and introduced noise. Demonstrating remarkable accuracy, the 2D CNN successfully classified different damage states with over 95% accuracy, effectively identifying a damage state that was visually undetectable. Furthermore, the architecture proved to be highly versatile, effectively handling variations in the number of sensors. uncertainties included in the experimental data, and elevated levels of measurement noise.

A Deep Learning-Based Animation Video Image Data Anomaly Detection and Recognition Algorithm

Anomaly detection plays a crucial role in the field of machine learning, as it involves constructing detection models capable of identifying abnormal samples that deviate from expected patterns, using unlabeled or normal samples. In recent years, there has been a growing interest in integrating anomaly detection into image processing to tackle challenges related to target detection, particularly when dealing with limited sample availability. This paper introduces a novel fully connected network model enhanced with a memory augmentation mechanism. By harnessing the comprehensive feature capabilities of the fully connected network, this model effectively complements the representation capabilities of convolutional neural networks. Additionally, it incorporates a memory module to retain knowledge of normal patterns, thereby enhancing the performance of existing models for video anomaly detection. Furthermore, we present a video anomaly detection system designed to identify abnormal image data within surveillance videos, leveraging the innovative network architecture described above.

Numerical solutions of boundary problems in partial differential equations: A deep learning framework with Green's function

This work introduces Green Networks (GreenNets), a novel class of neural network architectures designed to address a wide spectrum of boundary problems in partial differential equations (PDEs). Demonstrated versatility includes solving the Poisson equation, convection-diffusion equation, and unsteady incompressible Navier-Stokes equations. GreenNets comprises two specialized variants: Boundary Transformation Green Network (BT-GreenNet), focusing on transforming Dirichlet boundary conditions into source terms, and Boundary Integral Green Network (BI-GreenNet), considering boundary integral terms with shared weights between the boundary and source integrals. Comparisons with existing deep learning and traditional numerical methods underscore their accuracy and efficiency. BT-GreenNet excels in efficiency, and BI-GreenNet offers enhanced flexibility. The research pinnacle, Boundary Unified Green Network (BU-GreenNet), merges BT-GreenNet and BI-GreenNet to address coupled PDEs, specifically unsteady incompressible Navier-Stokes equations. BU-GreenNet stands out by surpassing traditional numerical approaches, offering remarkable computational efficiency while faithfully capturing the intricate evolution of fluid dynamics phenomena. This research redefines the means of solution to boundary problems in PDEs through innovative neural network architectures, emphasizing practical utility and computational efficiency.

Synergistic fusion network for landslide segmentation: the vital complementarity of geological information to remote sensing imagery

In the contemporary technological environment, access to a multitude of data sources, such as digital elevation models (DEM), slope data, and other geological information, has become essential for augmenting remote sensing imagery in landslide analysis. These datasets enhance landslide identification capabilities, but existing methods often fail to fully leverage their unique characteristics due to oversimplified fusion strategies. To overcome this challenge, we introduce the Synergistic Fusion Network (SFNet), which is specifically designed to utilize the complementary nature of geological data to significantly improve the accuracy and efficiency of landslide segmentation. SFNet incorporates Multi-Source Fusion Modules (MSFM) and Reverse Feature Refinement Modules(RFRM), through an innovative network architecture, enhancing remote sensing imagery with integrated geological information. In our experiments, SFNet outperforms existing mainstream networks, demonstrating superior F1 scores, thereby showcasing its ability to effectively segment landslide regions with high precision. This advancement highlights the importance of considering the unique contributions of each data source, especially the complementary effects of geological information.

Pushing the Boundaries of Solar Panel Inspection: Elevated Defect Detection with YOLOv7-GX Technology

During the maintenance and management of solar photovoltaic (PV) panels, how to efficiently solve the maintenance difficulties becomes a key challenge that restricts their performance and service life. Aiming at the multi-defect-recognition challenge in PV-panel image analysis, this study innovatively proposes a new algorithm for the defect detection of PV panels incorporating YOLOv7-GX technology. The algorithm first constructs an innovative GhostSlimFPN network architecture by introducing GSConv and depth-wise separable convolution technologies, optimizing the traditional neck network structure. Then, a customized 1 × 1 convolutional module incorporating the GAM (Global Attention Mechanism) attention mechanism is designed in this paper to improve the ELAN structure, aiming to enhance the network’s perception and representation capabilities while controlling the network complexity. In addition, the XIOU loss function is introduced in the study to replace the traditional CIOU loss function, which effectively improves the robustness and convergence efficiency of the model. In the training stage, the sample imbalance problem is effectively solved by implementing differentiated weight allocations for different images and categories, which promotes the balance of the training process. The experimental data show that the optimized model achieves 94.8% in the highest mAP value, which is 6.4% higher than the original YOLOv7 network, significantly better than other existing models, and provides solid theoretical and technical support for further research and application in the field of PV-panel defect detection.

Innovative Exploration of a Bio-Inspired Sensor Fusion Algorithm: Enhancing Micro Satellite Functionality through Touretsky's Decentralized Neural Networks

Insect-inspired sensor fusion algorithms have presented a promising avenue in the development of robust and efficient systems, owing to the insects' ability to process numerous streams of noisy sensory data. The ring attractor neural network architecture has been identified as a noteworthy model for the optimal integration of diverse insect sensors. Expanding on this, our research presents an innovative bio-inspired ring attractor neural network architecture designed to augment the performance of microsatellite attitude determination systems through the fusion of data from multiple gyroscopic sensors.Extensive simulations using a nonlinear model of the microsatellite, while incorporating specific navigational disturbances, have been conducted to ascertain the viability and effectiveness of this approach. The results obtained have been superior to those of alternative methodologies, thus highlighting the potential of our proposed bio-inspired fusion technique. The findings indicate that this approach could significantly improve the accuracy and robustness of microsatellite systems across a wide range of applications.

A Deep Learning Model for Neural Network Optimization for Glaucoma Classification Using Fundus and Oct Feature Fusion

Irreversible vision loss is a common consequence of glaucoma, demands accurate and timely diagnosis for effective management. This research aims to enhance glaucoma classification accuracy by fusing information from two distinct imaging modalities like using deep learning explores the fusion of these modalities through an innovative neural network architecture with optimization. This approach combines Deep Stochastic Variational Autoencoder Convolution Neural Networks (DSVAECNN) and Adam optimization techniques to enable robust and accurate classification of glaucoma. A multi-path architecture is designed to accommodate both imaging as optical coherence tomography (OCT) radiomics features and fundus morphological features simultaneously. To ensure the effectiveness of the model, this study investigates the implementation of advanced optimization algorithm such as Adam, to expedite convergence and alleviate the risk of over fitting. The resulting model demonstrates improved generalization capabilities, critically for accurate diagnosis across diverse patient populations. A comprehensive OCT and fundus images dataset is used to evaluate the proposed approach from a representative cohort of glaucoma patients and healthy individuals. Quantitative various metrics including sensitivity, accuracy and specificity are employed toward appraise the accomplishment of the fusion-based classification model. Comparisons with current techniques show the dominance of the proposed move toward in accurately detecting glaucoma cases. This research provides the advancement of glaucoma diagnosis by effectively harnessing the synergy between fundus image and OCT scans and findings helpful information for clinics, ultimately facilitating early detection and personalized management of glaucoma, thus preserving vision for affected individuals.

Cross-and-Diagonal Networks: An Indirect Self-Attention Mechanism for Image Classification.

In recent years, computer vision has witnessed remarkable advancements in image classification, specifically in the domains of fully convolutional neural networks (FCNs) and self-attention mechanisms. Nevertheless, both approaches exhibit certain limitations. FCNs tend to prioritize local information, potentially overlooking crucial global contexts, whereas self-attention mechanisms are computationally intensive despite their adaptability. In order to surmount these challenges, this paper proposes cross-and-diagonal networks (CDNet), innovative network architecture that adeptly captures global information in images while preserving local details in a more computationally efficient manner. CDNet achieves this by establishing long-range relationships between pixels within an image, enabling the indirect acquisition of contextual information. This inventive indirect self-attention mechanism significantly enhances the network's capacity. In CDNet, a new attention mechanism named "cross and diagonal attention" is proposed. This mechanism adopts an indirect approach by integrating two distinct components, cross attention and diagonal attention. By computing attention in different directions, specifically vertical and diagonal, CDNet effectively establishes remote dependencies among pixels, resulting in improved performance in image classification tasks. Experimental results highlight several advantages of CDNet. Firstly, it introduces an indirect self-attention mechanism that can be effortlessly integrated as a module into any convolutional neural network (CNN). Additionally, the computational cost of the self-attention mechanism has been effectively reduced, resulting in improved overall computational efficiency. Lastly, CDNet attains state-of-the-art performance on three benchmark datasets for similar types of image classification networks. In essence, CDNet addresses the constraints of conventional approaches and provides an efficient and effective solution for capturing global context in image classification tasks.

Progressive Adoption of RINA in IoT Networks: Enhancing Scalability and Network Management via SDN Integration

Thousands of devices are connected to the Internet as part of the Internet of Things (IoT) ecosystems. The next generation of IoT networks is expected to support this growing number of Intelligent IoT devices and tactile Internet solutions to provide real-time applications. In view of this, IoT networks require innovative network architectures that offer scalability, security, and adaptability. The Recursive InterNetwork Architecture (RINA) is a clean slate network architecture that provides a scalable, secure, and flexible framework for interconnecting computers. SDN technology is becoming a de facto solution to overcome network requirements, making RINA adoption difficult. This paper presents an architecture for integrating RINA with SDN technologies to lower the barriers of adopting RINA in IoT environments. The architecture relies on a RINA-based distributed application facility (DAF), a RINA southbound driver (SBI), and the RINA L2VPN. The RINA-based DAF manages RINA nodes along the edge–fog–cloud continuum. The SBI driver SDN enables the hybrid centralized management of SDN switches and RINA nodes. Meanwhile, the RINA L2VPN allows seamless communication between edge nodes and the cloud to facilitate the data exchange between network functions (NFs). Such integration has enabled a progressive deployment of RINA in current IoT networks without affecting their operations and performance.

Research on road surface crack detection based on SegNet network

To enhance the precision and reliability of road crack detection, this study introduces an innovative neural network architecture. Strategies were implemented to effectively address the issue of overfitting resulting from the intricacy of the proposed SegCrackNet. Dropout layers, multi-level output fusion, and T-bridge block structures are employed in the network. This optimization allows for a more comprehensive exploitation of contextual information, demonstrating its instrumental role in the efficient detection of subtle variations. Experimental findings clearly demonstrate substantial improvements when compared to other network models. On the Crack500, Crack200, and pavement images datasets, remarkable enhancements in the average Intersection over Union (IoU) scores were observed, with increases of 4.3%, 9.4%, and 3.7%, respectively.

MaskTrack: Auto-Labeling and Stable Tracking for Video Object Segmentation.

Video object segmentation (VOS) has witnessed notable progress due to the establishment of video training datasets and the introduction of diverse, innovative network architectures. However, video mask annotation is a highly intricate and labor-intensive task, as meticulous frame-by-frame comparisons are needed to ascertain the positions and identities of targets in the subsequent frames. Current VOS benchmarks often annotate only a few instances in each video to save costs, which, however, hinders the model's understanding of the complete context of the video scenes. To simplify video annotation and achieve efficient dense labeling, we introduce a zero-shot auto-labeling strategy based on the segment anything model (SAM), enabling it to densely annotate video instances without access to any manual annotations. Moreover, although existing VOS methods demonstrate improving performance, segmenting long-term and complex video scenes remains challenging due to the difficulties in stably discriminating and tracking instance identities. To this end, we further introduce a new framework, MaskTrack, which excels in long-term VOS and also exhibits significant performance advantages in distinguishing instances in complex videos with densely packed similar objects. We conduct extensive experiments to demonstrate the effectiveness of the proposed method and show that without introducing image datasets for pretraining, it achieves excellent performance on both short-term (86.2% in YouTube-VOS val) and long-term (68.2% in LVOS val) VOS benchmarks. Our method also surprisingly demonstrates strong generalization ability and performs well in visual object tracking (VOT) (65.6% in VOTS2023) and referring VOS (RVOS) (65.2% in Ref YouTube VOS) challenges.

A Convolutional Neural Network for Beam Prediction in 5G and Beyond Massive MIMO Systems

This paper presents an innovative Convolutional Neural Network (CNN) architecture designed for the prediction of beamforming vectors in Massive Multiple-Input Multiple-Output (MIMO) systems, a cornerstone technology in 5G and emerging wireless communication networks. As mmWave frequencies become central to modern MIMO systems, ensuring precise beam alignment amid susceptibility to blockages is imperative for maintaining robust communication links. Traditional beamforming algorithms, challenged by high complexity and sluggish adaptation to dynamic environmental conditions, are outperformed by the proposed CNN, which effectively captures complex channel patterns for accurate beam prediction. Leveraging a rich dataset from the DeepSense 6G scenarios, the study meticulously preprocesses the data through normalization and scaling before dividing it into training and testing subsets. The CNN, evaluated against real-world signal propagation behaviors, demonstrates a high prediction accuracy in beam indices, particularly in clear line-of-sight scenarios. The paper concludes with an assessment of the model's predictive performance, using Mean Absolute Error as a key metric, and proposes future enhancements to the model's architecture and training process. The implications of this work are significant, heralding a new era of intelligent, self-optimizing wireless networks that can adapt autonomously to environmental changes and user mobility

BrainNet: A Deep Learning Approach for Brain Tumor Classification

Cancer, regardless of its type, represents a formidable threat to human life and disrupts the delicate balance of normal bodily functions. Among the various forms of cancer, malignant brain tumors stand out as a leading cause of mortality in both adult and pediatric populations. The timely identification of these brain tumors is crucial for attaining precise diagnoses. Brain tumour identification and diagnosis are now made possible by the use of magnetic resonance imaging (MRI). However, the intricate and irregular shapes and locations of these tumors often pose challenges for complete comprehension. Typically, the expertise of neurosurgical specialists is required for the precise analysis of MRI scans. Unfortunately, in many developing countries, a shortage of skilled medical professionals and limited awareness about brain tumors compound the difficulties associated with obtaining timely and accurate MRI results. To address these notable challenges, this research introduces BrainNet, an innovative Convolutional Neural Network (CNN) architecture specifically designed for the classification of brain tumors into distinct categories. The established transfer learning models VGG13, VGG19, VGG16, InceptionResV2, and Squeeznet, all of which were pretrained on the Imagenet dataset, are outperformed by BrainNet in both how well it handles these problems and how well it outperforms them. The performance of the BrainNet CNN architecture is particularly impressive, with a precision score of 94.75 percent and accuracy rates of 99.96 percent during training and 97.71 percent during testing. This accomplishment has the potential to significantly improve brain tumour diagnosis and classification, particularly in areas with limited access to medical resources and knowledge.

A Novel CNN Model for Classification of Chinese Historical Calligraphy Styles in Regular Script Font.

Chinese calligraphy, revered globally for its therapeutic and mindfulness benefits, encompasses styles such as regular (Kai Shu), running (Xing Shu), official (Li Shu), and cursive (Cao Shu) scripts. Beginners often start with the regular script, advancing to more intricate styles like cursive. Each style, marked by unique historical calligraphy contributions, requires learners to discern distinct nuances. The integration of AI in calligraphy analysis, collection, recognition, and classification is pivotal. This study introduces an innovative convolutional neural network (CNN) architecture, pioneering the application of CNN in the classification of Chinese calligraphy. Focusing on the four principal calligraphy styles from the Tang dynasty (690-907 A.D.), this research spotlights the era when the traditional regular script font (Kai Shu) was refined. A comprehensive dataset of 8282 samples from these calligraphers, representing the zenith of regular style, was compiled for CNN training and testing. The model distinguishes personal styles for classification, showing superior performance over existing networks. Achieving 89.5-96.2% accuracy in calligraphy classification, our approach underscores the significance of CNN in the categorization of both font and artistic styles. This research paves the way for advanced studies in Chinese calligraphy and its cultural implications.

Pioneering Innovation in Network Architecture: Revolutionizing Connectivity in the Digital Era

This paper explores an innovative network architecture designed for the dynamic technological demands of the 21st century. Emphasizing scalability, performance optimization, and robust security, the architecture integrates technologies like software-defined networking, edge computing, and blockchain. It embodies adaptability, predictive operations, and fortified security measures. The methodology involves research, iterative design, simulation, and real-world deployment, showcasing scalability, enhanced performance, and robust security. This architecture sets new standards for connectivity, resilience, and security in the digital era, fostering ongoing innovation and a more efficient, secure digital future

Optimizing Image Classification: Automated Deep Learning Architecture Crafting with Network and Learning Hyperparameter Tuning.

This study introduces ETLBOCBL-CNN, an automated approach for optimizing convolutional neural network (CNN) architectures to address classification tasks of varying complexities. ETLBOCBL-CNN employs an effective encoding scheme to optimize network and learning hyperparameters, enabling the discovery of innovative CNN structures. To enhance the search process, it incorporates a competency-based learning concept inspired by mixed-ability classrooms during the teacher phase. This categorizes learners into competency-based groups, guiding each learner's search process by utilizing the knowledge of the predominant peers, the teacher solution, and the population mean. This approach fosters diversity within the population and promotes the discovery of innovative network architectures. During the learner phase, ETLBOCBL-CNN integrates a stochastic peer interaction scheme that encourages collaborative learning among learners, enhancing the optimization of CNN architectures. To preserve valuable network information and promote long-term population quality improvement, ETLBOCBL-CNN introduces a tri-criterion selection scheme that considers fitness, diversity, and learners' improvement rates. The performance of ETLBOCBL-CNN is evaluated on nine different image datasets and compared to state-of-the-art methods. Notably, ELTLBOCBL-CNN achieves outstanding accuracies on various datasets, including MNIST (99.72%), MNIST-RD (96.67%), MNIST-RB (98.28%), MNIST-BI (97.22%), MNST-RD + BI (83.45%), Rectangles (99.99%), Rectangles-I (97.41%), Convex (98.35%), and MNIST-Fashion (93.70%). These results highlight the remarkable classification accuracy of ETLBOCBL-CNN, underscoring its potential for advancing smart device infrastructure development.

MTLBORKS-CNN: An Innovative Approach for Automated Convolutional Neural Network Design for Image Classification

Convolutional neural networks (CNNs) have excelled in artificial intelligence, particularly in image-related tasks such as classification and object recognition. However, manually designing CNN architectures demands significant domain expertise and involves time-consuming trial-and-error processes, along with substantial computational resources. To overcome this challenge, an automated network design method known as Modified Teaching-Learning-Based Optimization with Refined Knowledge Sharing (MTLBORKS-CNN) is introduced. It autonomously searches for optimal CNN architectures, achieving high classification performance on specific datasets without human intervention. MTLBORKS-CNN incorporates four key features. It employs an effective encoding scheme for various network hyperparameters, facilitating the search for innovative and valid network architectures. During the modified teacher phase, it leverages a social learning concept to calculate unique exemplars that effectively guide learners while preserving diversity. In the modified learner phase, self-learning and adaptive peer learning are incorporated to enhance knowledge acquisition of learners during CNN architecture optimization. Finally, MTLBORKS-CNN employs a dual-criterion selection scheme, considering both fitness and diversity, to determine the survival of learners in subsequent generations. MTLBORKS-CNN is rigorously evaluated across nine image datasets and compared with state-of-the-art methods. The results consistently demonstrate MTLBORKS-CNN’s superiority in terms of classification accuracy and network complexity, suggesting its potential for infrastructural development of smart devices.