Reduce Memory Consumption Research Articles

IoT Security: Botnet Detection Using Self-Organizing Feature Map and Machine Learning

The rapid advancement of Internet of Things (IoT) technology has created potential for progress in various aspects of life. However, the increasing number of IoT devices also raises the risk of cyberattacks, particularly IoT botnets often exploited by attackers. This is largely due to the limitations of IoT devices, such as constraints in capacity, power, and memory, necessitating an efficient detection system. This study aims to develop a resource-efficient botnet detection system by using the Self-Organizing Feature Map (SOFM) dimensionality reduction method in combination with machine learning algorithms. The proposed method includes a feature engineering process using SOFM to address high-dimensional data, followed by classification with various machine learning algorithms. The experiments evaluate performance based on accuracy, sensitivity, specificity, False Positive Rate (FPR), and False Negative Rate (FNR). Results show that the Decision Tree algorithm achieved the highest accuracy rate of 97.24%, with a sensitivity of 0.9523, specificity of 0.9932, and a fast execution time of 100.66 seconds. The use of SOFM successfully reduced memory consumption from 3.08 GB to 923MB. Experimental results indicate that this approach is effective for enhancing IoT security in resource-constrained devices.

GoalBERT: A Lightweight Named-Entity Recognition Model Based on Multiple Fusion

Named-Entity Recognition (NER) as a core task in Natural Language Processing (NLP) aims to automatically identify and classify specific types of entities from unstructured text. In recent years, the introduction of Transformer architecture and its derivative BERT model has pushed the performance of NER to unprecedented heights. However, these models often have high requirements for computational power and memory resources, making it difficult to train and deploy them on small computing platforms. Although ALBERT as a lightweight model uses parameter sharing and matrix decomposition strategies to reduce memory consumption to some extent consumption, it does not effectively reduce the computational load of the model. Additionally, due to its internal sharing mechanism, the model’s understanding ability of text is reduced leading to poor performance in named-entity recognition tasks. To address these challenges, this manuscript proposes an efficient lightweight model called GoalBERT. The model adopts multiple fusion technologies by integrating a lightweight and efficient BiGRU that excels at handling context into part of the Transformer’s self-attention layers. This reduces the high computational demand caused by stacking multiple self-attention layers while enhancing the model’s ability to process context information. To solve the problem of gradient disappearance and explosion during training, residual connections are added between core layers for more stable training and steady performance improvement. Experimental results show that GoalBERT demonstrates recognition accuracy comparable to standard models with accuracy surpassing ALBERT by 10% in multi-entity type scenarios. Furthermore, compared to standard models, GoalBERT reduces memory requirements by 200% and improves training speed by nearly 230%. Experimental results indicate that GoalBERT is a high-quality lightweight model.

Hybrid Computational Intelligence Models for Robust Pattern Recognition and Data Analysis

In the era of big data, robust pattern recognition and accurate data analysis have become critical in various fields, including healthcare, finance, and industrial automation. This study presents a novel hybrid computational intelligence model that integrates deep learning techniques and evolutionary algorithms to enhance the precision and resilience of pattern recognition tasks. Our proposed model combines Convolutional Neural Networks (CNN) for high-dimensional feature extraction with a Genetic Algorithm (GA) for feature optimization and selection, providing a more efficient approach to processing complex datasets. The hybrid model achieved an accuracy of 98.7% on the MNIST dataset and outperformed conventional methods in terms of recall (95.5%) and precision (97.2%) on large-scale image classification tasks. Additionally, it demonstrated substantial improvements in computation time, reducing processing duration by 35% over traditional deep learning approaches. Experimental results on diverse datasets, including time-series and unstructured data, confirmed the model's versatility and adaptability, achieving F1-scores of 0.92 in healthcare data analysis and 0.89 in financial anomaly detection. By incorporating a Particle Swarm Optimization (PSO) algorithm, the model further optimized hyperparameters, leading to a 25% reduction in memory consumption without compromising model performance. This approach not only enhances computational efficiency but also enables the model to perform reliably in resource-constrained environments. Our results suggest that hybrid computational intelligence models offer a promising solution for robust, scalable pattern recognition and data analysis, addressing the evolving demands of real-world applications.

Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Bag-Level Classifier is a Good Instance-Level Teacher.

Multiple Instance Learning (MIL) has demonstrated promise in Whole Slide Image (WSI) classification. However, a major challenge persists due to the high computational cost associated with processing these gigapixel images. Existing methods generally adopt a two-stage approach, comprising a non-learnable feature embedding stage and a classifier training stage. Though it can greatly reduce memory consumption by using a fixed feature embedder pre-trained on other domains, such a scheme also results in a disparity between the two stages, leading to suboptimal classification accuracy. To address this issue, we propose that a bag-level classifier can be a good instance-level teacher. Based on this idea, we design Iteratively Coupled Multiple Instance Learning (ICMIL) to couple the embedder and the bag classifier at a low cost. ICMIL initially fixes the patch embedder to train the bag classifier, followed by fixing the bag classifier to fine-tune the patch embedder. The refined embedder can then generate better representations in return, leading to a more accurate classifier for the next iteration. To realize more flexible and more effective embedder fine-tuning, we also introduce a teacher-student framework to efficiently distill the category knowledge in the bag classifier to help the instance-level embedder fine-tuning. Intensive experiments were conducted on four distinct datasets to validate the effectiveness of ICMIL. The experimental results consistently demonstrated that our method significantly improves the performance of existing MIL backbones, achieving state-of-the-art results. The code and the organized datasets can be accessed by: https://github.com/Dootmaan/ICMIL/tree/confidence-based.

MPSA-Conformer-CTC/Attention: A High-Accuracy, Low-Complexity End-to-End Approach for Tibetan Speech Recognition.

This study addresses the challenges of low accuracy and high computational demands in Tibetan speech recognition by investigating the application of end-to-end networks. We propose a decoding strategy that integrates Connectionist Temporal Classification (CTC) and Attention mechanisms, capitalizing on the benefits of automatic alignment and attention weight extraction. The Conformer architecture is utilized as the encoder, leading to the development of the Conformer-CTC/Attention model. This model first extracts global features from the speech signal using the Conformer, followed by joint decoding of these features through CTC and Attention mechanisms. To mitigate convergence issues during training, particularly with longer input feature sequences, we introduce a Probabilistic Sparse Attention mechanism within the joint CTC/Attention framework. Additionally, we implement a maximum entropy optimization algorithm for CTC, effectively addressing challenges such as increased path counts, spike distributions, and local optima during training. We designate the proposed method as the MaxEnt-Optimized Probabilistic Sparse Attention Conformer-CTC/Attention Model (MPSA-Conformer-CTC/Attention). Experimental results indicate that our improved model achieves a word error rate reduction of 10.68% and 9.57% on self-constructed and open-source Tibetan datasets, respectively, compared to the baseline model. Furthermore, the enhanced model not only reduces memory consumption and training time but also improves generalization capability and accuracy.

Evaluating the impact of downsampling on 3D MRI images segmentation results based on similarity metrics

Medical imaging plays a crucial role in diagnosing patient conditions, with magnetic resonance imaging (MRI) standing as a significant modality for numerous years. However, leveraging convolutional neural network (CNN) architectures like U-Net and its variations for anatomical segmentation demands considerable memory, particularly when working with full 3D image sets. Therefore, downsampling 3D MRIs proves advantageous in reducing memory consumption. Nevertheless, downsampling leads to a reduction in voxel count, potentially impacting the performance of commonly used segmentation metrics. The jaccard similarity index (JSI), dice similarity coefficient (DSC), and structural similarity index (SSIM) are extensively employed in image segmentation contexts. Hence, this study employs all three metrics to assess downsampled images and evaluate the robustness of the metrics when used to evaluate the downsampled 3D MRI images. The results show that JSI and DSC are more robust than SSIM when handling the downsampled data.

Memory-bound k-mer selection for large and evolutionarily diverse reference libraries.

Using k-mers to find sequence matches is increasingly used in many bioinformatic applications, including metagenomic sequence classification. The accuracy of these downstream applications relies on the density of the reference databases, which are rapidly growing. Although the increased density provides hope for improvements in accuracy, scalability is a concern. Reference k-mers are kept in the memory during the query time, and saving all k-mers of these ever-expanding databases is fast becoming impractical. Several strategies for subsampling have been proposed, including minimizers and finding taxon-specific k-mers. However, we contend that these strategies are inadequate, especially when reference sets are taxonomically imbalanced, as are most microbial libraries. In this paper, we explore approaches for selecting a fixed-size subset of k-mers present in an ultra-large data set to include in a library such that the classification of reads suffers the least. Our experiments demonstrate the limitations of existing approaches, especially for novel and poorly sampled groups. We propose a library construction algorithm called k-mer RANKer (KRANK) that combines several components, including a hierarchical selection strategy with adaptive size restrictions and an equitable coverage strategy. We implement KRANK in highly optimized code and combine it with the locality-sensitive hashing classifier CONSULT-II to build a taxonomic classification and profiling method. On several benchmarks, KRANK k-mer selection significantly reduces memory consumption with minimal loss in classification accuracy. We show in extensive analyses based on CAMI benchmarks that KRANK outperforms k-mer-based alternatives in terms of taxonomic profiling and comes close to the best marker-based methods in terms of accuracy.

Dear-PSM: A deep learning-based peptide search engine enables full database search for proteomics.

Peptide spectrum matching is the process of linking mass spectrometry data with peptide sequences. An experimental spectrum can match thousands of candidate peptides with variable modifications leading to an exponential increase in candidates. Completing the search within a limited time is a key challenge. Traditional searches expedite the process by restricting peptide mass errors and variable modifications, but this limits interpretive capability. To address this challenge, we propose Dear-PSM, a peptide search engine that supports full database searching. Dear-PSM does not restrict peptide mass errors, matching each spectrum to all peptides in the database and increasing the number of variable modifications per peptide from the conventional 3-20. Leveraging inverted index technology, Dear-PSM creates a high-performance index table of experimental spectra and utilizes deep learning algorithms for peptide validation. Through these techniques, Dear-PSM achieves a speed breakthrough 7 times faster than mainstream search engines on a regular desktop computer, with a remarkable 240-fold reduction in memory consumption. Benchmark test results demonstrate that Dear-PSM, in full database search mode, can reproduce over 90% of the results obtained by mainstream search engines when handling complex mass spectrometry data collected from different species using various instruments. Furthermore, it uncovers a substantial number of new peptides and proteins. Dear-PSM has been publicly released on the GitHub repository https://github.com/jianweishuai/Dear-PSM.

On-the-Fly Unsteady Adjoint Aerodynamic and Aeroacoustic Optimization Method

An on-the-fly unsteady adjoint-based aerodynamic and aeroacoustic optimization methodology is presented, aiming to achieve practical engineering applications to explore high-efficiency and low-noise design for aerodynamic shapes. Firstly, a novel on-the-fly hybrid CFD-CAA approach is developed with a close integration of unsteady Reynolds-averaged Navier–Stokes equations and a fully viscous time-domain FW-H formulation. Subsequently, an adjoint-based sensitivity analysis method is proposed for unsteady aerodynamic and aeroacoustic problems with either stationary or moving boundaries, wherein a unified architecture for discrete-adjoint sensitivity analysis of both aerodynamics and aeroacoustics is achieved by integrating the on-the-fly hybrid CFD-CAA approach. The on-the-fly approach facilitates direct evaluation of partial derivatives required for solving adjoint equations, eliminating the need for explicitly preprocessing flow and adjoint variables at all time levels in a standalone adjoint CAA solver and consequently substantially reducing memory consumption. The proposed optimization methodology is implemented within an open-source suite SU2. Results show that the proposed on-the-fly adjoint methodology is capable of achieving highly accurate sensitivity derivatives while significantly reducing memory requirements by an order of magnitude, and further demonstrations of single-objective and coupled aerodynamic and aeroacoustic optimizations highlight the potential of the proposed method in exploring high-efficiency and low-noise design for aerodynamic shapes.

MultiSplit: An Efficient Algorithm for Packet Classification with Equivalent Priority

Packet classification is a core function of network devices for providing advanced services, with the key challenge being to optimize classification speed while maintaining low memory usage. So far, many have proposed software-based packet classification solutions, with most of them adopting a multi-classifier architectures to accommodate the distribution of rule sets. Unfortunately, the need to perform lookups on each classifier during the packet classification stage significantly increases overhead, severely limiting classification speed. To address this shortfall, an efficient packet classification framework based on decision tree algorithms named MultiSplit is proposed. By leveraging the relationships of coverage and priority within the rule set, a new attribute can be abstracted for each rule, termed equivalent priority. Through this preprocessing, MultiSplit significantly reduces redundant lookup overhead while supporting the multi-classifier framework. Additionally, MultiSplit introduces a novel decision tree algorithm that combines multiple splits and intra-level binary search, significantly improving rule separation efficiency. The experimental results show that MultiSplit reduces memory consumption by 49% and decreases memory access by 63%, on average, compared with state-of-the-art packet classification algorithms.

P4Rex: Accelerating regular expression matching with programmable switches

Regular expression matching is pivotal in numerous network applications. With the ever-increasing scale of data center traffic, deploying regular expression matching modules on traditional servers struggles to meet throughput demands. Emerging programmable switches have brought new prospects for high-speed pattern matching. However, deploying regular expression matching onto programmable switches presents the challenge of space explosion incurred by compiling regular expressions into a Deterministic Finite Automata (DFA). In this paper, we introduce P4Rex, a regular expression matching system designed for programmable switches. P4Rex synergistically leverages two following techniques: an efficient regular expression grouping algorithm that partitions regular expressions into different groups to reduce memory consumption, and DFA compression technology to achieve transition sharing. Experimental results demonstrate that P4Rex exhibits an average improvement of 17% and maximum improvement of 30% on memory consumption compared to prior regular expression grouping schemes and saves more than 10x memory consumption compared to deploying DFA directly on programmable switch.

Dual path features interaction network for efficient image super-resolution

Image super-resolution (SR) is a crucial task in computer vision that involves reconstructing a low-resolution (LR) image into its high-resolution (HR) counterpart. Transformer-based methods excel at establishing long-range dependency but face challenges with high-complexity computations and capturing fine-grained features. Conversely, CNN-based methods are advantageous in capturing fine-grained features but are limited by fixed receptive fields. Additionally, most SR methods suffer from channel redundancy, leading to higher computational overhead. In this paper, we propose a Dual Path Features Interaction Network (DPFINet) to achieve efficient image SR, which consists of two components: a) To alleviate the issue of feature channel redundancy, a Local–Global Features Modeling (LGFM) method is newly proposed, which concurrently models global and local features by splitting features along different channels. In LGFM, a Shift Window Linear Attention (SWLA) layer is adopted to effectively capture global information through a large shift window based on the split features. Meanwhile, a Multi-Scale Detail Enhancement (MSDE) layer is designed, where the split other features are encoded to facilitate detail reconstruction through an interactive fusion of semantic and local information, thereby addressing the limitations of SWLA in capturing fine-grained features. b) A Cross-Level Features Interaction (CLFI) method is proposed to fuse global and local features modeled by different network structures (SWLA and MSDE), where a novel residual fusion mechanism is designed to preserve both global and local information while complementing each other. Extensive experiments demonstrate that our method outperforms most state-of-the-art SR methods on five benchmark datasets. Notably, during inference, our approach improves performance by 0.41 dB and reduces memory consumption by approximately 79% compared to DiVANet (Behjati et al., 2023) on the Manga109 (×4) dataset.

Improved research on coral bleaching detection model based on FCOS model

Coral bleaching detection is vital for assessing coral reef health. This paper introduces FCOS_EfficientNET, an improved model that enhances accuracy, recall, and real-time performance in coral bleaching detection. Utilizing EfficientNet as the backbone, the model optimizes parameter throughput. We adopt the ReLU activation function and utilize cosine similarity and softmax to assign weights to datasets, modifying the attention structure to reduce memory consumption. The model also integrates BiFPN for better feature extraction and employs an improved training method to enhance detection accuracy. To cater to different scenarios, we have developed four variants: FCOS_EfficientNETb0, FCOS_EfficientNETb1, FCOS_EfficientNETb2, and FCOS_EfficientNETb3. Experimental results on the MS COCO dataset show that FCOS_EfficientNETb3 achieves a mean average precision (mAP) of 48.5%, while FCOS_EfficientNETb0 reaches a frame rate of 167.17 fps, highlighting the superior performance of the series. On a custom coral bleaching detection dataset, FCOS_EfficientNETb3 achieves 81.5% accuracy and a 59.3% recall rate, demonstrating the effectiveness of the model. FCOS_EfficientNETb1 and FCOS_EfficientNETb2 offer a balance between operations per second, frame rate, and mAP, making them suitable for mobile and edge computing. These models effectively track movement or changes in marine traffic around coral reefs with moderate fps and recall rates.

Memory-bound k -mer selection for large and evolutionary diverse reference libraries.

Using k -mers to find sequence matches is increasingly used in many bioinformatic applications, including metagenomic sequence classification. The accuracy of these down-stream applications relies on the density of the reference databases, which, luckily, are rapidly growing. While the increased density provides hope for dramatic improvements in accuracy, scalability is a concern. Reference k -mers are kept in the memory during the query time, and saving all k -mers of these ever-expanding databases is fast becoming impractical. Several strategies for subsampling have been proposed, including minimizers and finding taxon-specific k -mers. However, we contend that these strategies are inadequate, especially when reference sets are taxonomically imbalanced, as are most microbial libraries. In this paper, we explore approaches for selecting a fixed-size subset of k -mers present in an ultra-large dataset to include in a library such that the classification of reads suffers the least. Our experiments demonstrate the limitations of existing approaches, especially for novel and poorly sampled groups. We propose a library construction algorithm called KRANK (K-mer RANKer) that combines several components, including a hierarchical selection strategy with adaptive size restrictions and an equitable coverage strategy. We implement KRANK in highly optimized code and combine it with the locality-sensitive-hashing classifier CONSULT-II to build a taxonomic classification and profiling method. On several benchmarks, KRANK k -mer selection dramatically reduces memory consumption with minimal loss in classification accuracy. We show in extensive analyses based on CAMI benchmarks that KRANK outperforms k -mer-based alternatives in terms of taxonomic profiling and comes close to the best marker-based methods in terms of accuracy.

Cross-layer importance evaluation for neural network pruning

Filter pruning has achieved remarkable success in reducing memory consumption and speeding up inference for convolutional neural networks (CNNs). Some prior works, such as heuristic methods, attempted to search for suitable sparse structures during the pruning process, which may be expensive and time-consuming. In this paper, an efficient cross-layer importance evaluation (CIE) method is proposed to automatically calculate proportional relationships among convolutional layers. Firstly, every layer is pruned separately by grid sampling way to obtain the accuracy of the model for all sampling points. And then, contribution matrices are built to describe the importance of each layer to model accuracy. Finally, the binary search algorithm is used to search the optimal sparse structure under a target pruned value. Extensive experiments on multiple representative image classification tasks demonstrate that proposed method acquires better compression performance under a little time cost compared to existing pruning algorithms. For instance, it reduces more than 50% FLOPs with only a small loss of 0.93% and 0.43% in the top-1 and top-5 accuracy for ResNet50, respectively. At the cost of only 0.24% accuracy loss, the pruned VGG19 model parameters are successfully compressed by 27.23× and the throughput has increased by 2.46×. On the whole, CIE has an excellent effect on the deployment and application of the CNNs model in edge device in terms of efficiency and accuracy.

BinVPR: Binary Neural Networks towards Real-Valued for Visual Place Recognition.

Visual Place Recognition (VPR) aims to determine whether a robot or visual navigation system locates in a previously visited place using visual information. It is an essential technology and challenging problem in computer vision and robotic communities. Recently, numerous works have demonstrated that the performance of Convolutional Neural Network (CNN)-based VPR is superior to that of traditional methods. However, with a huge number of parameters, large memory storage is necessary for these CNN models. It is a great challenge for mobile robot platforms equipped with limited resources. Fortunately, Binary Neural Networks (BNNs) can reduce memory consumption by converting weights and activation values from 32-bit into 1-bit. But current BNNs always suffer from gradients vanishing and a marked drop in accuracy. Therefore, this work proposed a BinVPR model to handle this issue. The solution is twofold. Firstly, a feature restoration strategy was explored to add features into the latter convolutional layers to further solve the gradient-vanishing problem during the training process. Moreover, we identified two principles to address gradient vanishing: restoring basic features and restoring basic features from higher to lower layers. Secondly, considering the marked drop in accuracy results from gradient mismatch during backpropagation, this work optimized the combination of binarized activation and binarized weight functions in the Larq framework, and the best combination was obtained. The performance of BinVPR was validated on public datasets. The experimental results show that it outperforms state-of-the-art BNN-based approaches and full-precision networks of AlexNet and ResNet in terms of both recognition accuracy and model size. It is worth mentioning that BinVPR achieves the same accuracy with only 1% and 4.6% model sizes of AlexNet and ResNet.

Bitmap-Based Security Monitoring for Deeply Embedded Systems

Deeply embedded systems powered by microcontrollers are becoming popular with the emergence of Internet of Things (IoT) technology. However, these devices primarily run C/C++ code and are susceptible to memory bugs, which can potentially lead to both control data attacks and non-control data attacks. Existing defense mechanisms (such as control flow integrity (CFI), data flow integrity (DFI) and write integrity testing (WIT), etc.) consume a massive amount of resources, making them less practical in real products. To make it lightweight, we design a bitmap-based allowlist mechanism to unify the storage of the runtime data for protecting both control data and non-control data. The memory requirements are constant and small, regardless of the number of deployed defense mechanisms. We store the allowlist in the TrustZone to ensure its integrity and confidentiality. Meanwhile, we perform an offline analysis to detect potential collisions and make corresponding adjustments when if happens. We have implemented our idea on an ARM Cortex-M based development board. Our evaluation results show a substantial reduction in memory consumption when deploying the proposed CFI and DFI mechanisms, without compromising runtime performance. Specifically, our prototype enforces CFI and DFI at a cost of just 2.09% performance overhead and 32.56% memory overhead on average.

Maritime vessel classification based on a dual network combining EfficientNet with a hybrid network MPANet

AbstractShip classification is an important technique for enhancing maritime management and security. Visible and infrared sensors are generally employed to deal with the challenging problem and improve classification performance. Herein, a two‐branch feature fusion neural network structure is proposed to classify the visible and infrared maritime vessel images simultaneously. Specifically, in this two‐branch neural network, one branch is based on a deep convolutional neural network that is used to extract the visible image features, while the other is a hybrid network structure that is a multi‐scale patch embedding network called MPANet. The sub‐network MPANet can extract fine‐ and coarse‐grained features, in which the pooling operation instead of the multi‐head attention mechanism is utilized to reduce memory consumption. When there are infrared images, it is used to extract the infrared image features, otherwise, this branch is also utilized to extract visible image features. Therefore, this dual network is suitable with or without infrared images. The experimental results on the visible and infrared spectrums (VAIS) dataset demonstrate that the introduced network achieves state‐of‐the‐art ship classification performance on visible images and paired visible and infrared ship images.

Multi pattern matching algorithm for embedded computer network engineering intrusion detection system

In computer networks, security issues persist, and addressing hidden security risks is pivotal for ensuring network security. However, traditional single pattern matching algorithms like BM (Boyer-Moore) lack efficiency for network intrusion detection. This study employs multiple pattern matching algorithms to bolster the security of computer network engineering intrusion detection systems (IDS). A computer network intrusion detection system (NIDS) is designed using embedded technology to collect network logs and other pertinent data, subsequently comparing log data packets. The study delves into a multi pattern matching algorithm, AC (Aho-Corasick), which incorporates the SUNDAY algorithm to optimize unnecessary string matching jumps. Furthermore, the AC algorithm and BM algorithm are fused as control methods. Randomly generated 48M text data is utilized for testing purposes, comparing the AC algorithm, AC-BM algorithm, and AC-SUNDAY algorithm. For instance, when the pattern string length is 20 bytes, the memory consumption of the AC algorithm, AC-BM algorithm, and AC-SUNDAY algorithm is 12.2 MB, 9.8 MB, and 6.2 MB respectively. The findings indicate that applying the AC-SUNDAY algorithm in NIDS effectively reduces memory consumption and enhances the efficacy of network intrusion detection.

AIfES: A Next-Generation Edge AI Framework.

Edge Artificial Intelligence (AI) relies on the integration of Machine Learning (ML) into even the smallest embedded devices, thus enabling local intelligence in real-world applications, e.g. for image or speech processing. Traditional Edge AI frameworks lack important aspects required to keep up with recent and upcoming ML innovations. These aspects include low flexibility concerning the target hardware and limited support for custom hardware accelerator integration. Artificial Intelligence for Embedded Systems Framework (AIfES) has the goal to overcome these challenges faced by traditional edge AI frameworks. In this paper, we give a detailed overview of the architecture of AIfES and the applied design principles. Finally, we compare AIfES with TensorFlow Lite for Microcontrollers (TFLM) on an ARM Cortex-M4-based System-on-Chip (SoC) using fully connected neural networks (FCNNs) and convolutional neural networks (CNNs). AIfES outperforms TFLM in both execution time and memory consumption for the FCNNs. Additionally, using AIfES reduces memory consumption by up to 54% when using CNNs. Furthermore, we show the performance of AIfES during the training of FCNN as well as CNN and demonstrate the feasibility of training a CNN on a resource-constrained device with a memory usage of slightly more than 100 kB of RAM.