Tiny Edge Research Articles

Multi-scale feature extraction and TrasMLP encoder module for ocean HABs segmentation

Due to tiny edge and texture details of harmful algae blooms(HABs), existing segmentation networks are not effective for HABs segmentation. In order to solve the above problems, this paper proposes a Multi-scale Feature extraction and TrasMLP Encoder Fusion-based network (MFTS). To tackle the complex morphological characteristics and the complex backgrounds of HABs, a TrasMlp module which can effectively identify long-range patterns and adapt network parameters is introduced, enabling accurately parsing of complex algae images. Secondly, the deep convolution module is constructed by combining deep separable convolution with a two-channel attention mechanism to separate the target region from the background. In addition, this paper proposes a Weighted Feature Fusion of Deep Convolution and INRS Encoder Module classification network(FDIR) is proposed to quantify the performance of the image segmentation network. The segmentation results on HABs dataset from AICO Lab show that our proposed MFTS model achieves a miou of 90.02%, outperforming the performance of classical segmentation networks such as U-Net and Mask R-CNN. Compared to original HABs dataset, the segmented result shows a 5.1% improvement in the classification accuracy of the FDIR model.

XimSwap: Many-to-Many Face Swapping for TinyML

The unprecedented development of deep learning approaches for video processing has caused growing privacy concerns. To ensure data analysis while maintaining privacy, it is essential to address how to protect individuals’ identities. One solution is to anonymize data at the source, avoiding the transmission or storage of information that could lead to identification. This study introduces XimSwap, a novel deep learning technique for real-time video anonymization, which can remove facial identification features directly on edge devices with minimal computational resources. Our approach offers a comprehensive solution that guarantees privacy by design. This novel method for implementing face-swapping ensures that the pose and expression of a target face remain unchanged and can be used on embedded devices with very limited computational resources. By incorporating style transfer layers into convolutional ones and optimizing the network’s operation, we achieved a reduction of over 98% in the required operations and parameters compared with state-of-the-art architectures. Our approach also significantly reduces RAM usage, making it possible to implement the anonymization process on tiny edge devices, including microcontrollers, such as the STM32H743.

A multi-resolution fusion approach for human activity recognition from video data in tiny edge devices

Human Activity Recognition (HAR) is the process of automatic recognition of Activities of Daily Living (ADL) from human motion data captured in various data modalities by wearable and ambient sensors. Advances in Deep Learning, especially Convolutional Neural Networks (CNNs) and sequential models have revolutionalized HAR from video data sources. Although these models are incredibly accurate and utilize both spatial and temporal information, they require huge computation and memory resources — making them unsuitable for edge or wearable applications. Tiny Machine Learning (TinyML) aims to optimize these models in terms of compute and memory requirements – aiming to make them suitable for always-on resource constrained devices – leading to a reduction in communication latency and network traffic for HAR frameworks. In this paper, we propose a two-stream multi-resolution fusion architecture for HAR from video data modality. The context stream takes a resized image as input and the fovea stream takes the cropped center portion of the resized image as input, reducing the overall dimensionality. We tested two quantization methods: Post-Training Quantization (PTQ) and Quantization Aware Training (QAT) to optimize these models for deployment in edge devices and tested the performance in two challenging video datasets: KTH and UCF11. We performed ablation studies to validate the two-stream model performance. We deployed the proposed architecture in commercial resource constrained devices and monitored their performance in terms of inference latency and power consumption. The results indicate that the proposed architecture clearly outperforms other relevant single-stream models tested in this work in terms of accuracy, precision, recall, and F1-Score while also reducing the overall model size.

Embedded federated learning over a LoRa mesh network

In on-device training of machine learning models on microcontrollers a neural network is trained on the device. A specific approach for collaborative on-device training is federated learning. In this paper, we propose embedded federated learning on microcontroller boards using the communication capacity of a LoRa mesh network. We apply a dual board design: The machine learning application that contains a neural network is trained for a keyword spotting task on the Arduino Portenta H7. For the networking of the federated learning process, the Portenta is connected to a TTGO LORA32 board that operates as a router within a LoRa mesh network. We experiment the federated learning application on the LoRa mesh network and analyze the network, system, and application level performance. The results from our experimentation suggest the feasibility of the proposed system and exemplify an implementation of a distributed application with re-trainable compute nodes, interconnected over LoRa, entirely deployed at the tiny edge.

Selective Sensing of Mixtures of Gases with CMOS-SOI-MEMS Sensor Dubbed GMOS

The need to achieve digital gas sensing technology, namely, a technology to sense and transmit gas-enabled digital media, has been recognized as highly challenging. This challenge has motivated the authors to focus on complementary metal oxide semiconductor silicon on insulator micro electro-mechanical system (CMOS-SOI-MEMS) technologies, and the result is a new pellistor-like sensor, dubbed GMOS, with integrated signal processing. In this study, we describe the performance of such sensors for the selective detection of mixtures of gases. The novel key ideas of this study are: (i) the use of the GMOS for gas sensing; (ii) applying the Kalman filter to improve the signal-to-noise ratio; (iii) adding artificial intelligence (AI) with tiny edge approach.

A Micro Neural Network for Healthcare Sensor Data Stream Classification in Sustainable and Smart Cities.

A smart city is an intelligent space, in which large amounts of data are collected and analyzed using low-cost sensors and automatic algorithms. The application of artificial intelligence and Internet of Things (IoT) technologies in electronic health (E-health) can efficiently promote the development of sustainable and smart cities. The IoT sensors and intelligent algorithms enable the remote monitoring and analyzing of the healthcare data of patients, which reduces the medical and travel expenses in cities. Existing deep learning-based methods for healthcare sensor data classification have made great achievements. However, these methods take much time and storage space for model training and inference. They are difficult to be deployed in small devices to classify the physiological signal of patients in real time. To solve the above problems, this paper proposes a micro time series classification model called the micro neural network (MicroNN). The proposed model is micro enough to be deployed on tiny edge devices. MicroNN can be applied to long-term physiological signal monitoring based on edge computing devices. We conduct comprehensive experiments to evaluate the classification accuracy and computation complexity of MicroNN. Experiment results show that MicroNN performs better than the state-of-the-art methods. The accuracies on the two datasets (MIT-BIH-AR and INCART) are 98.4% and 98.1%, respectively. Finally, we present an application to show how MicroNN can improve the development of sustainable and smart cities.

An improved coupled PDE system applied to the inverse image denoising problem

<abstract><p>The problem of interest in this paper is the mathematical and numerical analysis of a new non-variational model based on a high order non-linear PDE system resulting from image denoising. This model is motivated by involving the decomposition approach of $ H^{-1} $ norm suggested by Guo et al. <sup>[<xref ref-type="bibr" rid="b1">1</xref>,<xref ref-type="bibr" rid="b2">2</xref>]</sup> which is more appropriate to represent the small details in the textured image. Our model is based on a diffusion tensor that improves the behavior of the Perona-Malik diffusion directions in homogeneous regions and the Weickert model near tiny edges with a high diffusion order. A rigorous analysis of the existence and uniqueness of the weak solution of the proposed reaction-diffusion model is cheked in a suitable functional framework, using the Schauder fixed point theorem. Finally, we carry out a numerical result to show the effectiveness of our model by comparing the results obtained with some competitive models.</p></abstract>

Well-posedness and simulation results of a coupled denoising PDE

In this paper, we are interested in the mathematical and numerical study of a new non variational model based on a high order non-linear PDE system for image denoising. The proposed model generalizes and improves the nonvariational model of Halim and Kumar (2020). Our model is based on a diffusion tensor that improves the behavior of the Perona–Malik diffusion directions in homogeneous regions and the Weickert model near tiny edges with a high diffusion order. The proposed approach takes into consideration the best of regularized Perona–Malik process in flat regions, and the benefit of the Weickert filter effect near sharp edges. To overcome the difficulty of the 4th order in the proposed equation, we involve the decomposition approach of H−1 norm suggested by Guo et al. (2011) [20, 21] to transform it into coupled second order equations which is more appropriate to represent the small details in the textured image. By the Schauder method, we establish the existence and uniqueness of the weak solution of the proposed reaction–diffusion System. In the end, some experiment results illustrate the effectiveness of the proposed model compared with the results of some existing models.

Shiftry: RNN inference in 2KB of RAM

Traditionally, IoT devices send collected sensor data to an intelligent cloud where machine learning (ML) inference happens. However, this course is rapidly changing and there is a recent trend to run ML on the edge IoT devices themselves. An intelligent edge is attractive because it saves network round trip (efficiency) and keeps user data at the source (privacy). However, the IoT devices are much more resource constrained than the cloud, which makes running ML on them challenging. Specifically, consider Arduino Uno, a commonly used board, that has 2KB of RAM and 32KB of read-only Flash memory. Although recent breakthroughs in ML have created novel recurrent neural network (RNN) models that provide good accuracy with KB-sized models, deploying them on tiny devices with such hard memory requirements has remained elusive. We provide, Shiftry, an automatic compiler from high-level floating-point ML models to fixed-point C-programs with 8-bit and 16-bit integers, which have significantly lower memory requirements. For this conversion, Shiftry uses a data-driven float-to-fixed procedure and a RAM management mechanism. These techniques enable us to provide first empirical evaluation of RNNs running on tiny edge devices. On simpler ML models that prior work could handle, Shiftry-generated code has lower latency and higher accuracy.

AffectiveNet: Affective-Motion Feature Learning for Microexpression Recognition

Microexpressions are hard to spot due to fleeting and involuntary moments of facial muscles. Interpretation of microemotions from video clips is a challenging task. In this article, we propose an affective-motion imaging that cumulates rapid and short-lived variational information of microexpressions into a single response. Moreover, we have proposed an AffectiveNet: Affective-motion feature learning network that can perceive subtle changes and learns the most discriminative dynamic features to describe the emotion classes. The AffectiveNet holds two blocks: MICRoFeat and MFL block. MICRoFeat block conserves the scale-invariant features, which allows network to capture both coarse and tiny edge variations. Whereas, the MFL block learns microlevel dynamic variations from two different intermediate convolutional layers. Effectiveness of the proposed network is tested over four datasets by using two experimental setups: person independent and cross dataset validation. The experimental results of the proposed network outperform the state-of-the-art approaches with significant margin for MER approaches.

Reduction of Salt and Pepper Noises from a Degraded Image Based on Fuzzy Techniques

Objectives: The present paper is concerned for removing the salt and pepper noises along with preserving the edge details of images from the degraded images. There are different techniques are available in literature for it with some merits and demerits. Methods: To overcome the demerits of some existing methods, we have presented a new technique based on fuzzy logic. The suggested technique is able to describe in two phases. In first phase, centroid defuzzification method is critically used for reducing the Salt and pepper noises and in second phase some techniques are used to preserve the different tiny edges of the degraded images. Findings: The proposed method tested and compared with some existing methods for 8-bit images of different percentage of noises. It gives the satisfactory result in terms of PSNR values and also preserved the different tiny edges. Here edges are computed by means of canny edge detector. Application: The suggested method can be applied to find the features of the images in medical imaging and in different industry.

Anomalous Light Emission and Wide Photoluminescence Spectra in Graphene Quantum Dot: Quantum Confinement from Edge Microstructure.

The physical origin of the observed anomalous photoluminescence (PL) behavior, that is, the large-size graphene quantum dots (GQDs) exhibiting higher PL energy than the small ones and the broadening PL spectra from deep ultraviolet to near-infrared, has been debated for many years. Obviously, it is in conflict with the well-accepted quantum confinement. Here we shed new light on these two notable debates by state-of-the-art first-principles calculations based on many-body perturbation theory. We find that quantum confinement is significant in GQDs with remarkable size-dependent exciton absorption/emission. The edge environment from alkaline to acidic conditions causes a blue shift of the PL peak. Furthermore, carbon vacancies are inclined to assemble at the GQD edge and form the tiny edge microstructures. The bound excitons, localized inside these edge microstructures, determine the anomalous PL behavior (blue and UV emission) of large-size GQDs. The bound excitons confined in the whole GQD lead to the low-energy transition.

Europe? A Speech

My talk has two main points. One is a longstanding desire for continentality by the extreme northwestern edge of our continent, the island of Eurasia. It is a tiny edge and Imperial Europe is even a tiny edge of that one. The other point is a desire for a self, a mastership of interpellation. To be the master of “Hey you,” which is what interpellation is. The one hailed thinks s/he is the right person. There is thus a complicity in interpellation, culturalism allows us to forget that.

Edge plasma dynamics during L-H transition in the JFT-2M tokamak**This article is dedicated to the memory of Professor Tihiro Ohkawa.

This article presents a radial electric field measurement by a heavy ion beam probe in the JFT-2M tokamak, during the L-H transition. An abrupt increase (time scale of O(100 µs)) of the strong edge radial electric field (localized in the radius with FWHM ∼7 mm) results in the increase of density gradient and turbulence reduction. Rapid inward propagation of the turbulence suppression front is observed at the transition. After the transition, the electric field structure in the tiny edge localized modes (ELMs) is analyzed. Transport self-regulation events observed in the vicinity of the L-H transition, i.e. the limit cycle oscillation (LCO) in the L-mode, the tiny ELM in the H-mode, as well as the L-H transition itself, are summarized in a single Lissajous diagram in the electric field-density gradient space, which provides a comprehensive explanation of the transition dynamics.

An Unique Edge Preserving Noise Filtering Technique for Impulse Noise Removal

Image de-noising is the technique to reduce noises from corrupted images. The aim of the image denoising is to improve the contrast of the image or perception of information in images for human viewers or to provide better output for other automated image processing techniques. This paper presents a new approach for color image de-noising with Fuzzy Filtering techniques using centroid method for defuzzification. It preserves any type of edges (including tiny edges) in any direction. The experimental result shows the effectiveness of the proposed method.