Qualcomm Snapdragon Research Articles

AI-Enhanced Edge Device for Real-Time Snoring Detection

This paper presents the development of a cutting-edge, non-invasive edge device designed to monitor snoring and provide timely, moderate haptic feedback to users. Utilizing the Qualcomm Snapdragon 8cx Gen 3 processor, the device offers robust computing power and AI capabilities for real-time processing, making it a versatile tool for health monitoring applications. The system integrates a high-fidelity MEMS microphone array capable of capturing nuanced audio signals and a TDK piezoelectric haptic actuator, which delivers precise alerts through customized vibrations. The research explores the potential of this advanced hardware in detecting and managing obstructive sleep apnea (OSA), a condition often underdiagnosed due to a lack of patient awareness. By leveraging state-of-the-art digital signal processing and deep learning techniques, the device aims to enhance user awareness and intervention in sleep-related disorders, offering a promising new avenue for improving patient outcomes and quality of life.

Transformative Dynamics: Industry 4.0 and IIoT's Comprehensive Impact on the Semiconductor Industry and Qualcomm Snapdragon's Evolution

Transformative Dynamics: Industry 4.0 and IIoT's Comprehensive Impact on the Semiconductor Industry and Qualcomm Snapdragon's Evolution

Benchmarking Space Mission Applications on the Snapdragon Processor Onboard the ISS

Future space missions will process and analyze imagery on board as well as plan and act more autonomously, placing greater demands on flight computing. Traditional flight hardware provides modest computing power, even when compared to common laptop and desktop computers. A new generation of commercial-off-the-shelf (COTS) processors designed for commercial electronics such as cell phones and tablets, such as the Qualcomm Snapdragon, deliver significant compute in a small size, weight, and power; and they offer hardware acceleration in the form of graphics processing units and digital signal processors. We benchmark a variety of instrument processing and mission planning software on a Qualcomm Snapdragon system on a chip currently hosted by Hewlett Packard Enterprise’s Spaceborne Computer-2 on board the International Space Station to highlight the potential of using embedded COTS processors on future space missions.

Speech Intelligibility: Lateral Lisp Detection for Real-Time Monitoring

Abstract Correcting lisps in speech can prove to be of great difficulty to many, as they may be unaware of whether they are lisping. To help those affected, we have developed a simple algorithm for the real-time identification of the sigmatismus lateralis in ”S” sounds within speech via analysis in the frequency domain. The algorithm identifies peaks within the lisp’s frequency band after calibration. A frequency band of 3000-4000 Hz has been identified to be generally accurate for lisp and 2500-3000 Hz for the correct pronunciation for a single male test subject. The algorithm splits given speech recordings into smaller segments and compares the number of lisps and non-lisps detected in these segments to categorize. From tests, it was concluded that a segment length of 0.5 s produces the best results. The algorithm does not detect every lisp section, however it does not raise false positives. Our implementation in Julia with multi-threaded per-file analysis is able to analyze 20 files of lengths between 5 s and 10 s within 0.21 s on a Qualcomm Snapdragon 860 smartphone chipset, meaning analysis is far faster than real-time. The proposed algorithm is a simple prototype algorithm capable of realtime analysis of audio in the frequency domain to identify whether lateral lisps are the dominant sibilant pronunciation in a given window. The method was only tested for a single test subject. However, a calibration algorithm capable of adjusting parameters to new individuals is proposed. The algorithm itself should be easily expandable to identify other speech impediments.

Benchmarking Deep Learning Models on Myriad and Snapdragon Processors for Space Applications

Future space missions can benefit from processing imagery on board to detect science events, create insights, and respond autonomously. One of the challenges to this mission concept is that traditional space flight computing has limited capabilities because it is derived from much older computing to ensure reliable performance in the extreme environments of space: particularly radiation. Modern commercial-off-the-shelf processors, such as the Movidius Myriad X and the Qualcomm Snapdragon, provide significant improvements in small size, weight, and power packaging; and they offer direct hardware acceleration for deep neural networks, although these processors are not radiation hardened. We deploy neural network models on these processors hosted by Hewlett Packard Enterprise’s Spaceborne Computer-2 on board the International Space Station (ISS). We find that the Myriad and Snapdragon digital signal processors (DSP)/artificial intelligence processors (AIP) provide speed improvement over the Snapdragon CPU in all cases except single-pixel networks (typically greater than 10 times for DSP/AIP). In addition, the discrepancy introduced through quantization and porting of our Jet Propulsion Laboratory models was usually quite low (less than 5%). Models were run multiple times, and memory checkers were deployed to test for radiation effects. To date, we have found no difference in output between ground and ISS runs, and no memory checker errors.

Multitask Scheduling on Distributed Cloudlet System Built Using SoCs

With the emergence of IoT, new computing paradigms have also emerged. Initial IoT systems had all the computing happening on the cloud. With the emergence of Industry 4.0 and IoT being the major building block, clouds are not the only solution for data storage and analytics. Cloudlet, Fog Computing, Edge Computing, and Dew Computing models are now available, providing similar capabilities as the cloud. The term cloudlet was introduced first in 2011, but research in this area has picked up only over the past five years. Unlike clouds, which are built with powerful server-class machines and GPUs, cloudlets are usually made using simpler devices such as SoCs. In this paper, we propose a complete novel distributed architecture for cloudlets, and we are also proposing algorithms for data storage and task allocation across various nodes in the cloudlet. This cloudlet system was built using Qualcomm Snapdragon 410c. We have analyzed the architecture and the algorithm for varying workloads, bandwidth and data storage. The primary aim of the algorithm and the architecture is to ensure uniform processing and data loads across the nodes of the system.

Automatic Sequential Stitching of High-Resolution Panorama for Android Devices Using Precapture Feature Detection and the Orientation Sensor

Image processing on smartphones, which are resource-limited devices, is challenging. Panorama generation on modern mobile phones is a requirement of most mobile phone users. This paper presents an automatic sequential image stitching algorithm with high-resolution panorama generation and addresses the issue of stitching failure on smartphone devices. A robust method is used to automatically control the events involved in panorama generation from image capture to image stitching on Android operating systems. The image frames are taken in a firm spatial interval using the orientation sensor included in smartphone devices. The features-based stitching algorithm is used for panorama generation, with a novel modification to address the issue of stitching failure (inability to find local features causes this issue) when performing sequential stitching over mobile devices. We also address the issue of distortion in sequential stitching. Ultimately, in this study, we built an Android application that can construct a high-resolution panorama sequentially with automatic frame capture based on an orientation sensor and device rotation. We present a novel research methodology (called "Sense-Panorama") for panorama construction along with a development guide for smartphone developers. Based on our experiments, performed by Samsung Galaxy SM-N960N, which carries system on chip (SoC) as Qualcomm Snapdragon 845 and a CPU of 4 × 2.8 GHz Kyro 385, our method can generate a high-resolution panorama. Compared to the existing methods, the results show improvement in visual quality for both subjective and objective evaluation.

Real-time fruit detection using deep neural networks on CPU (RTFD): An edge AI application

Intelligent harvesting can greatly reduce the manual labor for fruit harvesting and is one of the main application areas of intelligent agricultural machinery. Smart edge sensors must detect the growth status of tomatoes and strawberries in real-time for intelligent harvesting to be possible. Due to edge devices' relatively low image processing power, it remains challenging to implement deep learning algorithms for real-time detection in field applications. Therefore, scholars are strongly interested in utilizing computationally small, accurate, and quick models for edge harvesting machinery. In this research, we propose RTFD, a lightweight algorithm for edge CPU devices that detects fruit. Based on the PicoDet-S model, RTFD optimizes the efficiency of real-time detection for edge CPU computing devices by enhancing the model's structure, loss function and activation function. The experimental results demonstrate that the mAP of RTFD for tomato and strawberry datasets at various growth phases is 0.78 and 0.96, respectively, with a computational effort of 1.44 GFLOPs. RTFD improves mAP by 1.9% and 2.3% on the tomato and strawberry datasets, respectively, with minimal loss in computation and model parameters, making it more suitable for edge device deployment. Meanwhile, we proposed an approach for 8-bit quantization-aware training of the Conv2D and Linear layers of the model for improved edge deployment. This approach quantizes 32-bit floating-point values in the Conv2D and Linear layers into 8-bit integers, thereby reducing the model's size and accelerating detection speed. The testing findings indicate that the quantized RTFD model has a size of 1.33 MB and a CPU detection speed of around 11 FPS. Using the quantified trained model, we developed the Android application RTFD-CPU to evaluate the model's performance in test scenarios. RTFD-CPU was assessed on the smartphone Redmi K30pro (Qualcomm Snapdragon 865 CPU and 8 GB of RAM) with a detection speed of 19 FPS utilizing only the CPU. In summary, the proposed RTFD has great potential for intelligent picking machines, and the concept of redesigning the model structure, loss function, and activation function, as well as training by quantization to speed up the detection of deep neural networks, is anticipated to be successfully implemented in edge computing.

ReMoNet: Recurrent Multi-Output Network for Efficient Video Denoising

While deep neural network-based video denoising methods have achieved promising results, it is still hard to deploy them on mobile devices due to their high computational cost and memory demands. This paper aims to develop a lightweight deep video denoising method that is friendly to resource-constrained mobile devices. Inspired by the facts that 1) consecutive video frames usually contain redundant temporal coherency, and 2) neural networks are usually over-parameterized, we propose a multi-input multi-output (MIMO) paradigm to process consecutive video frames within one-forward-pass. The basic idea is concretized to a novel architecture termed Recurrent Multi-output Network (ReMoNet), which consists of recurrent temporal fusion and temporal aggregation blocks and is further reinforced by similarity-based mutual distillation. We conduct extensive experiments on NVIDIA GPU and Qualcomm Snapdragon 888 mobile platform with Gaussian noise and simulated Image-Signal-Processor (ISP) noise. The experimental results show that ReMoNet is both effective and efficient on video denoising. Moreover, we show that ReMoNet is more robust under higher noise level scenarios.

Instantaneous, Dual-Frequency, Multi-GNSS Precise RTK Positioning Using Google Pixel 4 and Samsung Galaxy S20 Smartphones for Zero and Short Baselines.

The recent development of the smartphone Global Navigation Satellite System (GNSS) chipsets, such as Broadcom BCM47755 and Qualcomm Snapdragon 855 embedded, makes instantaneous and cm level real-time kinematic (RTK) positioning possible with Android-based smartphones. In this contribution we investigate the instantaneous single-baseline RTK performance of Samsung Galaxy S20 and Google Pixel 4 (GP4) smartphones with such chipsets, while making use of dual-frequency L1 + L5 Global Positioning System (GPS), E1 + E5a Galileo, L1 + L5 Quasi-Zenith Satellite System (QZSS) and B1 BeiDou Navigation Satellite System (BDS) code and phase observations in Dunedin, New Zealand. The effects of locating the smartphones in an upright and lying down position were evaluated, and we show that the choice of smartphone configuration can affect the positioning performance even in a zero-baseline setup. In particular, we found non-zero mean and linear trends in the double-differenced carrier-phase residuals for one of the smartphone models when lying down, which become absent when in an upright position. This implies that the two assessed smartphones have different antenna gain pattern and antenna sensitivity to interferences. Finally, we demonstrate, for the first time, a near hundred percent (98.7% to 99.9%) instantaneous RTK integer least-squares success rate for one of the smartphone models and cm level positioning precision while using short-baseline experiments with internal and external antennas, respectively.

Smartphone-based real-time object recognition architecture for portable and constrained systems

Machine learning algorithms based on convolutional neural networks (CNNs) have recently been explored in a myriad of object detection applications. Nonetheless, many devices with limited computation resources and strict power consumption constraints are not suitable to run such algorithms designed for high-performance computers. Hence, a novel smartphone-based architecture intended for portable and constrained systems is designed and implemented to run CNN-based object recognition in real time and with high efficiency. The system is designed and optimised by leveraging the integration of the best of its kind from the state-of-the-art machine learning platforms including OpenCV, TensorFlow Lite, and Qualcomm Snapdragon informed by empirical testing and evaluation of each candidate framework in a comparable scenario with a high demanding neural network. The final system has been prototyped combining the strengths from these frameworks and led to a new machine learning-based object recognition execution environment embedded in a smartphone with advantageous performance compared with the previous frameworks.

Discrimination-Aware Network Pruning for Deep Model Compression.

We study network pruning which aims to remove redundant channels/kernels and hence speed up the inference of deep networks. Existing pruning methods either train from scratch with sparsity constraints or minimize the reconstruction error between the feature maps of the pre-trained models and the compressed ones. Both strategies suffer from some limitations: the former kind is computationally expensive and difficult to converge, while the latter kind optimizes the reconstruction error but ignores the discriminative power of channels. In this paper, we propose a simple-yet-effective method called discrimination-aware channel pruning (DCP) to choose the channels that actually contribute to the discriminative power. To this end, we first introduce additional discrimination-aware losses into the network to increase the discriminative power of the intermediate layers. Next, we select the most discriminative channels for each layer by considering the discrimination-aware loss and the reconstruction error, simultaneously. We then formulate channel pruning as a sparsity-inducing optimization problem with a convex objective and propose a greedy algorithm to solve the resultant problem. Note that a channel (3D tensor) often consists of a set of kernels (each with a 2D matrix). Besides the redundancy in channels, some kernels in a channel may also be redundant and fail to contribute to the discriminative power of the network, resulting in kernel level redundancy. To solve this issue, we propose a discrimination-aware kernel pruning (DKP) method to further compress deep networks by removing redundant kernels. To avoid manually determining the pruning rate for each layer, we propose two adaptive stopping conditions to automatically determine the number of selected channels/kernels. The proposed adaptive stopping conditions tend to yield more efficient models with better performance in practice. Extensive experiments on both image classification and face recognition demonstrate the effectiveness of our methods. For example, on ILSVRC-12, the resultant ResNet-50 model with 30 percent reduction of channels even outperforms the baseline model by 0.36 percent in terms of Top-1 accuracy. We also deploy the pruned models on a smartphone (equipped with a Qualcomm Snapdragon 845 processor). The pruned MobileNetV1 and MobileNetV2 achieve 1.93× and 1.42× inference acceleration on the mobile device, respectively, with negligible performance degradation. The source code and the pre-trained models are available at https://github.com/SCUT-AILab/DCP.

Foldable Market Expands with Samsung, Microsoft Offerings

Foldable Market Expands with Samsung, Microsoft Offerings

5G Sets Sail Around the Globe [Mobile Radio

The third annual Qualcomm Snapdragon Technology Summit was held on Maui, Hawaii, on 4-6 December 2018. This event featured several industry milestones related to 5G mobile communication systems and the Snapdragon mobile platforms. During the second day of the technology summit, Qualcomm Technologies unveiled the Snapdragon 855 chipset, the company?s newest generation in the Snapdragon 8 Series Mobile Platform. This is the world's first commercial mobile platform supporting, collectively, multigigabit 5G, industry-leading artificial intelligence (AI) and immersive extended reality, ushering in a new decade of revolutionary mobile devices. Using new chip architectures built on leading 7-nm process technology, the Snapdragon 855 will offer users long-lasting battery life and enhanced experiences in such areas as imaging, audio, gaming, and extended reality.

It?s Time to Go Big with 5G [Mobile Radio

In July 2018, Qualcomm Technologies unveiled the world's first fully integrated fifth-generation (5G), New Radio (NR), millimeter wave (mm-wave), and sub-6-GHz radio-frequency (RF) modules for smartphones and other mobile devices. The Qualcomm QTM052 mmwave antenna module family and the Qualcomm QPM56xx sub-6-GHz RF module family pair with the Qualcomm Snapdragon X50 5G modem to deliver modem-to-antenna capabilities across several spectrum bands in a very compact footprint suited for integration in mobile devices.

NTT Docomo implements fingerprint and iris biometrics

Japan's largest mobile carrier NTT Docomo is driving biometrics into the mobile sector. It is to adopt and implement the Qualcomm Snapdragon Sense ID 3D fingerprint biometrics platform. The platform, supported in the Qualcomm Snapdragon 810 processor, began to be used alongside third party biometric sensors for Docomo's mobile biometric authentication service on 27 May in Japan.

The Teardown: Samsung Galazy S5

Samsung's response in the S5 has been to deploy the 2.5GHz Qualcomm Snapdragon 801, a quad-core design that compares against a 2.3GHz clock on the preceding generation (in some markets, an alternative Exynos processor may be used). It runs against 2GB of RAM packaged on top.

New Parallel Approaches for Scalar Multiplication in Elliptic Curve over Fields of Small Characteristic

We present two new strategies for parallel implementation of scalar multiplication over elliptic curves. We first introduce a Montgomery-halving algorithm which is a variation of the original Montgomery-ladder for point multiplication. This Montgomery-halving can be run in parallel with the original Montgomery-ladder in order to concurrently compute part of the scalar multiplication. We also present two point thirding formulas in some subfamilies of curves $E(\mathbb {F}_{3^m})$ . We use these thirding formulas to implement scalar multiplication through (Third, Double)-and-add and (Third, Triple)-and-add parallel approaches. We also provide some implementation results of the presented parallel strategies which show a speed-up of 5-14 percent on an Intel Core i7 processor and a speed-up of 8-19 percent on a Qualcomm Snapdragon processor compared to non-parallelized approaches.

Qualcomm Snapdragon 800 processor first to use TSMC’s 28HPM advanced process technology

Qualcomm Snapdragon 800 processor first to use TSMC’s 28HPM advanced process technology