Spotting System Research Articles

Differential Analysis of Modern Text Spotting Methods : A Systematic Review

Text spotting, the task of detecting and recognizing text within images, is vital in applications like document analysis, autonomous navigation, and surveillance. The motivation for this review arises from the growing need for accurate automated text extraction methods, driven by the surge of visual data and the complexity of real-world environments. Despite advances in deep learning and computer vision, current text spotting techniques face significant challenges, including handling complex backgrounds, curved or distorted text, varied font styles, and low-resolution images. These limitations restrict their effectiveness in diverse, real-world settings. This systematic review aims to conduct a differential analysis of modern text spotting methods, highlighting their strengths, weaknesses, and performance in addressing such challenges. The objectives are to evaluate state-of-the-art techniques, identify gaps in the field, and propose future research directions. By critically synthesizing recent literature, this review provides insights that can help enhance the robustness and accuracy of text spotting systems, making them more adaptable to real-world conditions.

Artificial basilar membrane/hair cell integrated acoustic system for keyword spotting in noisy environments inspired by human cochlea

We report a novel speech recognition method using a noise-robust acoustic sensor system that integrates a spatially frequency-separating sensor with a nonlinear amplification algorithm, mimicking the cochlea’s basilar membrane and hair cells. The multichannel piezoelectric artificial basilar membrane (ABM) sensor detects specific sound frequencies with high sensitivity over 0.2–6 kHz. The signal processing model of the Artificial Hair Cell inspired by the signal transduction mechanism of inner hair cells, simultaneously enhances the frequency selectivity of ABM sensors and improves noise robustness. In a 0 dB SNR noisy environment, it effectively detected the voice signal with a maximum SNR of 57 dB. Furthermore, we converted the frequency-separated signals for speech sounds in various noisy environments into heatmap images and utilized them as input for a CNN-based speech recognition algorithm. Consequently, our system demonstrated noise-robust recognition performance with 94 % accuracy, even in noisy environments.

Advantages and Pitfalls of Dataset Condensation: An Approach to Keyword Spotting with Time-Frequency Representations

With the exponential growth of data, the need for efficient techniques to extract relevant information from datasets becomes increasingly imperative. Reducing the training data can be useful for applications wherein storage space or computational resources are limited. In this work, we explore the concept of data condensation (DC) in the context of keyword spotting systems (KWS). Using deep learning architectures and time-frequency speech representations, we have obtained condensed speech signal representations using gradient matching with Efficient Synthetic-Data Parameterization. From a series of classification experiments, we analyze the models and condensed data performances in terms of accuracy and number of data per class. We also present results using cross-model techniques, wherein models are trained with condensed data obtained from a different architecture. Our findings demonstrate the potential of data condensation in the context of the speech domain for reducing the size of datasets while retaining their most important information and maintaining high accuracy for the model trained with the condensed dataset. We have obtained an accuracy of 80.75% with 30 condensed speech representations per class with ConvNet, representing an addition of 24.9% in absolute terms when compared to 30 random samples from the original training dataset. However, we demonstrate the limitations of this approach in the cross-model tests. We also highlight the challenges and opportunities for further improving the accuracy of condensed data obtained and trained with different neural network architectures.

Robust Dual-Modal Speech Keyword Spotting for XR Headsets.

While speech interaction finds widespread utility within the Extended Reality (XR) domain, conventional vocal speech keyword spotting systems continue to grapple with formidable challenges, including suboptimal performance in noisy environments, impracticality in situations requiring silence, and susceptibility to inadvertent activations when others speak nearby. These challenges, however, can potentially be surmounted through the cost-effective fusion of voice and lip movement information. Consequently, we propose a novel vocal-echoic dual-modal keyword spotting system designed for XR headsets. We devise two different modal fusion approches and conduct experiments to test the system's performance across diverse scenarios. The results show that our dual-modal system not only consistently outperforms its single-modal counterparts, demonstrating higher precision in both typical and noisy environments, but also excels in accurately identifying silent utterances. Furthermore, we have successfully applied the system in real-time demonstrations, achieving promising results. The code is available at https://github.com/caizhuojiang/VE-KWS.

Design Of Emergency Keyword Recognition Using Arduino Nano BLE Sense 33 And Edge Impulse

This project focuses on Custom Keyword Voice Recognition (CKVR) for emergency response scenarios. A multilingual keyword spotting system is developed using the Arduino Nano 33 BLE Sense board and Edge Impulse. The system accurately recognizes the keyword "help" in English, Arabic, Kurdish, and Malay languages. The project utilizes Mel Frequency Cepstral Coefficients (MFCC) for feature extraction and employs deep learning techniques for model training. By optimizing the model through quantization and achieving 100% accuracy in training and testing phases, the system provides a reliable solution for identifying emergency keywords. The developed system has the potential to enhance safety in public spaces such as malls, hospitals, schools, and stations by quickly responding to individuals in distress. The project demonstrates the effectiveness of the chosen approach, highlighting the significance of MFCC processing, classification learning, and optimized model design in speech recognition. The successful development of this keyword spotting system opens doors for further advancements in the field and emphasizes the potential for innovative solutions that contribute to a safer and more responsive world.

A Low-Power Keyword Spotting System With High-Order Passive Switched-Capacitor Bandpass Filters for Analog-MFCC Feature Extraction

A Low-Power Keyword Spotting System With High-Order Passive Switched-Capacitor Bandpass Filters for Analog-MFCC Feature Extraction

A Resource-Efficient Keyword Spotting System Based on a One-Dimensional Binary Convolutional Neural Network

This paper proposes a resource-efficient keyword spotting (KWS) system based on a convolutional neural network (CNN). The end-to-end KWS process is performed based solely on 1D-CNN inference, where features are first extracted from a few convolutional blocks, and then the keywords are classified using a few fully connected blocks. The 1D-CNN model is binarized to reduce resource usage, and its inference is executed by employing a dedicated engine. This engine is designed to skip redundant operations, enabling high inference speed despite its low complexity. The proposed system is implemented using 6895 ALUTs in an Intel Cyclone V FPGA by integrating the essential components for performing the KWS process. In the system, the latency required to process a frame is 22 ms, and the spotting accuracy is 91.80% in an environment where the signal-to-noise ratio is 10 dB for Google speech commands dataset version 2.

FPGA Implementation of Keyword Spotting System Using Depthwise Separable Binarized and Ternarized Neural Networks.

Keyword spotting (KWS) systems are used for human-machine communications in various applications. In many cases, KWS involves a combination of wake-up-word (WUW) recognition for device activation and voice command classification tasks. These tasks present a challenge for embedded systems due to the complexity of deep learning algorithms and the need for optimized networks for each application. In this paper, we propose a depthwise separable binarized/ternarized neural network (DS-BTNN) hardware accelerator capable of performing both WUW recognition and command classification on a single device. The design achieves significant area efficiency by redundantly utilizing bitwise operators in the computation of the binarized neural network (BNN) and ternary neural network (TNN). In a complementary metal-oxide semiconductor (CMOS) 40 nm process environment, the DS-BTNN accelerator demonstrated significant efficiency. Compared with a design approach where BNN and TNN were independently developed and subsequently integrated as two separate modules into the system, our method achieved a 49.3% area reduction while yielding an area of 0.558 mm2. The designed KWS system, which was implemented on a Xilinx UltraScale+ ZCU104 field-programmable gate array (FPGA) board, receives real-time data from the microphone, preprocesses them into a mel spectrogram, and uses this as input to the classifier. Depending on the order, the network operates as a BNN or a TNN for WUW recognition and command classification, respectively. Operating at 170 MHz, our system achieved 97.1% accuracy in BNN-based WUW recognition and 90.5% in TNN-based command classification.

Plasmon Tunability and Field Enhancement of Gold Nanostar

Objective: The localized surface plasmon resonance (LSPR) and field enhancement of Gold nanosphere and nanostar were evaluated. Method: FDTD solutions, a product of Lumerical solutions Inc., Vancouver, Canada [17], was used to perform the electromagnetic simulations in this work. The impact of particle size and spike number on peak wavelength was studied quantitatively. Result: By altering the particle size and amount of spikes, we were able to detect a hot zone around nanostar. For Au nanostar, the peak wavelength for nanostar varies from visible to near-infrared. When compared to a nanosphere of the same dimension, the shift seen in nanostar is substantially higher, making it more suitable for biosensing applications. When the refractive index of the surrounding medium is increased, a red shift in peak wavelength is noticed, forming the basis for a plasmonic refractive index sensor. Aside from having a higher sensitivity, nanostar has a twofold hot spot system due to their unique surfaces. There is no evidence of spike aggregation in the near field pattern. As a result, it is thought to be a better nanostructure for biosensing applications. Conclusion: The LSPR and field enhancement for Au nanosphere and Nanostar were investigated using the FDTD method. The nanosphere's peak wavelength is in visible region, whereas the nanostar's range extends from visible to near-infrared, depending on the size and number of spikes. At 517 nm, the enhancement factor for a nanosphere was 102, but at 1282 nm, the enhancement factor for a nanostar with six spikes was 108.

Broadcast Attention Learning for Real Telephone Speech Keyword Spotting

With the development of mobile smart devices, keyword spotting plays an important role in the interaction between machines and users. However, low storage and low energy consumption of mobile devices limit the accuracies of keyword spotting tasks. Therefore, how to achieve a balance between the high accuracy and low consumption is a research hotspot for a keyword spotting system. Convolutional neural networks have been widely adopted in recent keyword spotting systems due to their superior accuracies, and the success of the transformer architecture in many areas demonstrates the effectiveness of self-attention. In this paper, we combine self-attention and convolutional neural networks, and propose the broadcast attention learning network (BA-net), using a small number of parameters while achieving the accuracies of 97.18% and 78.44% respectively on the Google speech command dataset and a real telephone speech dataset.

KEYWORD SPOTTING SYSTEM WITH NANO 33 BLE SENSE USING EMBEDDED MACHINE LEARNING APPROACH

Due to the obvious advancement of artificial intelligence, keyword spotting has become a fast-growing technology that was first launched a few years ago by hidden Markov models. Keyword spotting is the technique of finding terms that have been pre-programmed into a machine learning model. However, because the keyword spotting system model will be installed on a small and resource-constrained device, it must be minimal in size. It is difficult to maintain accuracy and performance when minimizing the model size. We suggested in this paper to develop a TinyML model that responds to voice commands by detecting words that are utilized in a cascade architecture to start or control a program. The keyword detection machine learning model was built, trained, and tested using the edge impulse development platform. The technique follows the model-building workflow, which includes data collection, preprocessing, training, testing, and deployment. 'On,' 'Off,' noise, and unknown databases were obtained from the Google speech command database V1 and applied for training and testing. The MFCC was used to extract features and CNN was used to generate the model, which was then optimized and deployed on the microcontroller. The model's evaluation represents an accuracy of 84.51% based on the datasets. Finally, the KWS was successfully implemented and assessed on Arduino Nano 33 BLE Sense for two studies in terms of accuracy at three different times and by six different persons.

A safety check method to maximize the effective reserve by optimizing the power of the tie-line in the power market

To ensure the stability of the electricity spot market and the safety of the provincial and regional power systems, a safety check method is proposed to maximize the effective reserve resources in the power system by optimizing the power of each tie-line. This safety check method accurately models the tie-line equipment and the effective reserve resources and is coupled with each constraint of the electricity spot market clearing model to form a safety check algorithm to optimize the power of tie-line power. The model involved in this paper is a linear model, which has a clear implementation method in practical dispatching applications. Through this method, the power configuration scheme of each tie-line to meet the electricity spot market constraints can be obtained, and the safety check results have the executability of the power market. The rationality and feasibility of the safety check algorithm results are verified by simulating the provincial-scale electricity spot market. According to the simulation results, this method can release effective reserve resources and provide more guarantees for the safe operation of the power grid. In addition, this method can save up to 4.9% of the total operation cost of the power system and improve the dispatching economy of the power system. This method is of great significance to ensure the safe operation of the power system and the day-ahead market and real-time market scheduling in the actual power spot system. In addition, this method also has great guiding significance for the analysis of the actual reserve situation of the power market after the event.

Fearless Steps APOLLO: Challenges in keyword spotting and topic detection for naturalistic audio streams

Fearless Steps (FS) APOLLO is a + 50,000 hr audio resource established by CRSS-UTDallas capturing all communications between NASA-MCC personnel, backroom staff, and Astronauts across manned Apollo Missions. Such a massive audio resource without metadata/unlabeled corpus provides limited benefit for communities outside Speech-and-Language Technology (SLT). Supplementing this audio with rich metadata developed using robust automated mechanisms to transcribe and highlight naturalistic communications can facilitate open research opportunities for SLT, speech sciences, education, and historical archival communities. In this study, we focus on customizing keyword spotting (KWS) and topic detection systems as an initial step towards conversational understanding. Extensive research in automatic speech recognition (ASR), speech activity, and speaker diarization using manually transcribed 125 h FS Challenge corpus has demonstrated the need for robust domain-specific model development. A major challenge in training KWS systems and topic detection models is the availability of word-level annotations. Forced alignment schemes evaluated using state-of-the-art ASR show significant degradation in segmentation performance. This study explores challenges in extracting accurate keyword segments using existing sentence-level transcriptions and proposes domain-specific KWS-based solutions to detect conversational topics in audio streams. [Work Sponsored by NSF via Grant No. 2016725 and EU’s Horizon 2021 R&I Program under MSC Grant No. 101062614.]

A comprehensive review and clinical guide to molecular and serological diagnostic tests and future development: In vitro diagnostic testing for COVID-19

Abstract COVID-19 is a contagious syndrome caused by SARS Coronavirus 2 (SARS-CoV-2) that requires rapid diagnostic testing to identify and manage in the affected persons, characterize epidemiology, and promptly make public health decisions and manage the virus present in the affected person and promptly make public health decisions by characterizing the epidemiology. Technical problems, especially contamination occurring during manual real-time polymerase chain reaction (RT-PCR), can result in false-positive NAAT results. In some cases, RNA detection technology and antigen testing are alternatives to RT-PCR. Sequencing is vital for tracking the SARS-CoV-2 genome’s evolution, while antibody testing is beneficial for epidemiology. SARS-CoV-2 testing can be made safer, faster, and easier without losing accuracy. Continued technological advancements, including smartphone integration, will help in the current epidemic and prepare for the next. Nanotechnology-enabled progress in the health sector has aided disease and pandemic management at an early stage. These nanotechnology-based analytical tools can be used to quickly diagnose COVID-19. The SPOT system is used to diagnose the coronavirus quickly, sensibly, accurately, and with portability. The SPOT assay consists of RT-LAMP, followed by pfAgo-based target sequence detection. In addition, SPOT system was used to detect both positive and negative SARS-CoV-2 samples. This combination of speed, precision, sensitivity, and mobility will allow for cost-effective and high-volume COVID-19 testing.

Enhancement of formant regions in magnitude spectra to develop children’s KWS system in zero resource scenario

The mismatches due to pitch, speaking rate, formant dispersion and ambient noise deteriorate the performance of an automatic keyword spotting (KWS) system. The work presented in this paper aims at reducing the aforementioned mismatches through front-end signal processing. In the proposed approach, the short-term magnitude spectra (ST-MS) are firstly computed with a smaller frameshift and then averaged over the adjacent frames to enhance the formant regions. The formant regions in the ST-MS have a higher magnitude than the nearing frequency regions. Consequently, temporal averaging of ST-MS over adjacent frames suppresses the high-frequency variation due to pitch, and formant dispersion. Furthermore, the formant peaks can be more accurately detected from the temporal averaged ST-MS when compared to detection from the original ST-MS. The Mel frequency cepstral coefficients (MFCC) computed from the temporally averaged magnitude spectra (TAS-MFCC) are pitch robust compared to the MFCC, and MFCC extracted from the reported spectral smoothing approaches employing variational mode decomposition (VMD-MFCC), pitch adaptive cepstral truncation (PACT-MFCC) and single-pole filter (SPS-MFCC). Performance of TA-MFCC feature in mismatched test condition is further improved by appending five logarithmically compressed resonant peaks at least separated by 400 Hz, here this feature is termed as TAS-MFCC-ARP. The spectral peaks mostly represent the formants in ST-MS. The performances of the deep neural network-hidden Markov model-based children’s KWS system reported in this work show that the TAS-MFCC-ARP provides a relative performance improvement of 103.83% compared to MFCC. The performance of the KWS system is further improved by data-augmented training through duration modification.

Construction of Natural Environment Information Management System of Scenic Spots Based on GIS

Construction of Natural Environment Information Management System of Scenic Spots Based on GIS

Validation of a forecasting system for Cercospora leaf spot of beet

RESUMO Com o objetivo de validar um sistema de previsão, com diferentes níveis de valores de severidade estimada comparada à pulverização convencional no controle da cercosporiose da beterraba, foram conduzidos experimentos em Rio do Sul/SC durante os ciclos de cultivo de 2018, 2019 e 2020. Os programas de pulverização foram estabelecidos de acordo com valores de severidade estimada (SE) do sistema de Marcuzzo et al., atribuindo-se valores acumulados de 0,15; 0,25, e 0,35 SE e no sistema convencional com pulverizações a cada 5 e 7 dias. Não houve diferença significativa entre os tratamentos quanto à produtividade em todos os ciclos. A AACPD, a severidade final e a taxa de progresso da doença não diferiram entre os tratamentos, mas no sistema de previsão com SE=0,35 o número de pulverizações foi de 28; 57 e 67% menor em relação ao sistema de aplicação semanal nos três anos de avaliação. O uso do sistema de previsão mostrou ser uma ferramenta viável no manejo do da cercosporiose da beterraba em Santa Catarina e encontra-se disponível gratuitamente para uso da cadeia produtiva na plataforma Agroconnect da EPAGRI-CIRAM disponível em http://www.ciram.sc.gov.br/agroconnect/.

Generation of Black-box Audio Adversarial Examples Based on Gradient Approximation and Autoencoders

Deep Neural Network (DNN) is gaining popularity thanks to its ability to attain high accuracy and performance in various security-crucial scenarios. However, recent research shows that DNN-based Automatic Speech Recognition (ASR) systems are vulnerable to adversarial attacks. Specifically, these attacks mainly focus on formulating a process of adversarial example generation as iterative, optimization-based attacks. Although these attacks make significant progress, they still take large generation time to produce adversarial examples, which makes them difficult to be launched in real-world scenarios. In this article, we propose a real-time attack framework that utilizes the neural network trained by the gradient approximation method to generate adversarial examples on Keyword Spotting (KWS) systems. The experimental results show that these generated adversarial examples can easily fool a black-box KWS system to output incorrect results with only one inference. In comparison to previous works, our attack can achieve a higher success rate with less than 0.004 s. We also extend our work by presenting a novel ensemble audio adversarial attack and testing the attack on KWS systems equipped with existing defense mechanisms. The efficacy of the proposed attack is well supported by promising experimental results.

Quality Driven Systematic Approximation for Binary-Weight Neural Network Deployment

Neural networks (NNs) with large scales of artificial neurons are increasingly used in recognition and classification tasks. In power-constrained scenarios, the tradeoff between performance and hardware consumptions must be carefully evaluated before silicon tape-out. In this paper, we proposed a systematic approach to design ultra-low power NN system. This work is motivated by the facts that NNs are resilient to approximation in many of the computations and NNs are outputting statistical tensors which are acceptable to less-than-perfect results. We resort to the front-back end approach with a twofold aim: (1) a fast and accurate design approach is proposed by estimating the computing quality of low-power approximate adder arrays, and it is adopted to evaluate the neural network system; (2) a quality configurable engine with different approximation degrees while processing NNs is implemented. The proposed work is demonstrated with a comprehensive keyword spotting (KWS) system as an ultra-low power NN engine. The experimental environment is setup with ten keywords from the google speech command dataset (GSCD) using an industrial 22-nm ultra-low-leakage (ULL) process. Comparing to the state-of-the-art KWS processors, the proposed approximate NN engine can demonstrate over 60% improvement in power efficiency and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.1\times $ </tex-math></inline-formula> area efficiency while achieving similar recognition accuracy.

Vision-Based Continuous Sign Language Spotting Using Gaussian Hidden Markov Model

The presence of movement epenthesis ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">me</i> ) between two consecutive signs in a continuous sign language makes sign language recognition (SLR) a challenging task. In this letter, we propose a vision-based continuous SL spotting system, which separates the meaningful signs from the sign sequences by removing the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">me</i> components from H.264/AVC compressed videos. The work is based on a two-state hidden Markov model (HMM) with Gaussian emission probability. The HMM is trained by the feature set extracted from the entire sign sequence video and finally, the hidden state-sequence is decoded using the Viterbi algorithm. From the decoded state-sequence, the sign spotting is done. The feature set comprises features extracted from the compressed domain as well as uncompressed domain analysis of the sign video. The video database is composed of American SL videos collected from Boston University database. From the experimental results, it is seen that the proposed system can spot the sign and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">me</i> frames with a spotting rate of about 83%.