Audio Output Research Articles

Gesture Vocalizer for Assisting Deaf and Dumb

Some individuals possess the innate ability to engage in communication with one another. However, regrettably, not everyone possesses the capacity for speech and hearing. Nonverbal communication, employed within the community of individuals with diverse abilities, serves as a method of interaction. To convey meaning, sign language combines simultaneous combinations of hand forms and movements with body, arm, and hand gestures as well as facial expressions. Those who are unable to speak use sign language to communicate with other people who are also speech-impaired as well as with average people who are familiar with the subtleties of sign language. Sometimes, an interpreter is necessary to translate the intentions behind the signs for those who can speak but may not fully understand sign language. However, consistent access to interpreters for sign language communication is not always feasible, and not everyone can master the intricacies of sign language. Hence, an alternative approach is the implementation of a device known as the Gesture Vocalizer. The Gesture Vocalizer functions as an intermediary. It translates the gestures produced by the person with speech impairment into audio output. The hardware iteration of this Gesture Vocalizer project is an array of microcontroller systems designed to enable inter-system communication between mute, deaf, and visually impaired communities and those who do not face such communication challenges. In this effort, we proposed a gesture vocalizer that is based on sensors and a microcontroller. The design of the gesture vocalizer primarily revolves around a hand glove integrated with a microcontroller system.

IoT - based Sustained Belt for Visually Impaired People

This paper is an IoT based Sustained Belt for Visually impaired people, which uses Raspberry Pi for image identification and image processing techniques to help a blind person to identify and rectify people and things in front of them and help them navigate everyday life and have a better lifestyle like any other common man. The key techniques that will be used are real-time images and object recognition as well as image processing. First would be an image, object recognition is to capture the image and then rectify the image, rectifying the image is done through image processing and then the rectified image is converted as audio output. The scope of this paper is to scale up a level where a blind person has a normal lifestyle, just like any other normal person, right without hustles or anything and as well as they can use this as his adjacent person, where this has inbuilt GPS and it is a mini computer as well it can be also useful for general questions, usually blind people when they can’t see things they have a lot of questions and be curious, it will be easier to know about them. What differentiates our paper from usual blind people’s normal stick is that the stick helps them not to crash anything but we are not focused on that, what we focused on is to have a very normal lifestyle like normal person

Noise Level Notifier

This paper involves utilizing an Arduino Uno as the primary hardware to measure the sound level in a library. The amount of noise in the region is measured using a sound sensor. The sound sensor signals are amplified using the operational amplifier function of the integrated circuit LM 567. There are two types of output available: audio and visual. The audio output takes the form of a personalized message that is played over speakers. LEDs are employed to offer visual feedback, with white LEDs used in noise-free environments (sound level 45 dB, yellow LEDs used when sound levels are above 65 decibels, and red LEDs used when sound levels are significantly above 80 decibels). A TIP 220 transistor is used to amplify the signals. A TIP 220 transistor amplifies the signals to create an output for the speaker. There is an audio message that corresponds to each sound level.

How to build a MATLAB demonstrator solving dynamical systems in real-time, with audio output and MIDI control

This paper explains and provides code to synthesize and control, in real-time, the audio signals produced by a dynamical system. The code uses only the Matlab programming language. It can be controlled with an external MIDI (Musical Instrument Data Interface) device, such as a MIDI keyboard or wind controller, or with mouse-operated sliders. In addition to the audio output, the demonstrator computes and displays the amplitude and fundamental frequency of the signal, which is useful to quantify the dynamics of the model. For the sake of this example, it is a type of Van der Pol oscillator, but more complex systems can be handled. The demonstrator holds potential for pedagogical and preliminary research applications, for various topics related to dynamical systems: direct and inverse bifurcations, transient effects such as dynamical bifurcations, artifacts introduced by integration schemes, and above all, the dynamics of self-sustained musical instruments.

Digital Film Music Creation Model Based on Inhomogeneous First Order Constant Coefficient Linear Differential Equation

Abstract Computer music production is an essential branch of computer technology in artistic creation. This is an interdisciplinary subject with information science and art. This paper proposes a creation model for digital music with first-order inhomogeneous first-order constant coefficient linear differential equations. This mode includes acoustic modes for waveform envelopes and bands. It has a straight-line mapping of phonetic patterns and rhythmic patterns. Then this paper uses the signal reconstruction principle and iterative extrapolation method to restore the signal in the frequency domain to obtain the simulated music. The experimental results show that this digital music creation model of inhomogeneous first-order constant coefficient linear differentiation can save a lot of labor costs. And this way can improve the audio quality and output effect. Compared with the undetermined factor method and the Laplace transform method in conventional advanced mathematics, the inhomogeneous first-order constant coefficient linear differentiation is more convenient in music composition.

Deep learning-based image captioning for visually impaired people

Vision loss can affect people of all ages. Severe or complete vision loss may occur when the eye or brain parts that need to process images are damaged. In this paper, in order to facilitate the blind, deep learning algorithms are used to caption the image for the blind person in which the blind can know about the object, distance and position of object. Whenever an image is captured via the camera, the scenes are recognized and predicted by the machine. After the prediction, it will be sent as an audio output to the user. Thus, with the help of this paper an artificial vision to the blind, can be achieved and help them to gain confidence while travelling alone.

Wave Digital Models of Piezoelectric Transducers for Audio Applications

The number of consumer electronics devices integrating piezoelectric (PE) transducers as flat-panel loudspeakers has recently experienced a steep increase. Being very thin, in fact, PE transducers well cope with the miniaturization process that characterizes the market. However, at the same time, their reduced dimensions cause the sound pressure level to be poor at low frequencies, impairing the overall acoustic response. In this article, we derive novel efficient discrete-time models of PE transducers in the wave digital (WD) domain such that they can be integrated in the future into signal processing algorithms for audio output enhancement. WD filters are, in fact, making headway among audio digital signal processing techniques based on circuit equivalent models thanks to their efficiency, robustness, and accuracy. Starting from circuital representations of the piezo constitutive equations, we derive both lumped and distributed models. In particular, we show how it is possible to implement the frequency-dependent elements characterizing Mason’s model in the WD domain, contrary to what can be done using mainstream SPICE-like simulators. Such WD implementations come in handy for the simulation and fast prototyping of PE systems, paving the way toward the design of model-based digital signal processing algorithms for enhancing the transducer acoustic performance.

TEXT2AV – Automated Text to Audio and Video Conversion

The paper aims to develop a machine learning-based system that can automatically convert text to audio and text to video as per the user’s request. Suppose Reading a large text is difficult for anyone, but this TTS model makes it easy by converting text into audio by producing the audio output by an avatar with lip sync to make it look more attractive and human-like interaction in many languages. The TTS model is built based on Waveform Recurrent Neural Networks (WaveRNN). It is a type of auto-regressive model that predicts future data based on the present. The system identifies the keywords in the input texts and uses diffusion models to generate high-quality video content. The system uses GAN (Generative Adversarial Network) to generate videos. Frame Interpolation is used to combine different frames into two adjacent frames to generate a slow- motion video. WebVid-20M, Image-Net, and Hugging-Face are the datasets used for Text video and LibriTTS corpus, and Lip Sync are the dataset used for text-to-audio. The System provides a user-friendly and automated platform to the user which takes text as input and produces either a high-quality audio or high-resolution video quickly and efficiently.

Embedded Digital Tuning Radio Based on stm32

With the development of science and technology, people’s demand for material culture is also increasing. The development of radio with good sound quality, easy portability, certain technological content, and moderate price can meet market demand well. The embedded digital tuning radio based on stm32 uses a 32-bit embedded chip with fast processing speed, convenient development, and high-cost performance. At the same time, it is combined with TDA audio decoding chip to achieve digital decoding, cooperates with the LM386 power amplifier module to achieve high-quality audio output, and displays the search frequency through the OLED LCD screen, with high display brightness and clear display effect. The whole work is very in line with the aesthetics of modern people and enhances the technological impression of products such as radios in people’s minds.

Judgment of togetherness in performances by musical duos.

Musicians experience varying degrees of togetherness with their co-performers when playing in ensembles. However, little is known about how togetherness is experienced by audiences and how interpersonal dynamics in body motion and sound support the judgment of togetherness. This research investigates audience sensitivity to audio and visual markers of interperformer coordination and expressivity in ensembles, in relation to modality of stimulus presentation and audience music background. A set of duo ensemble performances, comprising motion capture recordings of the musicians' upper bodies and instruments, were presented to participants with varying music background, including novices and semi-professional musicians. Participants were required to: (i) watch and listen, (ii) only watch, and (iii) only listen to the selected recordings, whilst providing dynamic ratings of how much togetherness between musicians they perceived. Results demonstrate that sound intensity and similarity in right arm motion (quantified using cross-wavelet transform analysis) were significant predictors of rated togetherness in novices, whilst sound synchronization and chest motion coordination predicted togetherness responses in semi-professional musicians. These results suggest the relevance of the quality of body motion coordination and of certain features of the audio outputs in the audience perception of togetherness. This research contributes to a better understanding of the perceptual mechanisms supporting socio-cognitive judgments of joint action activities.

Penguat Jembatan dengan Untai Pembalik Fase

The maximum output voltage of the audio amplifier is limited to the magnitude of the power supply voltage of the power transistor or the operational amplifier on the final amplifier. This limits the maximum power of the audio amplifier output. The way to enlarge the output power of the audio amplifier without increasing the voltage is the bridge method or bridged modes. With this method a bridge amplifier will be generated. This Bridge Amplifier is also known as Bridge-Tied Load (BTL) or Bridged Transformerless. The principle of Bridge Amplifiers is to use a pair of final amplifiers whose outputs have opposite phase each other. There are 3 ways to make a pair of power amplifiers have opposing phases: with internal modification, with an audio transformer (phase splitting audio input transformer) and with a simple active phase reversal splitter circuit). This paper will discuss Bridge Amplifiers with simple phase inverting circuits. A pair of audio power amplifiers using two TDA2050 chips which are operated at ± 19 Volt supply voltage. The phase inverting circuit using IC TL072. Sinusoidal signal with an amplitude of 200 mVp and a frequency of 1 KHz is used as an input signal. The results to be observed and measured are gain, input, output and bandwidth of the bridged amplifier compared to the usual amplifier

Co-designed mini-games for children with visual impairment: a pilot study on their usability

Digital games aimed at improving cognitive and/or motor-sensory skills need to be carefully designed to take into account the characteristics and needs of particular categories of users. Several novel mini-games explicitly aimed at children with visual impairment (VI) were co-designed by a multidisciplinary team which involved computer engineers and a therapy team from the Robert Hollman Foundation (Padova, Italy). These games are played by children moving within a large-scale interactive environment – i.e., a floor portion placed under a motion capture system capable of tracking one or more people – with the game linking the players movements to the audio and visual output to produce meaningful interactions. We report on a pilot study of the usability of the system involving 11 children with VI. The results allowed us to improve the system and to define a set of guidelines useful for designers and developers of similar systems.

Autonomous Assistance System for Visually Impaired using Tesseract OCR & gTTS

OCR makes machines to recognize text automatically. It not only saves retrieval times but also helps in digitalization of important documents. In this paper different techniques are used for converting textual matter from an paper document or an image into machine readable form (OCR text output) and then into audio output using gTTS (Google Text-to Speech, a Python library and CLI tool to interface with Google Translates text-to-speech API). In proposed model, the content writes spoken mp3 data to a file, a file-like object (byte string) for further audio manipulation with flexible pre-processing and tokenizing. The approach results into a graspable and comprehensible text output & then fed into audio output for inference with 99 % accuracy. This paper outlines the stages of development, the major challenges with some interesting result comparison with different state of approaches.

Indian Paper Currency Recognition with Audio Output System for Visually Impaired based on Image Processing using Transfer Learning

Abstract: We give an outline of camera-based computer vision technology in this paper. It can aid those who are not visually blessed in instantly identifying paper currency. For those who are blind or visually challenged, an effective paper money detection algorithm should have the following qualities: 1) Complete precision; and 2) adaptability in a range of circumstances in various environments and emergence. Many of the currency identification algorithms in use today are constrained to specific situations. We suggest a deep learning strategy focused on high accuracy in this project. This method works well for gathering more class-specific data and is better at handling partial closure and viewpoint changes. Evaluation of transfer learning also demonstrate its efficiency in coping with visual rotation, measures, and illumination changes.

A Survey on Machine Learning Based Techniques for Sign Language Translation System

Abstract: Sign language is the communication method used to bridge the gap among mute and deaf people and the rest of the people. But most people are not familiar with sign language, so communication becomes difficult among mute, deaf and the rest of the people. Improvement in machine learning technology has led to research on its application in sign language recognition and translation system. This paper presents a review of different sign language recognition and translation systems and the various techniques like classification algorithms that were implemented in them. We have found that although many sign language recognition and translation system exist none of them provides efficient and real-time conversation of sign language into audio and many sign language recognition and translation system are hand movement sensor-based which are not affordable to everyone. So, we have also discussed a sign language translation system integrated into a video calling application, in which we will be using a single shot multibox detector for hand detection, for feature extraction Inception V3 will be used, and support vector machines for classification, attention network for generating semantically correct sentences and a text to speech synthesizer for audio output. This system will be easy to access and affordable as it use

Automate extraction of braille text to speech from an image

Our project falls under the domain of Natural Language Processing and Machine Learning. With the advent of digitization, it became necessary to preserve data produced by humans. Voice and text are how we communicate. Visual impairment can be defined as partial or complete loss of vision. The primary goal of this project is to extract text from braille text images, so as to store it in a drive for later use. Although there are various ways to implement this, we'll use Deep Learning. Visually impaired people face numerous difficulties like interacting with the surroundings, navigation, etc. Different machine learning models like CNN, etc. have been used to solve this problem but usually have a high cost of detection in terms of computational time and hence they are not suitable for real-time applications [1]. To meet the project objective, we studied about Convolutional Neural Networks. The plan for picking this field is that it empowers machines to see the world as people do, see it likewise and even utilize the information for a huge number of undertakings like Image Recognition, Image Analysis and Classification, Media Recreation, Recommendation Systems, and so on [2]. This paper proposes a customized CNN model based on Braille Character dataset which has 26 different classes of all alphabets. This model uses a feature extractor that recognizes relevant features corresponding to each image, and then uses a classifier network to classify images into a set of classes. This model will take an image input, then pass those individual images to our classifier and then detect the braille language characters present in that image and give a translated English text and audio output. The final accuracy we obtained was 0.9615. These results show the model's robustness against factors like low light illumination.

A novel spatio-temporal convolutional neural framework for multimodal emotion recognition

Proposing a practical method for high-performance emotion recognition could facilitate human–computer interaction. Among existing methods, deep learning techniques have improved the performance of emotion recognition systems. In this work, a new multimodal neural design is presented wherein audio and visual data are combined as the input to a hybrid network comprised of a bidirectional long short term memory (BiLSTM) network and two convolutional neural networks (CNNs). The spatial and temporal features extracted from video frames are fused with Mel-Frequency Cepstral Coefficients (MFCCs) and energy features extracted from audio signals and BiLSTM network outputs. Finally, a Softmax classifier is used to classify inputs into the set of target categories. The proposed model is evaluated on Surrey Audio–Visual Expressed Emotion (SAVEE), Ryerson Audio–Visual Database of Emotional Speech and Song (RAVDESS), and Ryerson Multimedia research Lab (RML) databases. Experimental results on these datasets prove the effectiveness of the proposed model where it achieves the accuracy of 99.75%, 94.99%, and 99.23% for the SAVEE, RAVDESS, and RML databases, respectively. Our experimental study reveals that the suggested method is more effective than existing algorithms in adapting to emotion recognition in these datasets.

Developmental History of the Vocal Tract Organ

The Vocal Tract Organ has had a number of iterations resulting from advances in available technology as well as requirements of perceptual experiments and performance paradigms. The objective of this paper is to relate the development history of the Vocal Tract Organ from the original vision to what it is today as a modern version of the Vox Humana pipe organ stop for application in voice production and perception research. Descriptive METHODS/DESIGN: The latest Vocal Tract Organ is a polyphonic eight-channel eight-stop one manual Vocal Tract Organ that enables tab stop selected three-D printed vocal tracts to be used to create sound. This version includes eight stops (four for female vowel oral tracts and four for male vowel oral tracts). The stops are implemented using conventionally engraved pipe organ stop tabs labeled "Vox Humana Female" or "Vox Humana Male" followed by the three-D printed vowel: "EE", "AH", "ER" or "UU." This is described alongside the development stages from which it emerged and covers all previous versions of the Vocal Tract Organ. At the heart of the latest instrument is a Bela BeagleBone Black with a Bela cape audio expander board which incorporates eight 16-bit audio outputs at 44.1 kHz sampling rate (earlier versions based on the Arduino Mega board were limited to 8-bit audio at an audio sampling rate of 16.384 kHz which limited the overall output spectrum). The latest Vocal Tract Organ is programmed using the audio graphical programming language Pure Data which is directly compatible with the Bela system. The Pure Data patch creates eight larynx outputs at the pitches set by the keys depressed on the keyboard and these are routed to Vocal Tract Organ loudspeakers with three-D printed vocal tracts attached. The Bela system has enabled real-time synthesis of eight-note polyphonic sounds to eight separate three-D printed vocal tracts, each being selectable via an organ tab stop switch. The instrument has been cased in a purpose-designed and built prototype laser-cut enclosure that incorporates the eight tab stops, a MIDI keyboard input, a pipe organ style swell (volume) pedal connection, four stereo (eight channels) audio amplifiers and terminal connections for the eight loudspeakers. The Vocal Tract Organ functions as a musical instrument for performance and as an instrument for vowel and pitch perception research. Implementing it with the Bela family of processors allows for low audio latency of 1 ms and rapid prototyping due to being able to program directly with the high-level graphical audio programming language, Pure data (Pd).

Smart Assisting for Visually Impaired People

Project addresses the integration of a complete Text Read-out system designed for the visually challenged. The system consists of a web cam interfaced with Raspberry Pi which accepts a page of printed text. The OCR package installed in Raspberry Pi, scans it into a digital document which is then subjected to image processing like skew correction, segmentation to classify the text that is captured. After the classification, the text is readout by a text to speech conversion unit (TTS engine) installed in Raspberry Pi. The output is fed to an audio amplifier then it is read out. The simulation is just an initiation of the image to text conversion and text to speech conversion done by the OCR software installed in Raspberry Pi. The system finds interesting applications in libraries, auditoriums, offices where instructions and notices are to be read and also in the assisted paper documents. By using ultrasonic sensor, we will measure the distance between the blind people and obstacle then the distance will be played through audio output..

Cross-Modal Mutual Learning for Audio-Visual Speech Recognition and Manipulation

As a key characteristic in audio-visual speech recognition (AVSR), relating linguistic information observed across visual and audio data has been a challenge, benefiting not only audio/visual speech recognition (ASR/VSR) but also for manipulating data within/across modalities. In this paper, we present a feature disentanglement-based framework for jointly addressing the above tasks. By advancing cross-modal mutual learning strategies, our model is able to convert visual or audio-based linguistic features into modality-agnostic representations. Such derived linguistic representations not only allow one to perform ASR, VSR, and AVSR, but also to manipulate audio and visual data output based on the desirable subject identity and linguistic content information. We perform extensive experiments on different recognition and synthesis tasks to show that our model performs favorably against state-of-the-art approaches on each individual task, while ours is a unified solution that is able to jointly tackle the aforementioned audio-visual learning tasks.