Voice Data Research Articles

Early detection of mental health disorders using machine learning models using behavioral and voice data analysis

People of all demographics are impacted by mental illness, which has become a widespread and international health problem. Effective treatment and support for mental illnesses depend on early discovery and precise diagnosis. Notably, delayed diagnosis may lead to suicidal thoughts, destructive behaviour, and death. Manual diagnosis is time-consuming and laborious. With the advent of AI, this research aims to develop a novel mental health disorder detection network with the objective of maximum accuracy and early discovery. For this reason, this study presents a novel framework for the early detection of mental illness disorders using a multi-modal approach combining speech and behavioral data. This framework preprocesses and analyzes two distinct datasets to handle missing values, normalize data, and eliminate outliers. The proposed NeuroVibeNet combines Improved Random Forest (IRF) and Light Gradient-Boosting Machine (LightGBM) for behavioral data and Hybrid Support Vector Machine (SVM) and K-Nearest Neighbors (KNN) for voice data. Finally, a weighted voting mechanism is applied to consolidate predictions. The proposed model achieves robust performance and a competitive accuracy of 99.06% in distinguishing normal and pathological conditions. This framework validates the feasibility of multi-modal data integration for reliable and early mental illness detection.

ACCIDENT ANALYSIS USING IOT

Abstract—This project aims to develop an intelligent accident analysis system using IoT technology to improve road safety and emergency response. The system is designed to detect accidents in real-time, monitor key driving conditions, and automatically send alerts to emergency contacts along with the location of the incident. It also records voice data after an accident, serving as a basic black box for post-event analysis. In addition to accident detection, the system tracks driving behaviour and environmental factors, displaying relevant information on a screen for the driver and uploading data to an online platform for remote monitoring. By combining automation, communication, and data sharing, this project offers a smart and efficient solution for accident management and analysis. Keywords: Accident Detection, IoT Monitoring, Emergency Alert System, Real-time Tracking, Smart Vehicle Safety

Elderly Healthcare Support Services in Hospitals: Designing a Medical Terminal and Its Adaptive User Interface Based on the Geriatric Mental Model

In this study, we proposed an improved design strategy for enhancing the efficiency of, and satisfaction with, medical terminal use. Based on the geriatric mental model (GMM), we obtained the on-site mental information of elderly medical terminal users. Then, we coded the voice data obtained from our interviews and got the corresponding link node form. Subsequently, “Gephi” helped generate a clustered network of mental nodes. Finally, using the behavioral “affinity diagram”, we constructed a mental model (MM) of elderly patients using medical terminals for consultations, and accordingly, proposed a design strategy for an age-friendly medical terminal. The optimized medical terminal improves interface intuitiveness, feedback, and information architecture, reducing elderly users’ cognitive load. The added pop-up guide enhances usability and satisfaction. This study highlights how AUI and product design support age-friendly interaction and shows the value of MMs in improving elderly user experiences.

A hybrid approach for binary and multi-class classification of voice disorders using a pre-trained model and ensemble classifiers

Recent advances in artificial intelligence-based audio and speech processing have increasingly focused on the binary and multi-class classification of voice disorders. Despite progress, achieving high accuracy in multi-class classification remains challenging. This paper proposes a novel hybrid approach using a two-stage framework to enhance voice disorders classification performance, and achieve state-of-the-art accuracies in multi-class classification. Our hybrid approach, combines deep learning features with various powerful classifiers. In the first stage, high-level feature embeddings are extracted from voice data spectrograms using a pre-trained VGGish model. In the second stage, these embeddings are used as input to four different classifiers: Support Vector Machine (SVM), Logistic Regression (LR), Multi-Layer Perceptron (MLP), and an Ensemble Classifier (EC). Experiments are conducted on a subset of the Saarbruecken Voice Database (SVD) for male, female, and combined speakers. For binary classification, VGGish-SVM achieved the highest accuracy for male speakers (82.45% for healthy vs. disordered; 75.45% for hyperfunctional dysphonia vs. vocal fold paresis), while VGGish-EC performed best for female speakers (71.54% for healthy vs. disordered; 68.42% for hyperfunctional dysphonia vs. vocal fold paresis). In multi-class classification, VGGish-SVM outperformed other models, achieving mean accuracies of 77.81% for male speakers, 63.11% for female speakers, and 70.53% for combined genders. We conducted a comparative analysis against related works, including the Mel frequency cepstral coefficient (MFCC), MFCC-glottal features, and features extracted using the wav2vec and HuBERT models with SVM classifier. Results demonstrate that our hybrid approach consistently outperforms these models, especially in multi-class classification tasks. The results show the feasibility of a hybrid framework for voice disorder classification, offering a foundation for refining automated tools that could support clinical assessments with further validation.

Depression level prediction via textual and acoustic analysis.

Depression level prediction via textual and acoustic analysis.

Generative AI-Driven Multimodal Interaction System Integrating Voice and Motion Recognition

This research proposes a two-way interactive algorithm based on voice and motion recognition using generative AI technology to overcome the limitations of existing systems limited to simple command recognition. Current voice and motion recognition technologies are essential in enabling interaction between smart devices and users to enhance user experience. Still, they are mainly limited to recognizing and executing prescribed commands, which do not meet the diverse and complex needs of users. To solve these problems, this research aims to develop a technology that fuses and integrates voice and motion data based on advanced learning and prediction capabilities of generative AI, provides customized data optimized for each user's personality and situation in real-time, and enables more natural and efficient interactions. The main research content includes developing data analysis and processing algorithms that can integrally process multiple input channels, designing generative AI-based models for providing customized data to users, and implementing a two-way interactive system that maintains a natural conversation flow. In particular, the research is intended to combine generative AI language models with computer vision technology to comprehensively analyze user voice and motion data, enabling smart devices to understand and respond to user intent accurately. These technologies can potentially revolutionize the user experience in various areas, including smart homes, healthcare, education, and more. This study's results are expected to significantly contribute to the development of next-generation smart device interaction systems that could improve both efficiency and engagement of interactions.

Automated psychographic consumer segmentation: a text classification approach

Abstract This paper demonstrates that automated segmentation of consumers based on Self-Determination Theory (SDT) is feasible using relatively simple text classification methods. By analyzing 290 Swiss German consumer interviews across seven industries, we apply computational linguistics to identify and segment consumers according to their motivational profiles derived from SDT and its derivative, Basic Psychological Needs Theory (BPNT). The study compares seven machine learning algorithms, including one deep learning classifier, from five different conceptual bases. This reveals that simpler and more interpretable classifiers, such as those based on logistic regression, distance measures, or decision trees, outperform more complex neural network classifiers. Although the interview data were designed to manually identify motivation, the algorithms used were not pre-trained on SDT or BPNT concepts. The findings indicate that with a well-prepared data foundation, simpler algorithms can achieve high performance, making advanced algorithm design expertise unnecessary. This approach provides a practical blueprint for marketing managers to leverage the growing volume of text and voice data for consumer segmentation.

A quantum inspired machine learning approach for multimodal Parkinson’s disease screening

Parkinson’s disease, currently the fastest-growing neurodegenerative disorder globally, has seen a 50% increase in cases within just two years. As disease progression impairs speech, memory, and motor functions over time, early diagnosis is crucial for preserving patients’ quality of life. Although machine-learning-based detection has shown promise for detecting Parkinson’s disease, most studies rely on a single feature for classification and can be error-prone due to the variability of symptoms between patients. To address this limitation we utilized the mPower dataset, which includes 150,000 samples across four key biomarkers: voice, gait, tapping, and demographic data. From these measurements, we extracted 64 features and trained a baseline Random Forest model to select the features above the 80th percentile. For classification, we designed a simulatable quantum support vector machine (qSVM) that detects high-dimensional patterns, leveraging recent advancements in quantum machine learning. With this novel and simulatable architecture that can be run on standard hardware rather than resource-intensive quantum computers, our model achieves an accuracy of 90%, F-1 score of 0.90, and an AUC of 0.98—surpassing benchmark models. Utilizing an innovative classification framework built on a diverse set of features, our model offers a pathway for accessible global Parkinson’s screening.

Homomorphic encryption, privacy-preserving feature extraction, and decentralized architecture for enhancing privacy in voice authentication

This paper introduces a novel framework designed to bolster privacy protections within automated voice authentication systems, addressing mounting concerns as voice-based authentication grows in prominence. The widespread adoption of these systems has underscored apprehensions regarding the storage and processing of sensitive voice biometric data without adequate safeguards. To mitigate these risks, a modified framework is proposed, aiming to enhance privacy without compromising authentication accuracy and efficiency. Three key techniques are implemented to address these challenges. Firstly, advanced encryption methods are employed for secure voice data storage and transmission, through the homomorphic encryption to enable authentication processing on encrypted data. Secondly, a privacy-preserving feature extraction method is introduced, transforming raw voice inputs into irreversible representations to shield original biometric information. Additionally, the framework incorporates differential privacy mechanisms, adding controlled noise to aggregated voice data to prevent individual identification within large datasets. A user-centric consent and control model is proposed, empowering individuals to manage their voice profiles and authentication settings. Experimental findings demonstrate that the framework achieves enhanced authentication accuracy while markedly reducing privacy risks compared to conventional systems. This contribution addresses the ongoing challenge of balancing security and privacy in biometric authentication technologies.

Handover Scheme in LEO Satellite Networks Based on QoE for Streaming Media Services.

The development of satellite communications has received considerable attention in recent years. Early satellite communications were dominated by voice and low-speed data services, but now they must support high-speed multimedia services. Low Earth Orbit (LEO) satellites, because of their lower altitude orbits, have much smaller transmission loss and delay than Geostationary Earth Orbit (GEO) satellites, and they are an important part of the future realization of high-bandwidth and low-latency multimedia services. Among them, the on-demand streaming service has a large number of users in terrestrial communication and is also an important service component that will be in satellite communication environments in the future. However, LEO satellites face many challenges in handover and accessing due to their fast moving speed. Although many handover and access schemes for LEO satellites have been proposed and evaluated in existing studies, most of them stay at the level of quality of service (QoS), and few of them have been studied at the level of quality of experience (QoE). These studies also rarely consider the performance of multimedia services, including streaming services, in satellite communication environments, and there is no relevant simulation system to evaluate and examine them. Therefore, this paper builds a simulation system for streaming services in LEO satellite communication environments in order to simulate the initial buffering, rebuffering, and idle state of the users during service. Then, access and handover schemes for the QoE level of streaming service are proposed. Finally, our proposed scheme is evaluated based on this simulation system. From the simulation results, the simulation system proposed in this paper can successfully realize the various functions of users in on-demand streaming services and record the initial buffering and rebuffering events of users. And the streaming QoE-based access and handover scheme proposed in this paper can perform well in satellites, which operate within a resource-constrained environment.

Disease Prediction Using Machine Learning on Smartphone-Based Eye, Skin, and Voice Data: Scoping Review.

The application of machine learning methods to data generated by ubiquitous devices like smartphones presents an opportunity to enhance the quality of health care and diagnostics. Smartphones are ideal for gathering data easily, providing quick feedback on diagnoses, and proposing interventions for health improvement. We reviewed the existing literature to gather studies that have used machine learning models with smartphone-derived data for the prediction and diagnosis of health anomalies. We divided the studies into those that used machine learning models by conducting experiments to retrieve data and predict diseases, and those that used machine learning models on publicly available databases. The details of databases, experiments, and machine learning models are intended to help researchers working in the fields of machine learning and artificial intelligence in the health care domain. Researchers can use the information to design their experiments or determine the databases they could analyze. A comprehensive search of the PubMed and IEEE Xplore databases was conducted, and an in-house keyword screening method was used to filter the articles based on the content of their titles and abstracts. Subsequently, studies related to the 3 areas of voice, skin, and eye were selected and analyzed based on how data for machine learning models were extracted (ie, the use of publicly available databases or through experiments). The machine learning methods used in each study were also noted. A total of 49 studies were identified as being relevant to the topic of interest, and among these studies, there were 31 different databases and 24 different machine learning methods. The results provide a better understanding of how smartphone data are collected for predicting different diseases and what kinds of machine learning methods are used on these data. Similarly, publicly available databases having smartphone-based data that can be used for the diagnosis of various diseases have been presented. Our screening method could be used or improved in future studies, and our findings could be used as a reference to conduct similar studies, experiments, or statistical analyses.

Vowel segmentation impact on machine learning classification for chronic obstructive pulmonary disease

Vowel-based voice analysis is gaining attention as a potential non-invasive tool for COPD classification, offering insights into phonatory function. The growing need for voice data has necessitated the adoption of various techniques, including segmentation, to augment existing datasets for training comprehensive Machine Learning (ML) modelsThis study aims to investigate the possible effects of segmentation of the utterance of vowel "a" on the performance of ML classifiers CatBoost (CB), Random Forest (RF), and Support Vector Machine (SVM). This research involves training individual ML models using three distinct dataset constructions: full-sequence, segment-wise, and group-wise, derived from the utterance of the vowel "a" which consists of 1058 recordings belonging to 48 participants. This approach comprehensively analyzes how each data categorization impacts the model's performance and results. A nested cross-validation (nCV) approach was implemented with grid search for hyperparameter optimization. This rigorous methodology was employed to minimize overfitting risks and maximize model performance. Compared to the full-sequence dataset, the findings indicate that the second segment yielded higher results within the four-segment category. Specifically, the CB model achieved superior accuracy, attaining 97.8% and 84.6% on the validation and test sets, respectively. The same category for the CB model also demonstrated the best balance regarding true positive rate (TPR) and true negative rate (TNR), making it the most clinically effective choice. These findings suggest that time-sensitive properties in vowel production are important for COPD classification and that segmentation can aid in capturing these properties. Despite these promising results, the dataset size and demographic homogeneity limit generalizability, highlighting areas for future research.Trial registration The study is registered on clinicaltrials.gov with ID: NCT06160674.

Speech Emotion Recognition and Serious Games: An Entertaining Approach for Crowdsourcing Annotated Samples

Computer games have emerged as valuable tools for education and training. In particular, serious games, which combine learning with entertainment, offer unique potential for engaging users and enhancing knowledge acquisition. This paper presents a case study on the design, development, and evaluation of two serious games, “Silent Kingdom” and “Job Interview Simulator”, created using Unreal Engine 5 and incorporating speech emotion recognition (SER) technology. Through a systematic analysis of the existing research in SER and game development, these games were designed to elicit a wide range of emotion responses from player and collect voice data for the enhancement of SER models. By evaluating player engagement, emotional expression, and overall user experience, this study investigates the effectiveness of serious games in collecting speech data and creating more immersive player experiences. The research also explores the technical limitations of SER integration within game environments in real-time, as well as its impact on player enjoyment. Although there are some technology limitations due to the latency provided for real-time SER analysis, the results reveal that a properly developed game with integrated SER technology could become a more engaging and efficient tool for crowdsourcing speech data.

Protection of Acoustic Rights and Responses to Sound Infringement by Smart Energy Devices

With the advancement of artificial intelligence (AI) technology, the infringement of personal acoustic rights has become an increasingly severe issue. Objective factors, such as the ease of collecting voice data, have contributed to the frequent emergence of cases involving the infringement of personal acoustic rights through generative AI. In particular, the collection of voice data by smart energy devices has led to numerous legal and ethical issues. This paper examines the development and current status of legal frameworks protecting acoustic rights in Chinese law. It explores how the rise of AI and smart devices has challenged existing regulations and created new risks for personal acoustic rights. In response, this study advocates for the establishment of a robust voice licensing system, the incorporation of watermarking technology in AI-generated audio, and stricter regulatory oversight on usage scenarios and generated content. By implementing these measures, the study aims to strengthen the legal and technological safeguards for acoustic rights, ensuring effective protection in the age of AI.

Effects of Introducing Generative AI in Rehabilitation Clinical Documentation.

Introduction Healthcare professionals reportedly spend a significant proportion of their working hours on documentation. Therefore, we developed a generative AI solution specialized in creating clinical documentation for rehabilitation. This study aimed to examine the impact of generative AI on clinical documentation tasks. Methods Twelve rehabilitation professionals (physical therapists, occupational therapists, and speech-language pathologists) participated in this study. We compared conventional clinical documentation (Period A) with clinical documentation using a generative AI system (Period B). Measures taken for both periods included time required to complete the clinical documentation (documentation time), workload assessed using the National Aeronautics and Space Administration Task Load Index (NASA-TLX), and quality of the clinical documentation. Between-group comparisons of these measurements were performed. Additionally, we recorded the number of non-conversational voice memos (voice data inputs) in Period B. After the study, we assessed the participants' willingness to adopt generative AI (implementation intent) on a five-point scale. For statistical analysis, we compared documentation time, NASA-TLX scores, and documentation quality between the two periods. Time saved was determined by subtracting the documentation time of Period B from that of Period A, and a correlation analysis between the number of voice memos (voice data input) and the willingness to adopt the technology was conducted. Analyses were performed using R version 4.2.3(R Core Team, Durham, NC), with the level of significance set at 0.05. Results No significant difference was observed in the time required to prepare clinical documentation between Periods A and B. However, in Period B, the NASA-TLX time pressure score was significantly lower, while the quality of clinical documentation was significantly higher. Additionally, a strong positive correlation was observed between the reduction in documentation time and the number of voice memos (r = 0.71, p < 0.01), as well as a significant positive correlation with the willingness to adopt the system (r = 0.67, p < 0.05) during clinical documentation in Period B. Conclusion Our findings indicate that using generative AI for clinical documentation tasks can reduce time pressure and improve documentation quality. Moreover, the reduction in documentation time was associated with the frequency of voice memos and the degree of participants' willingness to adopt the system. These results suggest that, to achieve further reductions in workload and costs, considering the motivation and cooperative framework of healthcare professionals when introducing generative AI solutions is essential.

한국어 발화 텍스트 데이터 기반 성별 판별에 관한 사례연구

Natural language processing (NLP) technology is leading innovation in various fields along with the development of deep learning. For example, interactive artificial intelligence (AI) in an autonomous vehicle analyzes the user's utterance and provides personalized services such as recommending destinations, performing vehicle control commands, and temperature control. In addition, in the health care field, NLP technology is playing an important role in psychological counseling using voice data and disease diagnosis assistance services. At this time, it is known that applying natural language processing technology is more difficult than other languages because Korean language has unique linguistic characteristics such as freedom of word order, complex investigation, and mother change. In this study, gender is determined by reflecting the specificity of Korean speech data, and the optimal embedding technique and model are explored. To this end, we briefly introduced text embedding techniques such as TF-IDF, Doc2Vec, and BERT, various machine learning models such as decision trees, logistic regression, random forest, SVM, XGBoost, and RNN, and conducted case analysis using large-scale Korean speech data sets provided by AI Hub to compare the performance of data according to various analysis method.

Investigating Rijndael-Based Algorithms for Audio Ciphering

Voice encryption is crucial to protecting data during transmission and storage due to the widespread problem of document fabrication and forgery. Various methods have been used to protect important data, including encryption. Voice data encryption is a way to encrypt important messages within transmissions. Therefore, it needs a strong and efficient encryption algorithm, using the standard encryption algorithm known as Rijndeal. This method is reliable, secure and efficient, because the Rijndael algorithm is characterized by its key lengths and blocks of different sizes. The Rijndael algorithm is a method of decrypting audio files, making it a powerful tool for protecting sensitive information. This paper presents a method for encoding WAV audio files using the Rijndael algorithm that is compatible with many audio formats. This study converts audio data into a 256-bit audio sample that has been used to encode using the Rijndael algorithm, as this algorithm is known for its robustness and has won many awards. The working principle of this algorithm is to create multiple blocks with different sizes and keys (128 bits, 192 bits or 256 bits). The algorithm creates keys for the purpose of encryption, and they are combined into the audio files using the Rijndael algorithm and decrypted that has been noted as the return of the original text. The goal of this proposed methodology is to protect audio data from attacking people and reduce the chances of this data being disclosed. This methodology is used for the purpose of transferring, storing and keeping confidential audio files securely with security aspects. One of the most important features of this research is the strength of the encryption, which is represented by the Rigndael algorithm, which led to the file being broken and restored at a record rate of up to 100%. It also provides a cutting-edge and innovative methodology. For the purpose of securing, transmitting, storing and protecting audio data in all modern digital fields.

Parkinson's Disease Prediction Using Deep Learning Classification Algorithms

complaints arising from neurological disorders continue to increase today. At the same time, studies on diagnosis and treatment methods in medicine are increasing as technology advances. With the increasing interest in these areas, studies have been carried out on various diagnosis and follow-up systems related to Parkinson's disease. For this purpose, in this study, we studied the classification of a data set consisting of various voice recordings for each patient with the designed deep learning architecture in order to assist in the more objective diagnosis of Parkinson's disease. Although it is important for the estimation of the study to find different sound samples of each subject in the data set, it is not known how much these recordings represent all the sound recordings of the person. Recurrent neural networks, which are a deep learning architecture, are an efficient system that can achieve high success in voice data and can be preferred in the diagnosis and follow-up of Parkinson's disease. However, this study showed that in such a network design, much larger and more diverse data are needed to increase the classification rate, to make more accurate predictions in the field of medicine, and to make remote diagnosis.

Estimating the Pathophysiology of Phonotraumatic Vocal Hyperfunction Using Ambulatory Data and a Computational Model.

This study uses a voice production model to estimate muscle activation levels and subglottal pressure (PS) in patients with phonotraumatic vocal hyperfunction (PVH), based on ambulatory measurements of sound pressure level (SPL) and spectral tilt (H1-H2). In addition, variations in these physiological parameters are evaluated with respect to different values of the Daily Phonotrauma Index (DPI). The study obtained ambulatory voice data from patients diagnosed with PVH and a matched control group. To infer physiological parameters, ambulatory data were mapped onto synthetic data generated by a physiologically relevant voice production model. Inverse mapping strategies involved selecting model simulations that represented ambulatory distributions using stochastic (random) sampling weighted by probability with which different vowels occur in English. A categorical approach assessed the relationship between different values of DPI and changes in estimated physiological parameters. Results showed significant differences between the PVH and control groups in key parameters, including statistical moments of H1-H2, SPL, PS, and muscle activity of lateral cricoarytenoid (LCA) and cricothyroid (CT) muscles. Higher DPI values, reflecting more severe PVH, were associated with increased mean LCA activation and decreased LCA variability, along with decreased mean CT activation and increased median PS. These findings highlight the relationship between muscle activation patterns, PS, and the severity of vocal pathology as indicated by the DPI. It is hypothesized that a major driver of muscle activation and PS changes is the variation in maladaptive adjustments (vocal effort) when compensating for the presence of vocal pathology. This study demonstrated that noninvasive ambulatory voice data could be used to drive a voice production modeling process, providing valuable insights into underlying physiological parameters associated with PVH. Future research will focus on refining the predictive power of the modeling process and exploring the implications of these findings in further delineating the etiology and pathophysiology of PVH, with the ultimate goal to develop improved methods for the prevention, diagnosis, and treatment of PVH. https://doi.org/10.23641/asha.28352720.

Machine Learning and Neural Networks for IT-Diagnostics of Neurological Diseases

The article considers machine learning methods and neural networks for diagnosing neurological diseases (Alzheimer’s and Parkinson’s diseases) in patients based on voice analysis. Models of information about disease features (including frequency, jitter, mel-cepstral coefficients, etc.) extracted from voice data are presented. Various classifiers are used to train neural networks and recognize diseases. Among them are the GridSearchCV algorithm for optimizing the hyperparameters of the random forest classifier for recognizing Alzheimer’s disease (recognition accuracy is 87.6 %) and the KNN algorithm for training and testing on publicly available datasets of speech change features in patients with Parkinson’s disease. The KNN algorithm showed the best classification results compared to others, achieving an experimental accuracy of 94 % on the same datasets. It is noted that the use of multidimensional feature extraction and machine learning methods can improve the accuracy of early diagnosis of neurological diseases.