Breath Sounds Research Articles

Towards the Development of the Clinical Decision Support System for the Identification of Respiration Diseases via Lung Sound Classification Using 1D-CNN

Respiratory disorders are commonly regarded as complex disorders to diagnose due to their multi-factorial nature, encompassing the interplay between hereditary variables, comorbidities, environmental exposures, and therapies, among other contributing factors. This study presents a Clinical Decision Support System (CDSS) for the early detection of respiratory disorders using a one-dimensional convolutional neural network (1D-CNN) model. The ICBHI 2017 Breathing Sound Database, which contains samples of different breathing sounds, was used in this research. During pre-processing, audio clips were resampled to a uniform rate, and breathing cycles were segmented into individual instances of the lung sound. A One-Dimensional Convolutional Neural Network (1D-CNN) consisting of convolutional layers, max pooling layers, dropout layers, and fully connected layers, was designed to classify the processed clips into four categories: normal, crackles, wheezes, and combined crackles and wheezes. To address class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) was applied to the training data. Hyperparameters were optimized using grid search with k−fold cross-validation. The model achieved an overall accuracy of 0.95, outperforming state-of-the-art methods. Particularly, the normal and crackles categories attained the highest F1-scores of 0.97 and 0.95, respectively. The model’s robustness was further validated through 5−fold and 10−fold cross-validation experiments. This research highlighted an essential aspect of diagnosing lung sounds through artificial intelligence and utilized the 1D-CNN to classify lung sounds accurately. The proposed advancement of technology shall enable medical care practitioners to diagnose lung disorders in an improved manner, leading to better patient care.

Empowering Healthcare: TinyML for Precise Lung Disease Classification

Respiratory diseases such as asthma pose significant global health challenges, necessitating efficient and accessible diagnostic methods. The traditional stethoscope is widely used as a non-invasive and patient-friendly tool for diagnosing respiratory conditions through lung auscultation. However, it has limitations, such as a lack of recording functionality, dependence on the expertise and judgment of physicians, and the absence of noise-filtering capabilities. To overcome these limitations, digital stethoscopes have been developed to digitize and record lung sounds. Recently, there has been growing interest in the automated analysis of lung sounds using Deep Learning (DL). Nevertheless, the execution of large DL models in the cloud often leads to latency, dependency on internet connectivity, and potential privacy issues due to the transmission of sensitive health data. To address these challenges, we developed Tiny Machine Learning (TinyML) models for the real-time detection of respiratory conditions by using lung sound recordings, deployable on low-power, cost-effective devices like digital stethoscopes. We trained three machine learning models—a custom CNN, an Edge Impulse CNN, and a custom LSTM—on a publicly available lung sound dataset. Our data preprocessing included bandpass filtering and feature extraction through Mel-Frequency Cepstral Coefficients (MFCCs). We applied quantization techniques to ensure model efficiency. The custom CNN model achieved the highest performance, with 96% accuracy and 97% precision, recall, and F1-scores, while maintaining moderate resource usage. These findings highlight the potential of TinyML to provide accessible, reliable, and real-time diagnostic tools, particularly in remote and underserved areas, demonstrating the transformative impact of integrating advanced AI algorithms into portable medical devices. This advancement facilitates the prospect of automated respiratory health screening using lung sounds.

High performance method for COPD features extraction using complex network.

Objectives. The paper proposes a novel methodology for the classification of Chronic Obstructive Pulmonary Disease (COPD) utilizing respiratory sound attributes.Methods. The approach involves segmenting respiratory sounds into individual breaths and conducting extensive studies on this dataset. Spectral Transforms, various Wavelet Transforms are applied to capture distinct signal features. Complex Network is also employed to extract characteristic elements, generating novel representations of spectrogram data based on graph factors, including entropy, density, and position. The normalized and enriched data is then used to develop COPD classifiers using six machine learning algorithms, fine-tuning with appropriate training details and hyperparameter tuning.Results. Our results demonstrate robust performance, with ROC curves consistently exhibiting an Area Under the Curve (AUC) > 96% across different time-frequency transformations. Notably, the Random Forest algorithm achieves an AUC of 99.67%, outperforming other algorithms. Moreover, the Wavelet Daubechies 2 (Db2) consistently approaches 98% accuracy, particularly noteworthy in conjunction with the Naive Bayes algorithm.Conclusion. This study diagnosis patients through spectrogram images extracted from lung sounds. The application of Inverse Transforms, Complex Network, and Optimized Classification Algorithms yielded results beyond expectations. This methodology provides a promising approach for accurate COPD diagnosis, leveraging Machine Learning techniques applied to respiratory sound analysis.

A Framework for Detecting Pulmonary Diseases from Lung Sound Signals Using a Hybrid Multi-Task Autoencoder-SVM Model

Research focuses on the efficacy of Multi-Task Autoencoder (MTAE) models in signal classification due to their ability to handle many tasks while improving feature extraction. However, researchers have not thoroughly investigated the study of lung sounds (LSs) for pulmonary disease detection. This paper introduces a new framework that utilizes an MTAE model to detect lung diseases based on LS signals. The model integrates an autoencoder and a supervised classifier, simultaneously optimizing both classification accuracy and signal reconstruction. Furthermore, we propose a hybrid approach that combines an MTAE and a Support Vector Machine (MTAE-SVM) to enhance performance. We evaluated our model using LS signals from a publicly available database from King Abdullah University Hospital. The model attained an accuracy of 89.47% for four classes (normal, pneumonia, asthma, and chronic obstructive pulmonary disease) and 90.22% for three classes (normal, pneumonia, and asthma cases). Using the MTAE-SVM, the accuracy was further improved to 91.49% for four classes and 93.08% for three classes, respectively. The results indicate that the MTAE and MTAE-SVM have a considerable potential for detecting pulmonary diseases from lung sound signals. This could aid in the creation of more user-friendly and effective diagnostic tools.

Feasibility of snapshot testing using wearable sensors to detect cardiorespiratory illness (COVID infection in India).

The COVID-19 pandemic has challenged the current paradigm of clinical and community-based disease detection. We present a multimodal wearable sensor system paired with a two-minute, movement-based activity sequence that successfully captures a snapshot of physiological data (including cardiac, respiratory, temperature, and percent oxygen saturation). We conducted a large, multi-site trial of this technology across India from June 2021 to April 2022 amidst the COVID-19 pandemic (Clinical trial registry name: International Validation of Wearable Sensor to Monitor COVID-19 Like Signs and Symptoms; NCT05334680; initial release: 04/15/2022). An Extreme Gradient Boosting algorithm was trained to discriminate between COVID-19 infected individuals (n = 295) and COVID-19 negative healthy controls (n = 172) and achieved an F1-Score of 0.80 (95% CI = [0.79, 0.81]). SHAP values were mapped to visualize feature importance and directionality, yielding engineered features from core temperature, cough, and lung sounds as highly important. The results demonstrated potential for data-driven wearable sensor technology for remote preliminary screening, highlighting a fundamental pivot from continuous to snapshot monitoring of cardiorespiratory illnesses.

The unreliability of crackles: insights from a breath sound study using physicians and artificial intelligence

Background and introductionIn comparison to other physical assessment methods, the inconsistency in respiratory evaluations continues to pose a major issue and challenge.ObjectivesThis study aims to evaluate the difference in the identification ability of different breath sound.Methods/descriptionIn this prospective study, breath sounds from the Formosa Archive of Breath Sound were labeled by five physicians. Six artificial intelligence (AI) breath sound interpretation models were developed based on all labeled data and the labels from the five physicians, respectively. After labeling by AIs and physicians, labels with discrepancy were considered doubtful and relabeled by two additional physicians. The final labels were determined by a majority vote among the physicians. The capability of breath sound identification for humans and AI was evaluated using sensitivity, specificity and the area under the receiver-operating characteristic curve (AUROC).Results/outcomeA total of 11,532 breath sound files were labeled, with 579 doubtful labels identified. After relabeling and exclusion, there were 305 labels with gold standard. For wheezing, both human physicians and the AI model demonstrated good sensitivities (89.5% vs. 86.0%) and good specificities (96.4% vs. 95.2%). For crackles, both human physicians and the AI model showed good sensitivities (93.9% vs. 80.3%) but poor specificities (56.6% vs. 65.9%). Lower AUROC values were noted in crackles identification for both physicians and the AI model compared to wheezing.ConclusionEven with the assistance of artificial intelligence tools, accurately identifying crackles compared to wheezing remains challenging. Consequently, crackles are unreliable for medical decision-making, and further examination is warranted.

Lung Sounds Anomaly Detection with Respiratory Cycle Segmentation

Employing machine learning algorithms in the medical field has proven successful for some time now. Mostly computer vision techniques have been applied to medical images, while medical sound data has been somewhat overlooked. By using electronic stethoscopes, it is now possible to process both heartbeats and lung sounds. While some products are available for detecting anomalies in heartbeats, addressing lung-related anomalies presents a more intricate challenge. Applying a deep learning approach is hindered by insufficient data. Although some datasets do exist, the size and diversity of the data are too small for comprehensive analysis. This paper introduces a novel technique for detecting anomalies in lung sounds: first by combining two datasets, second by automatically segmenting each sound into respiratory cycles, and third by employing GFCCs as sound features.

Evaluation of the chronic oral toxicity of the classical ancient prescription Kai-Xin-San

Ethnopharmacological relevanceThe Kai-Xin-San (KXS), as an ancient classic prescription, has been used for the treatment of amnesia for thousands of years. Modern clinical and non-clinical pharmacological studies have found that it has significant therapeutic effects on dementia and depression, but there are relatively few studies on its safety. Aim of the studySubacute and chronic toxicity studies were conducted to investigate the symptoms, severity, target organs, development and recovery of toxic reactions, as well as the toxic dose. These studies provide technical data for ensuring the safety of KXS. Materials and methodsIn the sub-acute toxicity study, rats were orally administered KXS at doses of 0.80, 1.61, 3.22, and 6.43 g/kg body weight for a duration of 4 weeks. In the chronic toxicity study, rats were orally administered KXS at doses of 0.27, 0.81, and 2.43 g/kg body weight for a duration of 26 weeks, and a withdrawal study was conducted for a period of 4 weeks after the treatment.The rats were observed daily for clinical signs and mortality. Changes in body weight, food consumption, and water consumption were periodically monitored. Additionally, urinalysis results, hematological and biochemical parameters, relative organ weights, and pathology were monitored at specific observation time points. ResultsIn the sub-acute toxicity study, necropsy of dead and moribund rats revealed evident distension and swelling of the gastrointestinal tract, as well as thinning of the intestinal wall. The main adverse reactions observed included flatulence, piloerection, abnormal breathing sounds, and emaciation. Doses of 1.61 g/kg and below did not cause animal death. The gastrointestinal system is the main target organ of toxicity. In the chronic toxicity study, the no-observed-adverse-effect-level (NOAEL) of KXS was 0.27 g/kg, and its toxic effects were primarily concentrated in the gastrointestinal system. This led to secondary pathological changes in the immune system, hematopoietic system, and heart, suggesting that relevant indicators should be monitored when large doses are used clinically for an extended period of time. ConclusionsDuring the rodent toxicity evaluation, severe gastrointestinal damage was observed when KXS, powdered with crude drugs, was administered. The NOAEL for rats was found to be 0.27 g/kg/day.

8348 A Rare Case Of Liraglutide-Associated Angioedema Requiring Emergent Intubation

Abstract Disclosure: T. Amin: None. A. Mararenko: None. J. Cheng: None. R. Ong: None. S. Holland: None. K. Hu: None. Introduction: Glucagon-like peptide-1 (GLP-1) receptor agonists are widely used in the management of type 2 diabetes mellitus to enhance glycaemic control. While generally considered safe, recent reports indicate an increasing frequency of adverse drug reactions, with gastrointestinal symptoms such as pancreatitis becoming more prevalent. This case report highlights a rare occurrence of angioedema induced by the GLP-1 receptor agonist, Liraglutide. Case A 58-year-old Type 2 diabetic male presented with rapidly progressing tongue swelling, abdominal discomfort, and watery diarrhea, six hours after initiating Liraglutide therapy. On presentation, he was hemodynamically stable, afebrile, maintaining adequate oxygenation on room air. On examination, the patient was in moderate distress with generalized tongue swelling protruding out, with no swelling of eyelids or lips. No rashes were noted on exposed skin and clear breath sounds without wheezing on lung examination. Due to concern for an impending airway compromise, the patient was intubated. Chart review showed he had an episode of self-resolving tongue swelling a few months back, at which time he was taken off his Angiotensin receptor blocker. 2 weeks ago, he was started on a new brand of GLP-1 receptor agonist (Liraglutide) to achieve optimal glycemic control. No other drug allergies or allergic reactions were reported. C1-Esterase panel, eosinophil count, and inflammatory markers were within normal limits.On admission, Liraglutide was discontinued, he was placed on a basal-bolus insulin regimen, received methylprednisolone, famotidine, and diphenhydramine Subsequent improvement allowed for extubation on the third day, and the patient was discharged with plans to discontinue Liraglutide and an epinephrine pen for precautionary use. Conclusion: This case represents a rare and severe manifestation of angioedema associated with Liraglutide use in a diabetic patient. While angioedema has been infrequently reported with other GLP-1 receptor agonists such as Exenatide and Dulaglutide , our case underscores the importance of clinicians' awareness regarding this potential life-threatening complication with Liraglutide. Post-marketing surveillance data suggest that anaphylactic reactions with Liraglutide may occur rarely, emphasizing the need for vigilance among healthcare providers. Presentation: 6/2/2024

Implementation of home OSAHS screening device based on sleep sounds

Obstructive Sleep Apnea-Hypopnea Syndrome (OSAHS) is a common chronic sleeping respiratory condition. Screening for OSAHS approaches based on sleep breathing sounds has received much attention since it can provide fully noncontact monitoring at home. Deploying a diagnostic sound-based OSAHS system on a smartphone not only interferes with regular phone use but may also pose privacy problems, as well as suffer from decreasing accuracy owing to device variances. This work describes a local OSAHS monitoring system implementation designed for a dedicated device to estimate the apnea-hypopnea index and screening for OSAHS while doing away with concerns about the abovementioned issue. The OSAHS screening system is deployed on a Kendryte K510 (Canaan Inc.), a low-power RISC-V 64-bit CPU that has 2.5 TFLOPS, to analyze sleep breath sounds in a 9-s window. The design results are relatively low-cost and efficient for screening severe OSAHS. The equipment can be further used as a platform to evaluate the performance of the OSAHS diagnosis model.

Types of technology for teaching respiratory sounds: scoping review

Types of technology for teaching respiratory sounds: scoping review

Classification and Recognition of Lung Sounds Using Artificial Intelligence and Machine Learning: A Literature Review

This review explores the latest advances in artificial intelligence (AI) and machine learning (ML) for the identification and classification of lung sounds. The article provides a historical overview from the invention of the electronic stethoscope to the auscultation of lung sounds, emphasizing the importance of the rapid diagnosis of lung diseases in the post-COVID-19 era. The review classifies lung sounds, including wheezes and stridors, and explores their pathological relevance. In addition, the article deeply explores feature extraction strategies, measurement methods, and multiple advanced machine learning models for classification, such as deep residual networks (ResNets), convolutional neural networks combined with long short-term memory networks (CNN–LSTM), and transformer models (transformer). The article discusses the problems of insufficient data and replicating human expert experience and proposes future research directions, including improved data utilization, enhanced feature extraction, and classification using spectrograms. Finally, the article emphasizes the expanding role of AI and ML in lung sound diagnosis and their potential for further development in this field.

Unveiling the Impact of Respiratory Event-Related Hypoxia on Heart Sound Intensity During Sleep Using Novel Wearable Technology.

Cardiovascular disorders are the leading cause of mortality worldwide with obstructive sleep apnea (OSA) as the independent risk factor. Heart sounds are strong modalities to obtain clinically relevant information regarding the functioning of the heart valves and blood flow. The objective of this study was to use a small wearable device to record and investigate the changes in heart sounds during respiratory events (reduction and cessation of breathings) and their association with oxyhemoglobin desaturation (hypoxemia). Sleep assessment and tracheal respiratory and heart sounds were recorded simultaneously from 58 individuals who were suspected of having OSA. Sleep assessment was performed using in-laboratory polysomnography. Tracheal respiratory and heart sounds were recorded over the suprasternal notch using a small device with embedded microphone and accelerometer called the Patch. Heart sounds were extracted from bandpass filtered tracheal sounds using smoothed Hilbert envelope on decomposed signal. For each individual, data from 20 obstructive events during Non-Rapid Eye Movement stage-2 of sleep were randomly selected for analysis. A significant increase in heart sounds' intensities from before to after the termination of respiratory events was observed. Also, there was a significant positive correlation between the magnitude of hypoxemia and the increase in heart sounds' intensities (r>0.82, p<0.001). In addition, the changes in heart sounds were significantly correlated with heart rate and blood pressure. Our results indicate that heart sound analysis can be used as an alternative modality for assessing the cardiovascular burden of sleep apnea, which may indicate the risk of cardiovascular disorders.

Predictive value of clinico-radiological variables in the management of suspected foreign body aspiration in children: A ten-year retrospective cohort study

ABSTRACT Background: Definite history is not always present in children with foreign body aspiration (FBA), hence necessitating a high index of suspicion. Objective: To assess the predictive value of clinico-radiological variables among children presenting with features of suspected FBA and to document their course in a tertiary care teaching hospital. Materials and Methods: In this retrospective observational study, we included children aged below 15 years presenting with clinical features of suspected FBA. Data was obtained from case records. Multivariable binary logistic regression analysis (MVA) was performed to determine statistically significant predictors of FBA, and Weighted Risk Scores (WRS) were calculated to determine the significance of the model. Results: Of the 296 children included in the study and having undergone bronchoscopy, 84.5% were toddlers; 269 (90.87%) had a foreign body (FB) removed. Organic material was commonly found with 64.3% being peanut and right main bronchus, the commonest lodging site (50.6%). Of the 54 (18%) children with symptomatic but unwitnessed FBA, 45 (83.3%) had a FB retrieved. Choking with sudden onset cough, stridor, unilateral decreased breath sounds, abnormal chest radiograph, unilateral hyperinflation and mediastinal shift correlated with FBA (P &lt; 0.05). MVA confirmed statistical significance with choking with sudden onset cough to predict FBA (P = 0.024) with overall sensitivity and specificity of 60.9% and 77.8%, respectively. Total WRS with cut-off &gt;2 showed good power of discrimination with sensitivity, specificity and AUC of 60.2%, 77.8% and 0.704%, respectively. Conclusions: The proposed predictive value-based scoring system enhances accuracy in decision-making regarding bronchoscopy in children with suspected FBA.

Localized type tenosynovial giant cell tumor with metastases to lungs and pleura: a case report and literature review

BackgroundTenosynovial giant cell tumor is a rare soft tissue tumor of the synovium of joint, bursae, or tendon sheath. It is divided into localized or diffuse types on the basis of the growth pattern. Localized tenosynovial giant cell tumors are usually benign and treated successfully by excision. Diffuse tenosynovial giant cell tumors, in contrast to localized type, can destroy bone and cartilage and are associated with frequent local recurrences and distant metastasis. Localized type tenosynovial giant cell tumors rarely metastasize to distant organs. Here, we report a case of localized tenosynovial giant cell tumor presenting with lung metastases and systematically review literature.Case presentationA 55-year-old Asian male presented with a dry cough, right-sided chest pain and progressive dyspnea for 1 month. At 18 months before this presentation, he had undergone excision of a painless swelling on his right index finger. The swelling recurred within 3 months of excision, and a biopsy was then suggestive of a giant cell tumor. Given the suspicion of a giant cell tumor, a wide excision of the lesion was performed and the excisional biopsy was consistent with a diagnosis of tenosynovial giant cell tumor, localized type. At admission to our hospital, the patient had tachypnoea and absent breath sounds on the right side. A chest radiograph showed a right-sided pleural effusion with a homogenous opacity in the left mid-zone. A contrast-enhanced computed tomography of the chest and abdomen showed right massive pleural effusion and bilateral multiple lobulated heterogeneously enhancing pleural-based masses with areas of internal calcification. Pleural fluid analysis revealed an exudate with no malignant cells on cytology. A lung biopsy showed osteoclast-like giant cells and mononuclear spindle cells with areas of hemorrhage and necrosis, suggesting tenosynovial giant cell tumor metastasis. A final diagnosis of localized type tenosynovial giant cell tumor of the right index finger with metastases to the lungs and pleura was made. The patient passed away after receiving three cycles of denosumab injection owing to progressive disease.ConclusionLung metastasis is extremely rare in patients with localized tenosynovial giant cell tumor. The survival is usually poor in patients with lung metastasis. A close follow-up of patients with localized type tenosynovial giant cell tumor is necessary for early detection of pleuropulmonary complications.

Right Lung Agenesis Associated with Dextrocardia in Adulthood

Introduction: Pulmonary agenesis is a rare lung disease, occurring in approximately 1 out of every 100,000 births. Most cases of pulmonary agenesis result in death during the neonatal period. Although survival in cases of pulmonary agenesis is rare, it is possible to encounter lung agenesis in adults. Case: A 26-year-old female patient presented with progressive dyspnea. On physical examination, retraction of the breathing muscles, reduced chest excursions, and decreased breath sounds on the right side were observed. A chest X-ray raised suspicion of pulmonary agenesis. A computed tomography (CT) scan showed hypoplasia of the right lung, accompanied by mediastinal shift, hyperinflation of the left lung, and retraction of the diaphragm and liver to the right. Spirometry showed moderate obstruction. Conclusion: Pulmonary agenesis in adult patients is extremely rare. Chest X-rays and CT scans are the main diagnostic modalities for pulmonary agenesis. There is no specific treatment for asymptomatic cases. Management focuses on conservative and symptomatic care.

LungNeXt: A novel lightweight network utilizing enhanced mel-spectrogram for lung sound classification

Lung auscultation is essential for early lung condition detection. Categorizing adventitious lung sounds requires expert discrimination by medical specialists. This paper details the features of LungNeXt, a novel classification model specifically designed for lung sound analysis. Furthermore, we propose two auxiliary methods: RandClipMix (RCM) for data augmentation and Enhanced Mel-Spectrogram for Feature Extraction (EMFE). RCM addresses the issue of data imbalance by randomly mixing clips within the same category to create new adventitious lung sounds. EMFE augments specific frequency bands in spectrograms to highlight adventitious features. These contributions enable LungNeXt to achieve outstanding performance. LungNeXt optimally integrates an appropriate number of NeXtblocks, ensuring superior performance and a lightweight model architecture. The proposed RCM and EMFE methods, along with the LungNeXt classification network, have been evaluated on the SPRSound dataset. Experimental results revealed a commendable score of 0.5699 for the lung sound five-category task on SPRSound. Specifically, the LungNeXt model is characterized by its efficiency, with only 3.804M parameters and a computational complexity of 0.659G FLOPS. This lightweight and efficient model is particularly well-suited for applications in electronic stethoscope back-end processing equipment, providing efficient diagnostic advice to physicians and patients.

Custom Smartphone Application to Guide Locomotor-Respiratory Coupling in the Field Using Step-Adaptive Breathing Sounds.

While running is amongst the most popular activities for competition and leisure, an estimated 20-40% of runners may suffer from respiratory limitations. Some of these runners may benefit from breathing techniques to improve performance or alleviate respiratory discomfort. One such technique is locomotor-respiratory coupling (LRC), a frequency and phase synchronization of breath to step. Studies have demonstrated that LRC may benefit ventilatory efficiency via "step-driven flows," and some experts have argued it could be used for pacing exercise or increasing positive emotional states. Nevertheless, it may be difficult to perform without coaching or guidance. Here we propose RunRhythm, a custom smartphone application to deliver step-synchronized sound guidance for LRC. This concept builds on previous evidence that sound guidance can be effective and integrates features to maximize adherence and individualization. Preliminary results show that this application is a promising and efficacious method suitable for research on LRC in field exercise. Recommendations for use and further development are discussed to further develop this concept for the benefit of a wider population.

Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.

In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated, multimodal data-driven medical practice. We anticipate that this work will spark further discussion and inspire the development of innovative approaches in the next generation of medical M-LLM systems.

Fused Audio Instance and Representation for Respiratory Disease Detection

Audio-based classification techniques for body sounds have long been studied to aid in the diagnosis of respiratory diseases. While most research is centered on the use of coughs as the main acoustic biomarker, other body sounds also have the potential to detect respiratory diseases. Recent studies on the coronavirus disease 2019 (COVID-19) have suggested that breath and speech sounds, in addition to cough, correlate with the disease. Our study proposes fused audio instance and representation (FAIR) as a method for respiratory disease detection. FAIR relies on constructing a joint feature vector from various body sounds represented in waveform and spectrogram form. We conduct experiments on the use case of COVID-19 detection by combining waveform and spectrogram representation of body sounds. Our findings show that the use of self-attention to combine extracted features from cough, breath, and speech sounds leads to the best performance with an area under the receiver operating characteristic curve (AUC) score of 0.8658, a sensitivity of 0.8057, and a specificity of 0.7958. Compared to models trained solely on spectrograms or waveforms, the use of both representations results in an improved AUC score, demonstrating that combining spectrogram and waveform representation helps to enrich the extracted features and outperforms the models that use only one representation. While this study focuses on COVID-19, FAIR’s flexibility allows it to combine various multi-modal and multi-instance features in many other diagnostic applications, potentially leading to more accurate diagnoses across a wider range of diseases.