Multi Modal Research Articles

LLMasMMKG: LLM Assisted Synthetic Multi-Modal Knowledge Graph Creation For Smart City Cognitive Digital Twins

The concept of a Smart City (SC) Cognitive Digital Twin (CDT) presents significant potential for optimizing urban environments through sophisticated simulations, predictions, and informed decision-making. Comprehensive Knowledge Representations (KRs) that effectively integrate the diverse data streams generated by a city are crucial to realizing this potential. This paper addresses this by introducing a novel approach that leverages Large Language Models (LLMs) to automate the construction of synthetic Multi-Modal (MM) Knowledge Graphs (KGs) specifically designed for a SC CDT. Recognizing the challenges in fusing and aligning information from disparate sources, our method harnesses the power of LLMs for natural language understanding, entity recognition, and relationship extraction to seamlessly integrate data from sensor networks, social media feeds, official reports, and other relevant sources. Furthermore, we explore the use of LLM-driven synthetic data generation to address data sparsity issues, leading to more comprehensive and robust KGs. Initial outputs demonstrate the effectiveness of our approach in constructing semantically rich and interconnected synthetic KGs, highlighting the significant potential of LLMs for advancing SC CDT technology.

Development of Multi Modal Image Fusion Techniques in Night Vision Technology Using Deep Learning

This study is aimed at exploring new development in nighttime image processing based on nighttime photography/night vision. We use various learning methodologies based on neural networks to develop multisensor data processing techniques. The ability of Convolutional Neural Networks (CNN) to enhance image quality and clarity under conditions of low light or at night is the focus of our research. To do this A large database has been created, containing 3200 images both taken from the internet as well as our own. This data is preprocessed, normalized and features expanded using some common methods of data augmentation in order to optimize model performance. We compared our proposed CNN model with existing literature through a series of experiments. Performance was measured using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Feature Similarity Index (FSIM). The Results are now in. They showed that our model can improve various metrics with higher scores. And they show the real-time monitoring capabilities in urban streets for our model. This research brings multisensor image fusion based on night-adjacent techniques forward from night vision technology and provides insight into practical and potential future directions for using deep learning techniques to improve safety and security in many different environments.

Effect of Monovalent Cations on the Structure and Dynamics of Multimodal Chromatographic Surfaces.

While multimodal (MM) chromatography is a promising approach for purifying proteins, the lack of a fundamental understanding of how ion-ligand interactions govern selectivity limits its use in the biopharmaceutical industry. This study uses molecular dynamics simulations to study the interactions between simple monovalent cations and two commonly used structurally similar multimodal chromatography ligands, the Capto ligand and Nuvia cPrime, immobilized on the surface. On the Capto ligand surface, ion presence and type play a key role in modulating the formation of phenyl rings and carboxylate clusters. The flexible linkage attaching the Capto ligand to the self-assembled monolayer (SAM) surface allowed multiple ligands to form interactions with the small cations, while large cations interacted less strongly, following the order Li+ > Na+ > K+ > Cs+. Thus, smaller cations resulted in greater ordering on the surface and lower ion diffusivities, while larger cations resulted in less ordering and higher ion diffusivities, following the order Li+ < Na+ < K+ < Cs+. In contrast, due to the rigid attachment of Nuvia cPrime to the SAM surfaces, the cations bound less strongly and had a much smaller effect on ligand clustering or ordering. Additionally, ions in the presence of the Nuvia cPrime surface had generally greater diffusivities than those in the presence of the Capto ligand. Overall, the interaction of cations with the multimodal ligands can lead to unique configurations on the SAM that likely contribute to differential behavior in biological separations.

Enhancing the Zebra Optimization Algorithm with Chaotic Sinusoidal Map for Versatile Optimization

In this study, the Chaotic Sinusoidal Map (CSM)-enhanced Zebra Optimization Algorithm (CZOA)is introduced. CZOA combines CSM's integration strengths with ZOA's optimization skills. ZOA already exhibitsgreat optimization capabilities, but the addition of CSM increases its potential even more. This addition greatlystrengthens ZOA's exploration and exploitation skills and increases its flexibility for various optimization tasks.CZOA outperforms both the original ZOA and contemporary optimisation methods on 23 benchmark functions,including high-dimensional (FD), multimodal (MM), and unimodal (UM) challenges. Using the chaos of CSM toinvestigate regional optimal and determine better convergence and exploration-exploitation equilibrium are shownby CZOA, which also shows more profitable solution locations. CZOA demonstrates its resilience and versatilitythrough multiple benchmark activities, underscoring its potential as an adaptable optimisation tool. CZOAbecomes a potent metaheuristic by combining biological inspiration and chaotic dynamics to solve difficultoptimisation problems. Inspired by the natural behaviour of zebras, the Zebra Optimisation Algorithm (ZOA) is arelatively new optimisation technique. It makes use of a herd behaviour mechanism and the ideas of leadership andfollowing, in which members of the population—zebras in this case—cooperate to solve optimisation issues in thebest possible ways

IMAGE CAPTIONING WITH AUDIO INTEGRATION

Investigating the synergy between image captioning and audio integration, our study employs an advanced Encoder-Decoder framework. Integrating visual and auditory features, the model enhances caption precision. Through extensive experimentation, our results reveal a notable improvement in contextual understanding compared to conventional image captioning models. This research underscores the potential of multimodal approaches for enriching multimedia applications, particularly in contexts where comprehensive content comprehension and accessibility are paramount. Key Words: Deep Learning, Natural Language Processing, Multi Modal Attention Mechanism, Image Captioning.

Deep Learning-Based Depression Analysis Among College Students Using Multi Modal Techniques

Deep Learning-Based Depression Analysis Among College Students Using Multi Modal Techniques

Cross Model Attention based Deep Learning for Multi Modal Epilepsy Detection

Cross Model Attention based Deep Learning for Multi Modal Epilepsy Detection

Towards two-point neuron-driven energy-efficient multimodal open master hearing aid

Here we demonstrate a two-point neuron-inspired audio-visual (AV) open Master Hearing Aid (openMHA) framework for on-chip energy-efficientspeech enhancement (SE). The developed system is compared against state-of-the-art cepstrum-based audio-only (A-only) SE and conventional point-neuron inspired deep neural net (DNN) driven multimodal (MM) SE. Pilot experiments demonstrate that the proposed system outperforms audio-only SE in terms of speech quality and intelligibility and performs comparably to point neuron-inspired DNN with a significantly reduced energy consumption at any time, both during training and inferencing.

Sentiment-Driven Cryptocurrency Price Prediction: A Machine Learning Approach Utilizing Historical Data and Social Media Sentiment Analysis

This research explores the impact of social media sentiments on predicting Bitcoin prices using machine learning models, integrating on-chain data, and applying a Multi Modal Fusion Model. Historical crypto market, on-chain, and Twitter data from 2014 to 2022 were used to train models including K-Nearest Neighbors, Logistic Regression, Gaussian Naive Bayes, Support Vector Machine, Extreme Gradient Boosting, and Multi Modal Fusion. Performance was compared with and without Twitter sentiment data which was analysed using the Twitter-roBERTa and VADAR models. Inclusion of sentiment data enhanced model performance, with Twitter-roBERTa-based models achieving an average accuracy score of 0.81. The best performing model was an optimised Multi Modal Fusion model using Twitter-roBERTa, with an accuracy score of 0.90. This research underscores the value of integrating social media sentiment analysis and onchain data in financial forecasting, providing a robust tool for informed decision-making in cryptocurrency trading.

Generation of Medical Reports from Chest X-Ray Images using Multi Modal Learning Approach

Generation of Medical Reports from Chest X-Ray Images using Multi Modal Learning Approach

The Solution for Girls with No Sex Appeal: The Ironized Yeast – A Multi Modal Discourse Analysis of Vintage Advertisements

This paper presents a discourse analysis of three vintage advertisements for Ironized Yeast using Multi Modal Discourse Analysis and Critical Discourse Analysis. Drawing on the concept of erotic capital by Hakim (2011), the study explores how these advertisements exploit women’s insecurities about their body shape while offering a solution that goes beyond physical attractiveness. The analysis reveals that the advertisements promote the idea that women can leverage their erotic capital to enhance their sex appeal, thereby increasing their chances of success in the dating and marriage market. This research examines the construction and perpetuation of the seductive power associated with a curvy figure. The findings highlight the intricate interplay between discourses of attractiveness, power dynamics, and societal expectations, urging critical reflection on the influence of such representations on individuals and broader social dynamics.

A Comparative Study to Evaluate the Effectiveness of Perioperative Multi Modal Analgesia with Ultrasound Guided Bilateral Subcostal TAP Block for Post-Operative Analgesia in Patients undergoing Elective Laparoscopic Cholecystectomy under General Anaesthes

Background: Laparoscopic cholecystectomy is usually performed as a day care procedure for diseases involving the gall bladder. Pain in the immediate post-operative period is due to multiple factors and hence different modalities of pain relief are used. The present study was conducted to study the effectiveness of Peri-operative Multi-Modal Analgesia with ultrasound guided bilateral subcostal TAP block for Post-operative analgesia in 60 patients undergoing elective laparoscopic cholecystectomy.&#x0D; Methods: In this hospital based, randomized prospective interventional study patients were randomly allocated into 4 groups of 15 in each group, Group B received Bilateral Ultrasound guided Subcostal TAP Block with 20 ml of 0.25% Bupivacaine, Group P received Tab Pregabalin tablet 150 mg, Group D received Inj Dexamethasone 8 mg IV and Group C was the control group. The hemodynamic changes like heart rate, blood pressure, saturation was monitored both intra and post operatively. The post-operative VAS scoring, duration of analgesia, time for first rescue analgesic and the total dose of analgesics in 24 hours were noted.&#x0D; Results: The VAS Scores indicated that Group B&gt; P&gt; D&gt; C provided better analgesia to the patients with a P Value of &lt; 0.01. Group B took the longest mean duration (6.8± 1.15 hours) to take the 1st rescue analgesic and least mean total dose of rescue analgesic in 24 hours followed by Group P, Group D and Group C which was statistically significant.&#x0D; Conclusion: USG guided B/L TAP Block was superior to other modalities in providing postoperative analgesia for patients undergoing laparoscopic cholecystectomy.

Optimal Power Flow Using a Hybrid Improved Harris Hawks Optimization Algorithm-Pattern Search Method

The optimal power flow (OPF) problems for effective power system planning and operations are solved using a hybrid Improved Harris Hawks Optimization and Pattern Search (PS) (hIHHO-PS) algorithm in this research. The OPF problem is considered minimization of non-convex active power generation fuel cost (NAPGFC), total active power losses (TPL), shunt capacitor cost (SCC), voltage variation (VD), and maximization of social welfare (SW) and loadability factor (LF) objective functions. The proposed hIHHO-PS algorithm was tested on different benchmark test functions (unimodal (UM), multi-modal (MM), and fixed-dimension MM functions) and IEEE 30 bus test system, and obtained results are validated with existing algorithms. These results show the proposed hIHHO-PS algorithm outperformed the others. Further, to enhance the OPF results, optimally place the multi-line Flexible Alternating Current Transmission System (FACTS) devices (unified power flow controller (UPFC) and generalized unified power flow controller (GUPFC)) using severity index. Furthermore, test the effects of control modes of multi-line UPFC and GUPFC devices on OPF results. These results show the multi-line FACTS devices with control mode enhance the OPF results, especially GUPFC in control mode operation.

The therapeutic effects of multimodal exercise for people with Parkinson's: A longitudinal community-based study

IntroductionIndividuals with Parkinson’s Disease (PD) can develop a range of motor and non-motor symptoms due to its progressive nature and lack of effective treatments. Exercise interventions, such as multimodal (MM) programmes, may improve and sustain physical or cognitive function in PD. However, studies usually evaluate physical performance, cognition, and neuroprotective biomarkers separately and over short observation periods. MethodsPart one evaluates the effects of a weekly community-based MM exercise class (60 min) on physical function in people with PD (PwP). Exercise participants (MM-EX; age 65 ± 9 years; Hoehn and Yahr (H&Y) scale ≤ IV) completed a battery of functional assessments every 4 months for one (n = 27), two (n = 20) and three years (n = 15). In part two, cognition and brain-derived neurotrophic factor (BDNF) levels were assessed over 6-to-8 months and compared to aged-matched non-active PwP (na-PD, n = 16; age 68 ± 7 years; H&Y scale ≤ III) and healthy older adults (HOA, n = 18; age 61 ± 6 years). ResultsMM-EX significantly improved walking capacity (5% improvement after 8 months), functional mobility (11% after 4 months), lower extremity strength (15% after 4 months) and bilateral grip strength (9% after 28 months), overall, maintaining physical function across 3 years. Group comparisons showed that only MM-EX significantly improved their mobility, lower extremity strength, cognition and BDNF levels. ConclusionWeekly attendance to a community-based MM exercise group session can improve and maintain physical and cognitive function in PD, with the potential to promote neuroprotection.

Evaluation of preferred binding regions on ubiquitin and IgG1 FC for interacting with multimodal cation exchange resins using DEPC labeling/mass spectrometry.

There is significant interest in identifying the preferred binding domains of biological products to various chromatographic materials. In this work, we develop a biophysical technique that uses diethyl pyrocarbonate (DEPC) based covalent labeling in concert with enzymatic digestion and mass spectrometry to identify the binding patches for proteins bound to commercially available multimodal (MM) cation exchange chromatography resins. The technique compares the changes in covalent labeling of the protein in solution and in the bound state and uses the differences in this labeling to identify residues that are sterically shielded upon resin binding and, therefore, potentially involved in the resin binding process. Importantly, this approach enables the labeling of many amino acids and can be carried out over a pH range of 5.5-7.5, thus enabling the protein surface mapping at conditions of interest in MM cation exchange systems. The protocol is first developed using the model protein ubiquitin and the results indicate that lysine residues located on the front face of the protein show dramatic changes in DEPC labeling while residues present on other regions have minimal or no reductions. This indicates that the front face of ubiquitin is likely involved in resin binding. In addition, surface property maps indicate that the hypothesized front face binding region consists of overlapping positively charged and hydrophobic patches. The technique is then employed with an IgG1 FC and the results indicate that residues on the CH 2-CH 3 interface and the hinge are significantly sterically shielded upon binding to the resin. Further, these regions are again associated with significant overlap of positively charged and hydrophobic patches. On the other hand, while, residues on the CH 2 and the front face of the IgG1 FC also exhibited some changes in DEPC labeling upon binding, these regions have less distinct charged and hydrophobic patches. Importantly, the hypothesized binding patches identified for both ubiquitin and FC using this approach are shown to be consistent with previously reported NMR studies. In contrast to NMR, this new approach enables the identification of preferred binding regions without the need for isotopically labeled proteins or chemical shift assignments. The technique developed in this work sets the stage for the evaluation of the binding domains of a wide range of biological products to chromatographic surfaces, with important implications for designing biomolecules with improved biomanufacturability properties.

Multi Modal Feature Extraction for Classification of Vascular Dementia in Post-Stroke Patients Based on EEG Signal.

Dementia is a term that represents a set of symptoms that affect the ability of the brain's cognitive functions related to memory, thinking, behavior, and language. At worst, dementia is often called a major neurocognitive disorder or senile disease. One of the most common types of dementia after Alzheimer's is vascular dementia. Vascular dementia is closely related to cerebrovascular disease, one of which is stroke. Post-stroke patients with recurrent onset have the potential to develop dementia. An accurate diagnosis is needed for proper therapy management to ensure the patient's quality of life and prevent it from worsening. The gold standard diagnostic of vascular dementia is complex, includes psychological tests, complete memory tests, and is evidenced by medical imaging of brain lesions. However, brain imaging methods such as CT-Scan, PET-Scan, and MRI have high costs and cannot be routinely used in a short period. For more than two decades, electroencephalogram signal analysis has been an alternative in assisting the diagnosis of brain diseases associated with cognitive decline. Traditional EEG analysis performs visual observations of signals, including rhythm, power, and spikes. Of course, it requires a clinician expert, time consumption, and high costs. Therefore, a quantitative EEG method for identifying vascular dementia in post-stroke patients is discussed in this study. This study used 19 EEG channels recorded from normal elderly, post-stroke with mild cognitive impairment, and post-stroke with dementia. The QEEG method used for feature extraction includes relative power, coherence, and signal complexity; the evaluation performance of normal-mild cognitive impairment-dementia classification was conducted using Support Vector Machine and K-Nearest Neighbor. The results of the classification simulation showed the highest accuracy of 96% by Gaussian SVM with a sensitivity and specificity of 95.6% and 97.9%, respectively. This study is expected to be an additional criterion in the diagnosis of dementia, especially in post-stroke patients.

Two Stage Multi Modal Deep Learning Kannada Character Recognition Model Adaptive to Discriminative Patterns of Kannada Characters

Objectives: Designing optical character recognition systems for Kannada character is challenging due to higher self-similarity in characters and higher number of character classes. This work addresses the two major problems of reduced accuracy and higher false positives due to higher self-similarity in characters. Methods: This work proposes a two stage multi modal deep learning technique to handle the complexity in Kannada character recognition. The characters are first grouped based on morphological and structural similarity. A novel morphological/structural difference maximization convolution kernel based deep learning modal is trained for each of character group to recognize characters in that group. This divide and conquer strategy reduce the complexity of deep learning model in learning discriminative features for Kannada character recognition. Findings: The proposed two stage multi modal deep learning provides 89% recognition accuracy which is at least 6% higher compared to existing works. The false positives in proposed solution are at least10% lower compared to existing works. Novelty: A novel sector intensity distribution feature specific to curve structure of Kannada characters for deciding the number of groups of characters. Classifier is designed for each group of characters. Classification is done using convolutional neural network with a novel morphological/structural difference maximization convolution kernel to solve the structural similarity problem in Kannada character recognition. Keywords: Handwritten character Recognition; Deep Learning; Machine Learning; Pattern Recognition; Fuzzy Gaussian

Multimodal Neuromonitoring in Pediatric Neurocritical Care: Current Perspectives.

Children with neurological illness in the critical care unit are always at higher risk of developing secondary brain injury (SBI). Brain insult can lead to changes in cerebral autoregulation, intracranial pressure (ICP), cerebral oxygenation, and metabolism. This can cause a raised ICP, cerebral ischemia, hypoxia, excitotoxicity, cellular energy failure, and nonconvulsive status epilepticus. Simultaneous and continuous assessment of these parameters will help to improve patient care and neurological outcomes. Even thoughclinical examination and neuroimaging can help in the initial diagnosis of the neurological illness, they may not be helpful in continuous monitoring of cerebralpathophysiological changes. The ideal single neuromonitoring device to detect these real-time changes is currently unavailable. However, a range of invasive and noninvasive monitors are available to monitor these cerebral functional parameters. Invasive monitoring techniques include invasive ICP monitoring, cerebral autoregulation monitoring, brain tissue partial oxygen pressure, and cerebral microdialysis. Noninvasive-monitoring techniques include pupillometry, brain and ocular ultrasonography, near-infraredspectroscopy, and electrophysiological monitoring. Multimodal (MM) neuromonitoring involves incorporating these techniques and tools for the early identification and treatment of primary and secondary brain insults. The utility and feasibility of most of these techniques are well described in adult neurocritical care. Even though the evidence on their usage in children is primarily available in pediatric traumatic brain injury, the emerging data help to further expand their utility in pediatric nontraumatic coma. MM neuromonitoring aims to provide clinical and pathophysiological information to the intensivists to improve their understanding of the child's neurological status and to formulate patient-specific treatment approaches.

Hybrid Ensemble Based Machine Learning for Smart Building Fire Detection Using Multi Modal Sensor Data

Hybrid Ensemble Based Machine Learning for Smart Building Fire Detection Using Multi Modal Sensor Data

A Review on Bilayer Tablet Technology for Multi Modal Drug Delivery

Bilayer tablets are cutting-edge drug delivery technologies that enable the delivery of two medicines with various release patterns combined in a single unit. Bilayer tablets can deliver two incompatible medications in a single formulation, prolong the effect of the drug(s), and increase patient compliance. Bilayer tablets feature an active substance in one layer that is released immediately and another layer that is released later, either as a second dosage or in a prolonged release manner. The drawbacks of single-layered tablets are being overcome by the advancement of useful technology in bilayer tablets. Bilayer tablet technology requires a lot of resources, though. For each technical stage, a careful selection of excipients and manufacturing conditions is also necessary. High blood pressure patients frequently struggle or fail to control their BP with a single medication. To achieve treatment objectives, the majority of hypertension patients will require the use of two or more antihypertensive medications. When treating hypertension in patients whose blood pressure cannot be well controlled with monotherapy, combinations of antihypertensive medications from different categories have been demonstrated to be more successful than either medication alone. Combining two antihypertensive drugs with complementary modes of action may decrease blood pressure more effectively than using any one of these drugs by alone.&#x0D; Keywords: Bi-layer tablet, Tablet manufacturing, Bi-layer tableting techniques.