Human Interface Research Articles

Deep-SFER: deep convolutional neural network and MFCC an effective speech and face emotion recognition

There has been a lot of progress in recent years in the fields of expert systems, artificial intelligence (AI) and human machine interface (HMI). The use of voice commands to engage with machinery or instruct it to do a certain task is becoming more common. Numerous consumer electronics have SIRI, Alexa, Cortana, and Google Assistant built in. In the field of human-device interaction, emotion recognition from speech is a complex research subject. We can't imagine modern life without machines, so naturally there's a need to create a more robust framework for human-machine communication. A number of academics are now working on speech emotion recognition (SER) in an effort to improve the interaction between humans and machines. We aimed to identify four fundamental emotions: angry, unhappy, neutral and joyful from speech in our experiment. As you can hear below, we trained and tested our model using audio data of brief Manipuri speeches taken from films. This task makes use of convolutional neural networks (CNNs) to extract functions from speech in order to recognize different moods using the Mel-frequency cepstral coefficient (MFCC).

Real-time anomaly detection in electric motor operation noise

Anomaly detection plays a very important role in many fields to identify abnormalities occurring in the system earlier. This study proposes a new abnormality detection solution for 3-phase electric motors based on their working noise. Normal and abnormal operating noise data sets for an electric motor were acquired in the laboratory. These datasets are converted into the corresponding two-dimensional gray spectrogram image sets. The normal set is used to train the autoencoder (AE) model to find the abnormality evaluation threshold. This threshold is validated again with anomalous data sets. The trained AE is then quantized to be installed on a system consisting of two duo-core microcontroller units (MCUs) for real-time testing. Free real-time operating system (FreeRTOS), a real-time operating system, is used to schedule tasks on the system. Experimental results show that the designed anomaly detector can accurately detect over 99% of abnormal events. The system can communicate with a supervisory control and data acquisition (SCADA) application running on the S7-1200 programmable logic controller (PLC) platform using the Modbus transmission control protocol (TCP) protocol. The SCADA application can continuously record evaluated results from the system and adjust abnormal thresholds for the system directly on the human-machine interface (HMI) screen.

The HADRIAN novel human–machine interface prototype for automated driving: safety and impact assessment

AbstractThe current paper was performed within the HADRIAN project and focuses on exploring the effects of innovative Human–Machine Interface (HMI) prototypes on safety, driving performance, and driver perceptions. Employing driving simulator experiments and questionnaires, this study investigates whether HADRIAN innovative HMI enhances safety and receives positive evaluations from drivers. Specifically, the research centers on a driving simulator experiment that evaluates novel HMI prototypes designed to improve automated driving at SAE Levels 2 or 3. To facilitate HMI assessment, a tailored safety and impact assessment methodology was developed using unique Key Performance Indicators (KPIs). To benchmark and generate a total score for the HADRIAN HMI, data envelopment analysis was deployed based on the aforementioned KPIs. The findings shed light on the influence of HADRIAN HMI innovations on safety and perceived impact when compared to a baseline “state-of-the-art” HMI. Subsequently, a comprehensive discussion unfolds, highlighting the key KPIs that contributed significantly to the safety and perceived impact scores. This method and its outcomes can serve as a valuable resource for other HMI stakeholders, enabling them to employ similar human-centered assessment methodologies to assess the safety and perceived impact of potential HMI configurations.

Design and building CNC engraving router machine using Arduino Uno with GRBL control system

Computer Numerical Control Machines are one of the advances in automatically operated machining technology that can meet the need for complex and high-precision products. CNC machine parts generally have a mechanical and electrical system, also called a machine controller, that uses a Human Machine Interface at a high price. One of the software used to control the CNC machine is Arduino Uno, a PC-based GRBL System with a low price and the same function. This software is not far from the advantages of the HMI panel. Operating the machine with this software is also not that difficult. In this research, we intend to make a CNC machine using the Arduino Uno control system and software with a PC-based GRBL System, making it easier for operators to operate it. The results of the research that has been carried out show that the CNC has a size of 800x600x50mm with a system that is operated well and precisely

PLC/HMI Based Portable Workbench for PLC and Digital Logic Learning and Application Development

A Programmable logic controller (PLC) uses the digital logic circuits and their operating concepts in its hardware structure and its programming instructions and algorithms. Therefore, the deep understanding of these two items is staple for the development of control applications using the PLC. This target is only possible through the practical sensing of the various components or instructions of these two items and their applications. In this work, a user-friendly and re-configurable ladder, digital logic learning and application development design and testing platform has been designed and implemented using a Programmable Logic Controller (PLC), Human Machine Interface panel (HMI), four magnetic contactors, one Single-phase power line controller and one Variable Frequency Drive (VFD) unit. The PLC role is to implement the ladder and digital logic functions. The HMI role is to establish the virtual circuit wiring and also to drive and monitor the developed application in real time mode of application. The magnetic contactors are to play the role of industrial field actuators or to link the developed application control circuit to another field actuator like three phase induction motor. The Single -phase power line controller is to support an application like that of the soft starter. The VFD is to support induction motor driven applications like that of cut-to-length process in which steel coils are uncoiled and passed through cutting blade to be cut into required lengths. The proposed platform has been tested through the development of 14 application examples. The test results proved the validity of the proposed platform.

Ultrathin Transistors and Circuits for Conformable Electronics.

Adapting electronics to perfectly conform to nonplanar and rough surfaces, such as human skin, is a challenging task, which could open up new applications in fields of high economic and scientific interest, ranging from health to robotics, human-machine interface, and Internet of Things. The key to success lies in defining a technology that can lead to ultrathin devices, exploiting ultimately thin materials, with high mechanical flexibility and excellent electrical properties. Here, we report a hybrid approach for the development of high-performance, ultrathin and conformable electronic devices, based on the integration of semiconducting transition metal dichalcogenides, i.e., MoS2, with organic gate dielectric material, i.e., polyvinyl formal (PVF) combined with inkjet printed PEDOT:PSS electrodes. Through this novel approach, transistors and simple digital and analogue circuits are fabricated by a sequential stacking of ultrathin (nanometer) layers on a few micrometers thick polyimide substrate, which guarantees the high flexibility mandatory for the targeted applications.

A numerical conductive network model for investigating the strain-sensing response of graphene nanoplatelets-filled elastomeric strain sensors

Strain sensors have attracted great attentions for practical applications such as human-machine interface. It is challenging to model the electrical behavior of elastomeric strain sensors modified with nanomaterials. This study presents a finite element simulation to explore the piezoresistivity of graphene nanoplatelet (GNP)-filled elastomeric nanocomposites. The percolation network model is employed to determine the critical distance necessary for electrical percolation. The finite element percolation network model uses the Euclidian distance between GNPs distributed inside the representative volume element to calculate the resistivity. The influence of various parameters, including GNP alignment direction, aspect ratio, and volume fraction, on the resistivity changes of the nanocomposite is investigated. Results demonstrate that nearly 20% more tunneling distance is allowed for percolation of nanocomposite with 30% larger aspect ratio GNPs. Findings reveal that as the volume fraction of GNPs increases, the critical distance for the electrical percolation decreases significantly.

Matching vs. Mismatching Signals of External Human-Machine Interface and Vehicle Kinematics: An Examination of Pedestrian Crossing Behavior and Trust, Safety, and Affective Ratings in Interactions with Differently Sized Automated Vehicles

Highly automated vehicles (HAVs) will soon be introduced into mixed urban traffic. Pedestrians might have an idea of HAVs. Nevertheless, they probably have never interacted with them before. Moreover, pedestrians will not be able to communicate with HAVs like they are used to with manual vehicles. External human–machine interfaces (eHMIs) are possible design solutions for HAVs to ensure safe interaction with other road users. Light-based eHMIs positively affected pedestrians’ trust ratings, perceived safety, and willingness to cross. However, previous studies often neglected the effect of vehicle size, although larger-sized HAVs could be potentially perceived as the more significant threat. Additionally, the relationship between vehicle kinematics and eHMIs for differently sized HAVs remains an underexplored research topic. This study investigated the effects of vehicle size (small vs. large), eHMI state (dynamic eHMI vs. static eHMI vs. no eHMI), and vehicle kinematics (yielding vs. non-yielding) on pedestrian crossing behavior and their subjective assessment. In virtual reality, we created a shared space traffic scenario, in which the eHMI and vehicle kinematics matched or did not match. For yielding conditions, the results showed that participants felt more aroused with larger HAVs than with smaller HAVs. Moreover, pedestrians initiated their crossing significantly earlier when both vehicle sizes had a dynamic eHMI compared to a static eHMI vs. no eHMI. Additionally, pedestrians evaluated a dynamic eHMI with higher trust ratings, higher perceived safety, and more positive affective reactions. The results manifested that the use of dynamic eHMIs can effectively enhance pedestrian-vehicle communication with a large and a small HAV. For non-matching conditions, the participants tended to rely on the vehicle kinematics for both vehicle sizes. Overall, the study highlighted the potential of eHMIs for pedestrian interactions with HAVs of varying sizes when they are well-coordinated with the vehicle kinematics, aiming to enhance safety and efficiency in mixed-traffic environments.

Research on similarity bias in dual objective visual search based on nuclear power human-machine interface icons

Using icons from nuclear power interface as the research object, this study explored how icon similarity affected the performance of dual objective visual search. Firstly, the process of generating similarity bias was described from the perspective of human cognitive processing. The feature attributes of nuclear power icons were extracted, and then associated and mapped with similarity bias attributes. Secondly, a total of 16 instruction icons, device icons, and component icons were selected to propose icon coding logic for different similarity dimensions, and similarity experimental materials were designed. Finally, a dual objective search experiment with a 4 × 4 matrix was conducted to explore the impact of graph similarity on search performance and to determine the priority of perceptual similarity, semantic similarity, and memory similarity. High-level (H) similarity between the two targets resulted in superior visual search performance (p = 0.01 between the response times of similarity high and medium/low). Improving experiential familiarity enhanced search performance in cases of low-level (L) (p = 0.021) and medium-level (M) (p ≤ 0.009) icon similarity, but had no significant impact on search performance in cases of high-level (H) similarity (p ≥ 0.269). Compared to semantic similarity, enhancing perceptual similarity was more likely to improve search performance(p = 0.024).

Control and monitor bottle filling, capping and labelling machine using a programmable logic controller and human machine interface

Bottle filling, closing, and labellingmachine control and monitoring tool using Programmable Logic Controller (PLC) and Human-Machine Interface (HMI). This tool is designed to optimize the production process in the bottled beverage industry,with dimensions of 200 cm × 70 cm × 100 cm. The conveyor transports bottles through the filling, capping, and labeling units. In each unit, sensors are installed to detect the presence of bottles. Bottle liquid volume is determined through time settings inputted through the HMI,and the filling is performed by a water pump. After filling, bottles are capped and tightened mechanically using an air impact system. Labeling is handled by two DC motors which unroll the label and rotate the bottle, ensuringthe label adheres evenlyto all sides of the bottle. The number of bottles produced is calculated using a capacitive proximity sensor and displayed on the HMI. Testing results show that the system effectivelycontrols and monitors the bottle filling, capping, and labeling process viaPLC and HMI. Based on the test results on the bottle filling unit, the system can fill water into the bottle with a maximum percentage error of 0.5% on a 1000 ml volume bottle. In the capping unit, the success rate achieved is 70%, while the success rate in the bottle labeling unit is 80%.

Systematic review of cognitive impairment in drivers through mental workload using physiological measures of heart rate variability.

The intricate interplay between driver cognitive dysfunction, mental workload (MWL), and heart rate variability (HRV) provides a captivating avenue for investigation within the domain of transportation safety studies. This article provides a systematic review and examines cognitive hindrance stemming from mental workload and heart rate variability. It scrutinizes the mental workload experienced by drivers by leveraging data gleaned from prior studies that employed heart rate monitoring systems and eye tracking technology, thereby illuminating the correlation between cognitive impairment, mental workload, and physiological indicators such as heart rate and ocular movements. The investigation is grounded in the premise that the mental workload of drivers can be assessed through physiological cues, such as heart rate and eye movements. The study discovered that HRV and infrared (IR) measurements played a crucial role in evaluating fatigue and workload for skilled drivers. However, the study overlooked potential factors contributing to cognitive impairment in drivers and could benefit from incorporating alternative indicators of cognitive workload for deeper insights. Furthermore, investigated driving simulators demonstrated that an eco-safe driving Human-Machine Interface (HMI) significantly promoted safe driving behaviors without imposing excessive mental and visual workload on drivers. Recommendations were made for future studies to consider additional indicators of cognitive workload, such as subjective assessments or task performance metrics, for a more comprehensive understanding.

Otomatisasi Perhitungan OEE pada Mesin Filling ILAPAK 50g Menggunakan PLC dan HMI

This study addressed the limitations in calculating Overall Equipment Effectiveness (OEE) manually, which frequently led to inaccuracies and inefficiencies in industrial processes. The purpose of this research was to design an automated OEE calculation system for the ILAPAK 50g filling machine to improve data accuracy and reduce dependency on manual input. A Design Science Research (DSR) methodology was employed, involving stages of system analysis, design, implementation, and evaluation. The system utilized key components, including an Omron CP2E PLC, proximity sensors, relays, and a Human-Machine Interface (HMI) for real-time data acquisition. The findings indicated that the automated system significantly improved data accuracy, achieving reliable, continuous OEE calculations across different machine shifts. The system successfully replaced manual logbook methods, reducing human estimation errors, and displayed OEE data locally on the HMI, making it accessible for operational decisions. Furthermore, the implemented automation reduced overall downtime, enhanced machine availability, and minimized errors associated with manual record-keeping. This research has practical implications for industries aiming to optimize production efficiency. The automated OEE system demonstrated potential for scalability to other machines in manufacturing settings, highlighting its adaptability and utility in modern industrial environments. Future research could explore integration with cloud-based databases to enhance data storage and accessibility, providing a foundation for advanced predictive maintenance and performance monitoring across multiple machines.

Control of Multi-Robot Arms in Object Retrieval Based on Human-Machine Interface

Multi-robot arm control is the result of the development of advanced robotics technology. With the advancement in the field of human-machine interface, controlling robot arms has become more efficient and can be done intuitively by humans. In this study, we designed three robot arms of 4-DoF, each of which is controlled by a computer in a visual program interface. This research aims to develop a human-machine interface-based multi-robot arm control system that allows humans to interact with the robots directly. The movement method of each robot uses Trajectory Planning, which works when the operator gives motion commands through the interface display. Multi-robot communication with a computer using USB hub serial format. The computer is the master, and each Arduino on the robot is the enslaved. Three robot arms have been tested and controlled by the HMI computer, and all of them move according to the commands of the operator. The time required by each robot to complete its mission is ± 10 seconds. The results of this study are expected to open new opportunities in the application of robotics in various fields, such as the manufacturing industry, health services, and transportation.

A New Iterative Algorithm for Magnetic Motion Tracking.

Motion analysis is of great interest to a variety of applications, such as virtual and augmented reality and medical diagnostics. Hand movement tracking systems, in particular, are used as a human-machine interface. In most cases, these systems are based on optical or acceleration/angular speed sensors. These technologies are already well researched and used in commercial systems. In special applications, it can be advantageous to use magnetic sensors to supplement an existing system or even replace the existing sensors. The core of a motion tracking system is a localization unit. The relatively complex localization algorithms present a problem in magnetic systems, leading to a relatively large computational complexity. In this paper, a new approach for pose estimation of a kinematic chain is presented. The new algorithm is based on spatially rotating magnetic dipole sources. A spatial feature is extracted from the sensor signal, the dipole direction in which the maximum magnitude value is detected at the sensor. This is introduced as the "maximum vector". A relationship between this feature, the location vector (pointing from the magnetic source to the sensor position) and the sensor orientation is derived and subsequently exploited. By modelling the hand as a kinematic chain, the posture of the chain can be described in two ways: the knowledge about the magnetic correlations and the structure of the kinematic chain. Both are bundled in an iterative algorithm with very low complexity. The algorithm was implemented in a real-time framework and evaluated in a simulation and first laboratory tests. In tests without movement, it could be shown that there was no significant deviation between the simulated and estimated poses. In tests with periodic movements, an error in the range of 1° was found. Of particular interest here is the required computing power. This was evaluated in terms of the required computing operations and the required computing time. Initial analyses have shown that a computing time of 3 μs per joint is required on a personal computer. Lastly, the first laboratory tests basically prove the functionality of the proposed methodology.

Active constrained motion control for a robot-assisted endoscope manipulator in pediatric minimal access surgery.

Robot-assisted laparoscopic surgery has three main system requirements: safety, simplicity, and intuitiveness. However, accidental movement of the endoscope due to body fatigue and misunderstanding of the verbal orders between the surgeon and assistant will contribute to highly unexpected tool-tissue interactions, particularly in pediatric minimal access surgery with restricted working space. This study introduces a compact, lightweight endoscope manipulator with a mechanical remote-center-motion function. Using a custom-designed human-machine interface, the surgeon can intuitively control the movement of the endoscope manipulator over their view. In addition, an active constrained motion control algorithm is proposed to generate a forbidden-region constraint for avoiding collisions between the endoscope tip and surrounding organs in a pediatric abdominal cavity with restricted space. Simulations and experiments demonstrate the performance of the proposed compact endoscope manipulator and the active constrained surface tracking control scheme.

Senescent Syncytiotrophoblast Secretion During Early Onset Preeclampsia.

Preeclampsia is a severe hypertensive disorder in pregnancy that causes preterm delivery, maternal and fetal morbidity, mortality, and life-long sequelae. Understanding the pathogenesis of preeclampsia is a critical first step toward protecting mother and child from this syndrome and increased risk of cardiovascular disease later in life. However, effective early predictive tests and therapies for preeclampsia are scarce. To identify novel markers and signaling pathways for early onset preeclampsia, we profiled human maternal-fetal interface units (fetal villi and maternal decidua) from early onset preeclampsia and healthy controls using single-nucleus RNA sequencing combined with spatial transcriptomics. The placental syncytiotrophoblast is in direct contact with maternal blood and forms the barrier between fetal and maternal circulation. We identified different transcriptomic states of the endocrine syncytiotrophoblast nuclei with patterns of dysregulation associated with a senescence-associated secretory phenotype and a spatial dysregulation of senescence in the placental trophoblast layer. Elevated senescence markers were validated in placental tissues of clinical multicenter cohorts. Importantly, several secreted senescence-associated secretory phenotype factors were elevated in maternal blood already in the first trimester. We verified the secreted senescence markers, PAI-1 (plasminogen activator inhibitor 1) and activin A, as identified in our single-nucleus RNA sequencing model as predictive markers before clinical preeclampsia diagnosis. This indicates that increased syncytiotrophoblast senescence appears weeks before clinical manifestation of early onset preeclampsia, suggesting that the dysregulated preeclamptic placenta starts with higher cell maturation resulting in premature and increased senescence-associated secretory phenotype release. These senescence-associated secretory phenotype markers may serve as an additional early diagnostic tool for this syndrome.

Sodium Alginate/MXene-Based Flexible Humidity Sensors with High-Humidity Durability and Application Potentials in Breath Monitoring and Non-Contact Human-Machine Interfaces.

Flexible humidity sensors (FHSs) with fast response times and durability to high-humidity environments are highly desirable for practical applications. Herein, an FHS based on crosslinked sodium alginate (SA) and MXene was fabricated, which exhibited high sensitivity (impedance varied from 107 to 105 Ω between 10% and 90% RH), good selectivity, prompt response times (response/recover time of 4 s/11 s), high sensing linearity (R2 = 0.992) on a semi-logarithmic scale, relatively small hysteresis (~5% RH), good repeatability, and good resistance to highly humid environments (negligible changes in sensing properties after being placed in 98% RH over 24 h). It is proposed that the formation of the crosslinking structure of SA and the introduction of MXene with good conductivity and a high specific surface area contributed to the high performance of the composite FHS. Moreover, the FHS could promptly differentiate the respiration status, recognize speech, and measure fingertip movement, indicating potential in breath monitoring and non-contact human-machine interactions. This work provides guidance for developing advanced flexible sensors with a wide application scope in wearable electronics.

Otomatisasi Penyemprotan Polyester Menggunakan Kawasaki Cobot dengan Human Machine Interface (HMI) Berbasis Web

Brago Luchttechniek BV is a company specializing in the production of PIR (Polyisocyanurate) air ducts, facing challenges in maintaining accuracy and consistency in the polyester insulation spraying process on its products using Kawasaki collaborative robots (cobot) for automation. To address these challenges, this study designed a web-based Human Machine Interface (HMI) to automate the polyester insulation spraying process on PIR air ducts using the Kawasaki Cobot. A web-based HMI was chosen for its ease of development, flexibility, and integration capabilities with modern technology. This HMI uses the TCP/IP protocol to send real-time command signals to automation devices such as the Kawasaki Cobot and devices connected to the Programmable Logic Controller (PLC). Testing results demonstrated an improvement in the spraying process efficiency, reducing the time from 15–18 minutes conventionally to 5–7 minutes with the web-based HMI automation. Additionally, the web-based HMI successfully communicated with other devices connected to the robot, such as the conveyor belt and rotating table, via TCP/IP protocols, with each device having its own ID and IP address to facilitate linear message delivery, prevent collisions, and minimize errors. After eight successful automation tests on four different products, the system operated smoothly and systematically through the available HMI controls. The system was also evaluated by four respondents, with an overall score of 4.33 out of 5, indicating a high level of satisfaction.

Development of a humanized mouse model with functional human materno-fetal interface immunity.

Materno-fetal immunity possesses specialized characteristics to ensure pathogen clearance while maintaining tolerance to the semiallogeneic fetus. Most of our understanding on human materno-fetal immunity is based on conventional rodent models that may not precisely represent human immunological processes owing to the huge evolutionary divergence. Herein, we developed a pregnant human immune system (HIS) mouse model through busulfan preconditioning, which hosts multilineage human immune subset reconstitution at the materno-fetal interface. Human materno-fetal immunity exhibits a tolerogenic feature at the midgestation stage (embryonic day [E] 14.5), and human immune regulatory subsets were detected in the decidua. However, the immune system switches to an inflammatory profile at the late gestation stage (E19). A cell-cell interaction network contributing to the alternations in the human materno-fetal immune atmosphere was revealed based on single-cell RNA-Seq analysis, wherein human macrophages played crucial roles by secreting several immune regulatory mediators. Furthermore, depletion of Treg cells at E2.5 and E5.5 resulted in severe inflammation and fetus rejection. Collectively, these results demonstrate that the pregnant HIS mouse model permits the development of functional human materno-fetal immunity and offers a tool for human materno-fetal immunity investigation to facilitate drug discovery for reproductive disorders.

Development and validation of a machine learning‐based model of ischemic stroke risk in the Chinese elderly hypertensive population

AbstractMachine learning (ML) has made some significant contributions to stroke prevention, but the stability and accuracy of existing models for clinical applications are uncertain. This study develops and validates an interpretable ML model using metabolic and coagulation biomarkers to predict ischemic stroke in elderly hypertensive patients in Northwest China. The prediction model used 453 electronic medical records for the model building (80% as a training set and 20% as a test set) and 132 for external validation. The final seven key features (D‐dimer, cystatin C, homocysteine, hemoglobin A1c, prothrombin time, low‐density lipoprotein C, and triglyceride glucose‐body mass index) were selected by the advanced approach, elastic net, and classical wrapping approaches. The final model, eXtreme gradient boosting, was identified as having superior performance than the other 9 classifers (random forest, Gaussian process, multilayer perceptron, logistic regression, support vector machine, K‐nearest neighbor, decision tree, Gaussian naive bayes, and ensemble model), with area under the receiver‐operating characteristic curves of 0.97 and 0.94 for the test and external validation sets, respectively. The final model demonstrates excellent stability, accuracy, and clinical usefulness through various metrics and decision curve analysis. Additionally, an online human–machine interface application has been developed for clinical practice to help early identification and intervention for ischemic stroke in elderly hypertensive patients.