Intuitive Control Research Articles

Pattern-based Remote Switch for Domestic and Technological Operations

Abstract: This research explores the development of a remote-control system for domestic appliances using user-drawn patterns on a grid, aiming to provide an intuitive and accessible control method. The system is designed around a 3x3 grid of infrared (IR) LEDs and receivers that detect patterns drawn by the user with a finger or stylus. Each grid coordinate is mapped to a specific action, such as turning lights on or off or adjusting the speed of a fan. The detected patterns are transmitted to a NodeMCU ESP8266 microcontroller, which decodes the patterns and activates the corresponding appliance through Wi-Fi commands and relays. The innovative aspects of this system include its simple and intuitive interaction method, allowing users to define custom patterns for various actions, which enhances personalized control. The pattern-based interaction is particularly beneficial for individuals with limited dexterity or mobility, offering an accessible alternative to traditional remote controls. This project demonstrates the potential of combining hardware components, such as IR LEDs and receivers, with embedded C programming to create a user-friendly and efficient home automation solution. The proposed system not only simplifies the user experience but also provides a flexible and customizable interface for controlling domestic appliances.

A low-cost prosthetic arm with real-time haptic feedback and gesture control for enhanced user autonomy

In daily life, manipulating and feeling objects is essential for autonomy and engagement in various activities. However, individuals relying on prosthetic limbs often encounter barriers such as high costs, ineffective feedback, and limited design adaptability. Unlike conventional prosthetic limbs, which frequently struggle to integrate seamlessly and provide intuitive control, the robotic arm outlined in this study represents a leap in functionality and user experience. At the core of its design lies a specialized glove outfitted with a calibrated array of flex-sensing resistors capable of precisely detecting and interpreting the wearer's movements. These sensors input to the robotic arm, translating the user's gestures into motions that follow human hands. The fingertips are equipped with force-sensing resistors to discern the magnitude of force applied during gripping actions. This real-time sensory feedback is conveyed to the user through a haptic system integrated into the glove. By modulating the intensity and pattern of vibrations, users can gauge their grip strength and manipulate their grasp of the object accordingly. Comprehensive experiments were conducted across various scenarios to assess its performance, agility, strength, and sensory accuracy. Through refinement, the author aims to optimize functionality and reliability, enhancing practicality for users. This project has broader implications for the prosthetics community. By utilizing the scalability and cost-effectiveness of this model, the author envisions tailored solutions that empower individuals with limb differences to live more independently.

Cooperative augmented assembly (CAA): augmented reality for on-site cooperative robotic fabrication

Recent years have witnessed significant advances in computational design and robotic fabrication for large-scale manufacturing. Although these advances have enhanced the speed, precision, and reproducibility of digital fabrication processes, they often lack adaptability and fail to integrate manual actions in a digital model. Addressing this challenge, the present study introduces cooperative augmented assembly (CAA), a phone-based mobile Augmented Reality (AR) application that facilitates cooperative assembly of complex timber structures between humans and robots. CAA enables augmented manual assembly, intuitive robot control and supervision, and task sharing between humans and robots, creating an adaptive digital fabrication process. To allocate tasks to manual or robotic actions, the mobile AR application allows multiple users to access a shared digital workspace. This is achieved through a flexible communication system that allows numerous users and robots to cooperate seamlessly. By harnessing a cloud-based augmented reality system in combination with an adaptive digital model, CAA aims to better incorporate human actions in robotic fabrication setups, facilitating human–machine cooperation workflows and establishing a highly intuitive, adaptable digital fabrication process within the Architecture, Engineering, and Construction sector.

Electrophysiological Signal Validation of Regenerative Peripheral Nerve Interface at Nerve Ending: A Preliminary Rat Model Experiment.

As the number of extremity amputations continues to rise, so does the demand for prosthetics. Emphasizing the importance of a nerve interface that effectively amplifies and transmits physiological signals through peripheral nerve surgery is crucial for achieving intuitive control. The regenerative peripheral nerve interface (RPNI) is recognized for its potential to provide this technical support. Through animal experiment, we aimed to confirm the actual occurrence of signal amplification. Rats were divided into three experimental groups: control, common peroneal nerve transection, and RPNI. Nerve surgeries were performed for each group, and electromyography (EMG) and nerve conduction studies (NCS) were conducted at the initial surgery, as well as at 2, 4, and 8 weeks postoperatively. All implemented RPNIs exhibited viability and displayed adequate vascularity with the proper color. Clear differences in latency and amplitude were observed before and after 8 weeks of surgery in all groups (p < 0.05). Notably, the RPNI group demonstrated a significantly increased amplitude compared with the control group after 8 weeks (p = 0.031). Latency increased in all groups 8 weeks after surgery. The RPNI group exhibited relatively clear signs of denervation with abnormal spontaneous activities (ASAs) during EMG. This study is one of few preclinical studies that demonstrate the electrophysiological effects of RPNI and validate the neural signals. It serves as a foundational step for future research in human-machine interaction and nerve interfaces.

Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces.

Intracortical brain-computer interfaces (iBCIs) enable people with tetraplegia to gain intuitive cursor control from movement intentions. To translate to practical use, iBCIs should provide reliable performance for extended periods of time. However, performance begins to degrade as the relationship between kinematic intention and recorded neural activity shifts compared to when the decoder was initially trained. In addition to developing decoders to better handle long-term instability, identifying when to recalibrate will also optimize performance. We propose a method, "MINDFUL", to measure instabilities in neural data for useful long-term iBCI,without needing labels of user intentions. Longitudinal data were analyzed from two BrainGate2 participants with tetraplegia as they used fixed decoders to control a computer cursor spanning 142 days and 28 days, respectively. We demonstrate a measure of instability that correlates with changes in closed-loop cursor performance solely based on the recorded neural activity (Pearson r = 0.93 and 0.72, respectively). This result suggests a strategy to infer online iBCI performance from neural data alone and to determine when recalibration should take place for practical long-term use.

Modeling the Dynamics of Prosthetic Fingers for the Development of Predictive Control Algorithms

In the field of biomechanical modeling, the development of a prosthetic hand with dexterity comparable to the human hand is a multidisciplinary challenge involving complex mechatronic systems, intuitive control schemes, and effective body interfaces. Most current commercial prostheses offer limited functionality, typically only one or two degrees of freedom (DoF), resulting in reduced user adoption due to discomfort and lack of functionality. This research aims to design a computationally efficient low-level control algorithm for prosthetic hand fingers to be able to (a) accurately manage finger positions, (b) anticipate future information, and (c) minimize power consumption. The methodology employed is known as model-based predictive control (MBPC) and starts with the application of linear identification techniques to model the system dynamics. Then, the identified model is used to implement a generalized predictive control (GPC) algorithm, which optimizes the control effort and system performance. A test bench is used for experimental validation, and the results demonstrate that the proposed control scheme significantly improves the prosthesis’ dexterity and energy efficiency, enhancing its potential for daily use by people with hand loss.

An intuitive guidewire control mechanism for robotic intervention.

Teleoperated Interventional Robotic systems (TIRs) are developed to reduce radiation exposure and physical stress of the physicians and enhance device manipulation accuracy and stability. Nevertheless, TIRs are not widely adopted, partly due to the lack of intuitive control interfaces. Current TIR interfaces like joysticks, keyboards, and touchscreens differ significantly from traditional manual techniques, resulting in a shallow, longer learning curve. To this end, this research introduces a novel control mechanism for intuitive operation and seamless adoption of TIRs. An off-the-shelf medical torque device augmented with a micro-electromagnetic tracker was proposed as the control interface to preserve the tactile sensation and muscle memory integral to interventionalists' proficiency. The control inputs to drive the TIR were extracted via real-time motion mapping of the interface. To verify the efficacy of the proposed control mechanism to accurately operate the TIR, evaluation experiments using industrial grade encoders were conducted. A mean tracking error of 0.32 ± 0.12mm in linear and 0.54 ± 0.07° in angular direction were achieved. The time lag in tracking was found to be 125ms on average using pade approximation. Ergonomically, the developed control interface is 3.5mm diametrically larger, and 4.5g. heavier compared to traditional torque devices. With uncanny resemblance to traditional torque devices while maintaining results comparable to state-of-the-art commercially available TIRs, this research successfully provides an intuitive control interface for potential wider clinical adoption of robot-assisted interventions.

Analysis of electrode locations on limb condition effect for myoelectric pattern recognition

BackgroundGesture recognition using surface electromyography (sEMG) has garnered significant attention due to its potential for intuitive and natural control in wearable human–machine interfaces. However, ensuring robustness remains essential and is currently the primary challenge for practical applications.MethodsThis study investigates the impact of limb conditions and analyzes the influence of electrode placement. Both static and dynamic limb conditions were examined using electrodes positioned on the wrist, elbow, and the midpoint between them. Initially, we compared classification performance across various training conditions at these three electrode locations. Subsequently, a feature space analysis was conducted to quantify the effects of limb conditions. Finally, strategies for group training and feature selection were explored to mitigate these effects.ResultsThe results indicate that with the state-of-the-art method, classification performance at the wrist was comparable to that at the middle position, both of which outperformed the elbow, consistent with the findings from the feature space analysis. In inter-condition classification, training under dynamic limb conditions yielded better results than training under static conditions, especially at the positions covered by dynamic training. Additionally, fast and slow movement speeds produced similar performance outcomes. To mitigate the effects of limb conditions, adding more training conditions reduced classification errors; however, this reduction plateaued after four conditions, resulting in classification errors of 22.72%, 22.65%, and 26.58% for the wrist, middle, and elbow, respectively. Feature selection further improved classification performance, reducing errors to 19.98%, 19.75%, and 27.14% at the respective electrode locations, using three optimal features derived from single-condition training.ConclusionsThe study demonstrated that the impact of limb conditions was mitigated when electrodes were placed near the wrist. Dynamic limb condition training, combined with feature optimization, proved to be an effective strategy for reducing this effect. This work contributes to enhancing the robustness of myoelectric-controlled interfaces, thereby advancing the development of wearable intelligent devices.

Research on enhancing human-machine interaction in medical exoskeleton devices through the integration of artificial intelligence

This paper explores the integration of Artificial Intelligence (AI) to enhance human-machine interaction in medical exoskeleton devices. AI technologies such as machine learning, natural language processing (NLP), and predictive analytics can significantly improve the efficiency and comfort of these devices. Machine learning algorithms analyze sensor data in real-time, optimizing control strategies to adapt to user movements. NLP enables intuitive control through voice commands, reducing the cognitive and physical burden on users. Predictive analytics anticipates user needs, enhancing responsiveness and reducing the risk of errors. Case studies of AI-integrated exoskeletons, like the EksoGT and ReWalk, demonstrate improved rehabilitation outcomes and user satisfaction. Despite challenges such as data privacy and the need for significant investment, the benefits of AI integration are substantial. This paper provides insights into the potential of AI to transform medical exoskeletons, offering new levels of independence and quality of life for individuals with mobility impairments. Future research should focus on developing advanced AI algorithms and exploring new applications to further enhance user experience and device performance.

VocalVision: Smart Wheelchair Maintenance with Pressure Sensors and Machine Learning

The Vocal Vision system introduces an innovative approach to enhancing electric wheelchair maintenance and control. It utilizes a network of sensors embedded within the wheelchair's wheels to gather real-time data on tire pressure, temperature, tread wear, and alignment. This data is wirelessly transmitted to a central control unit. Advanced algorithms, incorporating machine learning and predictive analytics, analyze the data to detect irregularities and predict maintenance needs. Users can control direction, speed, and perform complex maneuvers with precision using voice commands and eye gestures. The wheelchair integrates OpenCV for eye gesture recognition and Google Speech Recognition API for voice commands, enabling intuitive control methods. This proposed method introduces a new assistive technology for individuals with disabilities, leveraging cutting-edge technologies.

Development of an innovative microvehicle for urban mobility

This paper presents the development of a novel microvehicle concept under the H2020 LEONARDO project, which targets the limitations of current micromobility solutions. By combining the features of a monowheel and a kick scooter, the vehicle is designed to enhance safety, comfort, and convenience for urban transport. It employs an innovative control mechanism, where pushing the steering column accelerates and pulling it decelerates, thereby improving user experience. The design integrates advanced fast-charging energy storage with a lightweight, compact build, facilitating easy maneuverability and storage, making it suitable for urban environments. The development process includes extensive re-design efforts; in-house testing; and detailed structural, electrical, and regulatory analyses, ensuring compliance with existing standards. A large-scale demonstration in a real urban setting validates the practicality and effectiveness of vehicles. This microvehicle emphasizes ease of use, merging the best aspects of a kick scooter and a monowheel to overcome challenges related to range, safety, and integration with public transportation. A key innovation is the potential inclusion of a battery-sharing system that enhances versatility and user appeal. By focusing on a seamless blend of intuitive control, compact design, and efficient energy use, this vehicle addresses the significant limitations of the current micromobility solutions. This paper highlights the potential of vehicles to significantly improve urban transportation by offering a practical, environmentally friendly, and user-friendly alternative that enhances the efficiency and attractiveness of urban mobility options.

Recent progress on smart lower prosthetic limbs: a comprehensive review on using EEG and fNIRS devices in rehabilitation.

The global rise in lower limb amputation cases necessitates advancements in prosthetic limb technology to enhance the quality of life for affected patients. This review paper explores recent advancements in the integration of EEG and fNIRS modalities for smart lower prosthetic limbs for rehabilitation applications. The paper synthesizes current research progress, focusing on the synergy between brain-computer interfaces and neuroimaging technologies to enhance the functionality and user experience of lower limb prosthetics. The review discusses the potential of EEG and fNIRS in decoding neural signals, enabling more intuitive and responsive control of prosthetic devices. Additionally, the paper highlights the challenges, innovations, and prospects associated with the incorporation of these neurotechnologies in the field of rehabilitation. The insights provided in this review contribute to a deeper understanding of the evolving landscape of smart lower prosthetic limbs and pave the way for more effective and user-friendly solutions in the realm of neurorehabilitation.

Transforming the Anthropomorphic Passive Free-Flow Foot Prosthesis Into a Powered Foot Prosthesis With Intuitive Control and Sensation (Bionic FFF).

Approximately 89% of all service members with amputations do not return to duty. Restoring intuitive neural control with somatosensory sensation is a key to improving the safety and efficacy of prosthetic locomotion. However, natural somatosensory feedback from lower-limb prostheses has not yet been incorporated into any commercial prostheses. We developed a neuroprosthesis with intuitive bidirectional control and somatosensation and evoking phase-dependent locomotor reflexes, we aspire to significantly improve the prosthetic rehabilitation and long-term functional outcomes of U.S. amputees. We implanted the skin and bone integrated pylon with peripheral neural interface pylon into the cat distal tibia, electromyographic electrodes into the residual gastrocnemius muscle, and nerve cuff electrodes on the distal tibial and sciatic nerves. Results. The bidirectional neural interface that was developed was integrated into the existing passive Free-Flow Foot and Ankle prosthesis, WillowWood, Mount Sterling, OH. The Free-Flow Foot was chosen because it had the highest Index of Anthropomorphicity among lower-limb prostheses and was the first anthropomorphic prosthesis brought to market. Conclusion. The cats walked on a treadmill with no cutaneous feedback from the foot in the control condition and with their residual distal tibial nerve stimulated during the stance phase of walking.

User Evaluation of a Shared Robot Control System Combining BCI and Eye Tracking in a Portable Augmented Reality User Interface.

This study evaluates an innovative control approach to assistive robotics by integrating brain-computer interface (BCI) technology and eye tracking into a shared control system for a mobile augmented reality user interface. Aimed at enhancing the autonomy of individuals with physical disabilities, particularly those with impaired motor function due to conditions such as stroke, the system utilizes BCI to interpret user intentions from electroencephalography signals and eye tracking to identify the object of focus, thus refining control commands. This integration seeks to create a more intuitive and responsive assistive robot control strategy. The real-world usability was evaluated, demonstrating significant potential to improve autonomy for individuals with severe motor impairments. The control system was compared with an eye-tracking-based alternative to identify areas needing improvement. Although BCI achieved an acceptable success rate of 0.83 in the final phase, eye tracking was more effective with a perfect success rate and consistently lower completion times (p<0.001). The user experience responses favored eye tracking in 11 out of 26 questions, with no significant differences in the remaining questions, and subjective fatigue was higher with BCI use (p=0.04). While BCI performance lagged behind eye tracking, the user evaluation supports the validity of our control strategy, showing that it could be deployed in real-world conditions and suggesting a pathway for further advancements.

Development of a human machine interface for robotically assisted surgery optimized for laparoscopic workflows.

In robotic-assisted surgery (RAS), the input device is the primary site for the flow of information between the user and the robot. Most RAS systems remove the surgeon's console from the sterile surgical site. Beneficial for performing lengthy procedures with complex systems, this ultimately lacks the flexibility that comes with the surgeon being able to remain at the sterile site. A prototype of an input device for RAS is constructed. The focus lies on intuitive control for surgeons and a seamless integration into the surgical workflow within the sterile environment. The kinematic design is translated from the kinematics of laparoscopic surgery. The input device uses three degrees of freedom from a flexible instrument as input. The prototype's performance is compared to that of a commercially available device in an evaluation. Metrics are used to evaluate the surgeons' performance with the respective input device in a virtual environment implemented for the evaluation. The evaluation of the two input devices shows statistically significant differences in the performance metrics. With the proposed prototype, the surgeons perform the tasks faster, more precisely, and with fewer errors. The prototype is an efficient and intuitive input device for surgeons with laparoscopic experience. The placement in the sterile working area allows for seamless integration into the surgical workflow and can potentially enable new robotic approaches.

Multiple Self-Powered Sensor-Integrated Mobile Manipulator for Intelligent Environment Detection.

A multiple self-powered sensor-integrated mobile manipulator (MSIMM) system was proposed to address challenges in existing exploration devices, such as the need for a constant energy supply, limited variety of sensed information, and difficult human-computer interfaces. The MSIMM system integrates triboelectric nanogenerator (TENG)-based self-powered sensors, a bionic manipulator, and wireless gesture control, enhancing sensor data usability through machine learning. Specifically, the system includes a tracked vehicle platform carrying the manipulator and electronics, including a storage battery and a microcontroller unit (MCU). An integrated sensor glove and terminal application (APP) enable intuitive manipulator control, improving human-computer interaction. The system responds to and analyzes various environmental stimuli, including the droplet and fall height, temperature, pressure, material type, angles, angular velocity direction, and acceleration amplitude and direction. The manipulator, fabricated using 3D printing technology, integrates multiple sensors that generate electrical signals through the triboelectric effect of mechanical motion. These signals are classified using convolutional neural networks for accurate environmental monitoring. Our database shows signal recognition and classification accuracy exceeding 94%, with specific accuracies of 100% for pressure sensors, 99.55% for angle sensors, and 98.66, 95.91, 96.27, and 94.13% for material, droplet, temperature, and acceleration sensors, respectively.

EEG Analyses of visual cue effects on executed movements

BackgroundIn electroencephalographic (EEG) or electrocorticographic (ECoG) experiments, visual cues are commonly used for timing synchronization but may inadvertently induce neural activity and cognitive processing, posing challenges when decoding self-initiated tasks. New methodTo address this concern, we introduced four new visual cues (Fade, Rotation, Reference, and Star) and investigated their impact on brain signals. Our objective was to identify a cue that minimizes its influence on brain activity, facilitating cue-effect free classifier training for asynchronous applications, particularly aiding individuals with severe paralysis. Results22 able-bodied, right-handed participants aged 18–30 performed hand movements upon presentation of the visual cues. Analysis of time-variability between movement onset and cue-aligned data, grand average MRCP, and classification outcomes revealed significant differences among cues. Rotation and Reference cue exhibited favorable results in minimizing temporal variability, maintaining MRCP patterns, and achieving comparable accuracy to self-paced signals in classification. Comparison with existing methodsOur study contrasts with traditional cue-based paradigms by introducing novel visual cues designed to mitigate unintended neural activity. We demonstrate the effectiveness of Rotation and Reference cue in eliciting consistent and accurate MRCPs during motor tasks, surpassing previous methods in achieving precise timing and high discriminability for classifier training. ConclusionsPrecision in cue timing is crucial for training classifiers, where both Rotation and Reference cue demonstrate minimal variability and high discriminability, highlighting their potential for accurate classifications in online scenarios. These findings offer promising avenues for refining brain-computer interface systems, particularly for individuals with motor impairments, by enabling more reliable and intuitive control mechanisms.

SceneDirector: Interactive Scene Synthesis by Simultaneously Editing Multiple Objects in Real-Time.

Intelligent tools for creating synthetic scenes have been developed significantly in recent years. Existing techniques on interactive scene synthesis only incorporate a single object at every interaction, i.e., crafting a scene through a sequence of single-object insertions with user preferences. These techniques suggest objects by considering existent objects in the scene instead of fully picturing the eventual result, which is inherently problematic since the sets of objects to be inserted are seldom fixed during interactive processes. In this article, we introduce SceneDirector, a novel interactive scene synthesis tool to help users quickly picture various potential synthesis results by simultaneously editing groups of objects. Specifically, groups of objects are rearranged in real-time with respect to a position of an object specified by a mouse cursor or gesture, i.e., a movement of a single object would trigger the rearrangement of the existing object group, the insertions of potentially appropriate objects, and the removal of redundant objects. To achieve this, we first propose an idea of coherent group set which expresses various concepts of layout strategies. Subsequently, we present layout attributes, where users can adjust how objects are arranged by tuning the weights of the attributes. Thus, our method gives users intuitive control of both how to arrange groups of objects and where to place them. Through extensive experiments and two applications, we demonstrate the potentiality of our framework and how it enables concurrently effective and efficient interactions of editing groups of objects.

A novel affordable user interface for robotic surgery training: design, development and usability study.

The use of robotic systems in the surgical domain has become groundbreaking for patients and surgeons in the last decades. While the annual number of robotic surgical procedures continues to increase rapidly, it is essential to provide the surgeon with innovative training courses along with the standard specialization path. To this end, simulators play a fundamental role. Currently, the high cost of the leading VR simulators limits their accessibility to educational institutions. The challenge lies in balancing high-fidelity simulation with cost-effectiveness; however, few cost-effective options exist for robotic surgery training. This paper proposes the design, development and user-centered usability study of an affordable user interface to control a surgical robot simulator. It consists of a cart equipped with two haptic interfaces, a VR visor and two pedals. The simulations were created using Unity, which offers versatility for expanding the simulator to more complex scenes. An intuitive teleoperation control of the simulated robotic instruments is achieved through a high-level control strategy. Its affordability and resemblance to real surgeon consoles make it ideal for implementing robotic surgery training programs in medical schools, enhancing accessibility to a broader audience. This is demonstrated by the results of an usability study involving expert surgeons who use surgical robots regularly, expert surgeons without robotic surgery experience, and a control group. The results of the study, which was based on a traditional Peg-board exercise and Camera Control task, demonstrate the simulator's high usability and intuitive control across diverse user groups, including those with limited experience. This offers evidence that this affordable system is a promising solution for expanding robotic surgery training.

Peer-to-Peer Ultra-Wideband Localization for Hands-Free Control of a Human-Guided Smart Stroller.

We propose a hands-free control system for a human-guided smart stroller. The proposed method uses real-time peer-to-peer localization technology of the human and stroller to realize an intuitive hands-free control system based on the relative position between the human and the stroller. The control method is also based on functional and mechanical safety to ensure the safety of the stroller's occupant (child) and the pilot (parent) during locomotion. In this paper, first, we present a preliminary investigation of the humans' preference for the relative position in the context of hands-free guided strollers. Then, we present the control method and a prototype implemented with an electric wheelchair and UWB sensors for localization. We present an experimental evaluation of the proposed method with 14 persons walking with the developed prototype to investigate the usability and soundness of the proposed method compared to a remote joystick and manual operation. The evaluation experiments were conducted in an indoor environment and revealed that the proposed method matches the performance of joystick control but does not perform as well as manual operation. Notably, for female participants, the proposed method significantly surpasses joystick performance and achieves parity with manual operation, which shows its efficacy and potential for a smart stroller. Also, the results revealed that the proposed method significantly decreased the user's physical load compared to the manual operation. We present discussions on the controllability, usability, task load, and safety features of the proposed method, and conclude this work with a summary assessment.