Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms.

As the population aging becomes more severe, the development of Wheelchair Mounted Robotic Arms (WMRA) has gained greater attention. There are remaining issues that haven't been properly tackled, such as human-robot interaction and real-time performance. In this paper, laser pointer is used to facilitate the interaction between human and the WMRA, and an improved target matching method is proposed for laser point detection in home environment. Firstly, the laser point's characteristics are amplified through the use of channel separation technique and reflective materials. Then, the laser point is separated from the image using background difference method. Finally, through ASUS Xtion, the distance between laser point and the centroid coordination of object is calculated, and then Kinova Jaco robotic arm is used for grasping. The experimental result shows that the algorithm can effectively detect laser point in the home environment, and as a human-robot interaction, the robotic arm successfully completes the task of demo grasp.

Purpose Despite the benefits of wheelchair-mounted robotic arms (WMRAs), occupational therapists are not yet widely involved in the recommendation or implementation of these assistive devices. The purpose of this study was to investigate and compare the current practices and perspectives of occupational therapists who had and had not recommended a WMRA on the recommendation, training, and implementation of WMRAs. Methods This was a descriptive cross-sectional study. An online survey was sent to Canadian, European, and American occupational therapists who had or had not worked with WMRAs. Respondents were asked close-ended questions about their experience, role, barriers, motivations, and future needs regarding WMRAs. We compared results between respondents who had and had not recommended WMRAs using descriptive statistics. Results Ninety-three North American and European occupational therapists completed the survey. Of those, 29 (31.2%) had recommended a WMRA, mostly the JACO robotic arm (n = 26, 89.7%) in rehabilitation centres (n = 18, 62.1%). Their perspectives on their role and barriers related to WMRAs were similar to those who had never recommended a WMRA. All respondents recognised the relevance of occupational therapists’ contribution, and most reported interest in WMRAs (n = 76, 81.7%). However, many barriers emerged, mainly related to limited funding (n = 49, 76.6%), lack of training and knowledge (n = 38, 59.4%), and resource constraints (n = 37, 54.4%). Future needs identified matched these barriers. Conclusion This survey provides novel insight into occupational therapists’ perspectives on WMRAs. It highlights that health professionals need to have easier access to funding, formal training, and resources to support their involvement with WMRAs. Implications for rehabilitation Most occupational therapists are interested in working with WMRAs, considering the potential of these devices to support individuals with upper extremity impairments in their daily activities. They also recognise their unique contribution to the assessment, recommendation, and implementation process among multidisciplinary teams. WMRA recommendation is relevant in various clinical settings and with a wide range of client populations. Nevertheless, it appears that occupational therapists working with adults, in rehabilitation centres or specialised clinics, may have more opportunities to get involved in this process and to attend formal training on this technology, as compared to other settings. Many barriers remain, impeding occupational therapists’ role in the recommendation and implementation of WMRAs. Addressing these barriers may increase the number of devices that are successfully adopted and utilised by individuals with upper extremity impairments. In particular, future research and health policies should focus on access to sufficient funding, formal training, and resources for occupational therapists relative to their role in recommending and implementing WMRAs.

Point cloud modeling using the homogeneous transformation for non-cooperative pose estimation

This paper focuses on kinematic analysis and evaluation of wheelchair mounted robotic arms (WMRA). It addresses the kinematics of the WMRA with respect to its ability to reach common positions while performing activities of daily living (ADL). A procedure is developed for the kinematic analysis and evaluation of a WMRA. In an effort to evaluate two commercial WMRAs, the procedure for kinematic analysis is applied to each manipulator. Design recommendations and insights with regard to each device are obtained and used to design a new WMRA to overcome the limitations of these devices. This method benefits the researchers by providing a standardized procedure for kinematic analysis of WMRAs that is capable of evaluating independent designs.

As the population aging becomes more and more serious, WMRA (Wheelchair Mounted Robotic Arms) has been rapidly developed. But it still has many problems in human-robot interaction. In this paper, laser pointer is used as an interactive method for wheelchair. First, the camera acquires the color image, then each channel of the image is carried on the weight process, causing the object with high brightness in the image to reduce. This operation eliminates environmental interference, and then color image is converted into HSV image. The background difference method and target matching method are used to detect the laser point in the home environment. In the experiment, the laser point in the home environment can be well detected, and the laser pointer is used as an interactive interface for the electric wheelchair, which enables the wheelchair to achieve the functions of forward, backward, left, right and so on.

Purpose The increasing prevalence of upper limb dysfunctions due to stroke, spinal cord injuries, and multiple sclerosis presents a critical challenge in assistive technology: designing robotic arms that are both energy‑efficient and capable of effectively performing activities of daily living (ADLs). This challenge is exacerbated by the need to ensure these devices are accessible for non‑expert users and can operate within the spatial constraints typical of everyday environments. Despite advancements in wheelchair‑mounted robotic arms (WMRAs), existing designs do not achieve an optimal balance—minimizing energy consumption and space while maximizing kinematic performance and workspace. Most robotic arms can perform a range of ADLs, but they do not account for outdoor environments where energy conservation is crucial. Furthermore, the need for WMRAs to be compact in idle configurations—essential for navigating through doors or between aisles—adds another layer of complexity to their design. This paper addresses these multifaceted design challenges by proposing a novel objective function to optimize the link lengths of WMRAs, aiming to reduce energy consumption without compromising the robots’ operational capabilities. Materials and Methods To achieve this optimization, the scatter search method was employed, incorporating considerations of collision and singularity avoidance while ensuring the arm remains compact when not in use. The proposed design was evaluated through simulations and experimental validation with both healthy subjects and individuals with lower limb dysfunctions. Results and Conclusions The optimized WMRA demonstrated significant improvements in energy efficiency and spatial adaptability while maintaining the required kinematic performance for ADLs. The validation process confirmed the practical applicability of the proposed design, highlighting its potential to enhance mobility and independence for individuals with upper limb impairments. This study contributes to the field of disability and rehabilitation by providing a structured approach to designing assistive robotic arms that better align with real‑world constraints and user needs.

Wheelchair Mounted Robotic Arms: Occupational Therapy Perceptions and Practices

This paper focuses on kinematic analysis, evaluation and design of wheelchair mounted robotic arms (WMRA). It addresses the kinematics of the WMRA with respect to its ability to reach common positions while performing activities of daily living (ADL). A procedure is developed for the kinematic analysis and evaluation of WMRAs. In an effort to evaluate two commercial WMRAs, the procedure for kinematic analysis is applied to each manipulator. Design recommendations and insights with regard to each device are obtained and used to design a new WMRA to overcome the limitations of these devices. This method benefits the researchers by providing a standardized procedure for kinematic analysis of WMRAs that is capable of evaluating independent designs

Electric wheelchair-mounted robotic arms can help patients with disabilities to perform their activities in daily living (ADL). Joysticks or keypads are commonly used as the operating interface of Wheelchair-mounted robotic arms. Under different scenarios, some patients with upper limb disabilities such as finger contracture cannot operate such interfaces smoothly. Recently, manual interfaces for different symptoms to operate the wheelchair-mounted robotic arms are being developed. However, the stop the wheelchairs in an appropriate position for the robotic arm grasping task is still not easy. To reduce the individual’s burden in operating wheelchair in narrow spaces and to ensure that the chair always stops within the working range of a robotic arm, we propose here an operating system for an electric wheelchair that can automatically drive itself to within the working range of a robotic arm by capturing the position of an AR marker via a chair-mounted camera. Meanwhile, the system includes an error correction model to correct the wheelchair’s moving error. Finally, we demonstrate the effectiveness of the proposed system by running the wheelchair and simulating the robotic arm through several courses.

In this paper, we present an object detection and pose estimation algorithm that runs on low-cost and limited-processing hardware. The algorithm relies on two main steps: in the detection phase, it employs a Convolutional Neural Network (CNN) – namely MobileNet SSD – to recognize and track objects of interest in the scene; while in the pose estimation phase, the algorithm uses stereo correspondences to 3D reconstruct the spatial coordinates of multiple ORB features in the bounding box of the recognized object. The final position of the object in space is computed using a weighted average of the same spatial coordinates of the stereo-corresponded key points, where the weights are exponentially proportional to the level of ORB stereo matching. This algorithm was deployed on embedded systems and it achieved comparable results with respect to state-of-art, GPU-dependent, deep-learning algorithms for object pose estimation. We tested our approach in a calibrated environment where the object displacement (in either X or Y directions) can be easily measured. We used objects from the Yale-CMU-Berkeley (YCB) dataset to compare our estimation results with a deep learning based method, but our approach is not limited to this object selection. In the end, we achieved an estimated error of less than 10 mm. These results demonstrate that our approach can be used in affordable embedded systems for assistive technology in tasks such as for control of robotic arms for pick-and-place.

Due to the important role that accurate pose estimation plays in the field of vision measurement, a careful tracing and analysis of errors introduced in all aspects is necessary. In this paper, we launch an in-depth and meticulous research to seek for and analyze factors causing pose errors. These factors contain camera intrinsic parameters, 3D world coordinates of feature points, corresponding 2D image coordinates and different pose estimation algorithms. They are usually ignored in practical applications, leading to inaccurate pose. We develop mathematical tools to construct error models to estimate the actual error occurring in the implementation of a pose estimation algorithm. Our central idea is to make only one variable different at a time by controlling various variables. Then the effect of that single factor can be determined. Afterwards our approach is applied to ample synthetic experiments. The relationship between the influencing factors and pose estimation error is established and provides reference and basis to control errors and achieve more accurate pose.KeywordsVision MeasurementError TracingError AnalysisPose Estimation

Six-dimensional (6D) pose estimation is an important branch in the field of robotics focused on enhancing the ability of robots to manipulate and grasp objects. The latest research trend in 6D pose estimation is to directly predict the positions of two-dimensional (2D) keypoints from a single red, green, and blue (RGB) image through convolutional neural networks (CNNs) and establish a corresponding relationship with the three-dimensional (3D) keypoints of the model. Then, the perspective-n-point (PnP) algorithm is used to recover the 6D pose parameters. Currently, two challenges are encountered in pose estimation based on an RGB image. On the one hand, an RGB image lacks depth information, and it is thus difficult to directly obtain the corresponding geometric object information. On the other hand, when depth information is available, it is difficult to efficiently fuse the features of the RGB image with the features of the corresponding depth image. In this paper, we propose a bidirectional depth residual fusion network with a depth prediction (DP) network to estimate the 6D poses of objects (BDR6D). The BDR6D network predicts the depth information of objects using an RGB image, converts the depth information into point cloud information, and performs feature extraction and representation together with the RGB information during the feature extraction and representation stages. Specifically, the RGB image is fed into the BDR6D network, the DP network predicts the depth information of the objects in the image, and the depth map and RGB image are input into a point cloud network (PCN) and CNN, respectively, for feature extraction and representation. We build the bidirectional depth residual (BDR) structure so that the CNN and PCN can share information during feature extraction and representation. This approach allows the two networks to use each other’s local and global information to improve feature extraction and representation. For the keypoint selection stage, we propose an effective 2D keypoint selection method that considers the appearance and geometric information of the object of interest. We evaluate the proposed method with three benchmark datasets and compare it with other 6D pose estimation algorithms. The experimental results show that our method outperforms the state-of-the-art approach. Finally, we deploy our proposed method in conjunction with the Universal Robots 5 manipulator (UR5) robot to grasp and manipulate objects. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The purpose of this paper is to solve the problem of 6D pose estimation for robot grasping. The existing RGB image-based pose estimation approach faces two challenges. On the one hand, a single RGB image lacks depth information, so that it is difficult to directly obtain the corresponding geometric object information. On the other hand, when depth information is available, it is difficult to efficiently fuse the features of the RGB image with the features of the corresponding depth image. To solve the above problems, a novel network that can predict the depth information of objects from an RGB image and fuse the depth information with the RGB information to estimate the 6D pose of objects is proposed. Furthermore, we propose an effective 2D keypoint selection method that considers the appearance and geometric information of objects of interest. We evaluate the proposed approach based on three benchmark datasets and the UR5 robot platform and verify that our method is effective.

Purpose This review examines wheelchair-mounted robotic arms (WMRAs) as an emerging assistive technology that enhances independence and quality of life for individuals with upper- and lower-limb disabilities. By enabling independent performance of activities of daily living (ADLs), WMRAs hold significant promise for disability and rehabilitation. The article aims to critically evaluate the state of the art in WMRA research and development, identifying persistent challenges and highlighting promising innovations. Materials and Methods The review systematically analyzes literature on WMRAs published between 2001 and 2025. The analysis emphasizes design specifications, degrees of freedom, actuation methods, control strategies, and performance evaluations. A comparative synthesis is conducted to assess how existing systems support ADL execution, while also integrating technical considerations with user-centered outcomes. Results and Conclusions The findings indicate that current WMRA designs face significant limitations, including restricted workspace coverage, inadequate gripper dexterity, suboptimal kinematic configurations, limited payload capacity, high cost, and lack of modularity. Safety mechanisms remain underdeveloped, creating barriers to broader adoption. Nevertheless, advancements in AI-driven control systems, modular design strategies, and integration with complementary assistive technologies demonstrate promising progress. The review concludes that WMRAs have substantial potential to improve autonomy and daily functioning for individuals with disabilities. Addressing technical and practical shortcomings is essential to ensure successful real-world deployment. These insights contribute to disability and rehabilitation research, as they highlight pathways to enhance accessibility, safety, and cost-effectiveness in assistive technologies that support independent living.

Vision-based pose estimation is an important application of machine vision. Currently, analytical and iterative methods are used to solve the object pose. The analytical solutions generally take less computation time. However, the analytical solutions are extremely susceptible to noise. The iterative solutions minimize the distance error between feature points based on 2D image pixel coordinates. However, the non-linear optimization needs a good initial estimate of the true solution, otherwise they are more time consuming than analytical solutions. Moreover, the image processing error grows rapidly with measurement range increase. This leads to pose estimation errors. All the reasons mentioned above will cause accuracy to decrease. To solve this problem, a novel pose estimation method based on four coplanar points is proposed. Firstly, the coordinates of feature points are determined according to the linear constraints formed by the four points. The initial coordinates of feature points acquired through the linear method are then optimized through an iterative method. Finally, the coordinate system of object motion is established and a method is introduced to solve the object pose. The growing image processing error causes pose estimation errors the measurement range increases. Through the coordinate system, the pose estimation errors could be decreased. The proposed method is compared with two other existing methods through experiments. Experimental results demonstrate that the proposed method works efficiently and stably.

We introduce a framework for dynamic evaluation of the fingers movements: flexion, extension, abduction and adduction. This framework estimates angle measurements from joints computed by a hand pose estimation algorithm using a depth sensor (Realsense SR300). Given depth maps as input, our framework uses Pose-REN [1], which is a state-of-art hand pose estimation method that estimates 3D hand joint positions using a deep convolutional neural network. The pose estimation algorithm runs in real-time, allowing users to visualise 3D skeleton tracking results at the same time as the depth images are acquired. Once 3D joint poses are obtained, our framework estimates a plane containing the wrist and MCP joints and measures flexion/extension and abduction/adduction angles by applying computational geometry operations with respect to this plane. We analysed flexion and abduction movement patterns using real data, extracting the movement trajectories. Our preliminary results show this method allows an automatic discrimination of hands with Rheumatoid Arthritis (RA) and healthy patients. The angle between joints can be used as an indicative of current movement capabilities and function. Although the measurements can be noisy and less accurate than those obtained statically through goniometry, the acquisition is much easier, non-invasive and patient-friendly, which shows the potential of our approach. The system can be used with and without orthosis. Our framework allows the acquisition of measurements with minimal intervention and significantly reduces the evaluation time.