Soft Objects Research Articles

Object Recognition with SSD MobileNet Pre-Trained Model in the Cashier Application

Object recognition is a type of image processing technique that is frequently employed in current applications such as facial identification, vehicle detection, and automated cashiers. One issue with barcode and RFID cashier apps is that they cannot scan several products at the same time. The cashier application employing object identification using picture images is believed to be able to distinguish more than one object in order to speed up the transaction process. The usage of SSD pre-trained models with MobileNet architecture to detect items in automatic cashier applications is discussed in this paper. This study put the model to the test on three types of soft drink objects: coca-cola, floridina, and good day. A smartphone camera was used to collect the data, which totaled 203 images. The findings indicated that the product object identification method was 82.9% accurate, 97.5% precise, and 84.7% recall. The object recognition process takes between 365 and 827 milliseconds, with an average time of 695 milliseconds (0.69 seconds).

Fathers, Breastfeeding, and Infant Sleep Practices: Findings From a State-Representative Survey.

To assess infant breastfeeding initiation and any breastfeeding at 8 weeks and safe sleep practices (back sleep position, approved sleep surface, and no soft objects or loose bedding ["soft bedding"]) by select paternal characteristics among a state-representative sample of fathers with new infants. Pregnancy Risk Assessment Monitoring System (PRAMS) for Dads, a novel population-based cross-sectional study, surveyed fathers in Georgia 2-6 months after their infant's birth. Fathers were eligible if the infant's mother was sampled for maternal PRAMS from October 2018 to July 2019. Of 250 respondents, 86.1% reported their infants ever breastfed and 63.4% reported breastfeeding at 8 weeks. Initiation and breastfeeding at 8 weeks were more likely to be reported by fathers who reported wanting their infant's mother to breastfeed than those who did not want her to breastfeed or had no opinion (adjusted prevalence ratio [aPR] = 1.39; 95% confidence interval [CI], 1.15-1.68; aPR = 2.33; 95% CI, 1.59-3.42, respectively) and fathers who were college graduates than those with ≤high school diploma (aPR = 1.25; 95% CI, 1.06-1.46; aPR = 1.44; 95% CI, 1.08-1.91, respectively). Although about four-fifths (81.1%) of fathers reported usually placing their infants to sleep on their back, fewer fathers report avoiding soft bedding (44.1%) or using an approved sleep surface (31.9%). Non-Hispanic Black fathers were less likely to report back sleep position (aPR = 0.70; 95% CI, 0.54-0.90) and no soft bedding (aPR = 0.52; 95% CI, 0.30-0.89) than non-Hispanic white fathers. Fathers reported suboptimal infant breastfeeding rates and safe sleep practices overall and by paternal characteristics, suggesting opportunities to include fathers in promotion of breastfeeding and infant safe sleep.

The biomechanics of tooth strength: testing the utility of simple models for predicting fracture in geometrically complex teeth.

Teeth must fracture foods while avoiding being fractured themselves. This study evaluated dome biomechanical models used to describe tooth strength. Finite-element analysis (FEA) tested whether the predictions of the dome models applied to the complex geometry of an actual tooth. A finite-element model was built from microCT scans of a human M3. The FEA included three loading regimes simulating contact between (i) a hard object and a single cusp tip, (ii) a hard object and all major cusp tips and (iii) a soft object and the entire occlusal basin. Our results corroborate the dome models with respect to the distribution and orientation of tensile stresses, but document heterogeneity of stress orientation across the lateral enamel. This implies that high stresses might not cause fractures to fully propagate between cusp tip and cervix under certain loading conditions. The crown is most at risk of failing during hard object biting on a single cusp. Geometrically simple biomechanical models are valuable tools for understanding tooth function but do not fully capture aspects of biomechanical performance in actual teeth whose complex geometries may reflect adaptations for strength.

A Palm-Shape Variable-Stiffness Gripper Based on 3D-Printed Fabric Jamming

Soft grippers have excellent adaptability for a variety of objects and tasks. Jamming-based variable stiffness materials can further increase soft grippers' gripping force and capacity. Previous universal grippers enabled by granular jamming have shown great capability of handling objects with various shapes and weight. However, they require a large pushing force on the object during gripping, which is not suitable for very soft or free-hanging objects. In this paper, we create a novel palm-shape anthropomorphic variable-stiffness gripper enabled by jamming of 3D printed fabrics. This gripper is conformable and gentle to objects with different shapes, requires little pushing force, and increases gripping strength only when necessary. We present the design, fabrication and performance of this gripper and tested its conformability and gripping capacity. Our design utilizes soft pneumatic actuators to drive two wide palms to enclose objects, thanks to the excellent conformability of the structured fabrics. While the pinch force is low, the palm can significantly increase stiffness to lift heavy objects with a maximum gripping force of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$17\,$</tex-math></inline-formula> N and grip-to-pinch force ratio of 42. We also explore different variable-stiffness materials in the gripper, including sheets for layer jamming, to compare their performances. We conduct gripping tests on standard objects and daily items to show the great capacity of our gripper design.

Hybrid Optimization-based Cutting Simulation for Soft Objects

Cutting simulation for soft objects is a crucial technology in surgical simulators. The capabilities of a virtual surgery system are determined by the cutting efficiency and simulation performance. Taking both into full consideration, we present a real-time soft object cutting simulation method. The surface-volume model coupling a triangular mesh with a tetrahedral mesh is used as the geometric model. To drive deformable soft objects, we utilize position-based dynamics. The simulation incorporates a flexible progressive cutting technique that updates the topology and produces the cut surface simultaneously with the movement of the virtual scalpel. The incision’s tension and constraint are applied to the simulation steps, ensuring that the incision opens automatically and retains its shape without the need for complicated control. A hybrid optimization is performed to improve the cut mesh and reduce ill-conditioned sliver tetrahedrons while keeping the simulation in parallel. Experiments demonstrate that our method can provide stable and efficient soft object cuttings and the objects can maintain good performance in the subsequent simulation. Additionally, we apply the human slice data to generate real textures for the cut surface and employ a force feedback device to introduce the haptic effect, further enhancing the cutting simulation efficacy.

One-step in situ construction of anisotropic bilayer hydrogel with high sensitivity and wide detection range for adaptive tactile sensing

As an excellent human body signal receptor, the skin of the human body can accurately sense changes in the surrounding environment, transmit the signal to the brain and allow the muscles to make timely feedback. Based on this, bionic pressure sensors have received extensive attention. However, for traditional pressure sensors, it is still facing severe challenges to achieve wide range and high sensitivity at the same time. Therefore, this work utilized a simple and efficient in-situ polymerization strategy to construct a bilayer hydrogel consisting of a soft layer and a tough layer, achieving high sensitivity (62.7 kPa−1) under 0–5 kPa and a wide sensing range (up to 600 kPa). In addition, the sensitivities at 5–100 kPa, 100–250 kPa, and 250–600 kPa were 23.6 kPa−1, 9.63 kPa−1, and 0.42 kPa−1, respectively. The soft layer introduces low-density monomers to induce weak chemical cross-linking to improve sensitivity, while the tough layer introduces high-density monomers and nano-scale polyacrylamide to form a high-density chemical/physical double cross-linking network, thereby improving the pressure monitoring range. The effects of monomer density, nano-polyacrylamide, and the thickness ratio of the soft and tough layers in the bilayer gel were simulated by finite element simulation, and the results were in good agreement with the experimental results. On this basis, the hydrogel is assembled on the mechanical arm as a pressure sensor, and forms a tactile feedback system simulating human skin-brain-muscle system with the controller and computer. The system can adaptively and self-feedback grasp soft and tough objects without causing damage to the objects. This work provides new strategies and research directions for bionic robots, bioinspired soft materials, artificial skin, intelligent sensing and other fields.

Faster Indentation Influences Skin Deformation To Reduce Tactile Discriminability of Compliant Objects.

To discriminate the compliance of soft objects, we rely upon spatiotemporal cues in the mechanical deformation of the skin. However, we have few direct observations of skin deformation over time, in particular how its response differs with indentation velocities and depths, and thereby helps inform our perceptual judgments. To help fill this gap, we develop a 3D stereo imaging method to observe contact of the skin's surface with transparent, compliant stimuli. Experiments with human-subjects, in passive touch, are conducted with stimuli varying in compliance, indentation depth, velocity, and time duration. The results indicate that contact durations greater than 0.4 s are perceptually discriminable. Moreover, compliant pairs delivered at higher velocities are more difficult to discriminate because they induce smaller differences in deformation. In a detailed quantification of the skin's surface deformation, we find that several, independent cues aid perception. In particular, the rate of change of gross contact area best correlates with discriminability, across indentation velocities and compliances. However, cues associated with skin surface curvature and bulk force are also predictive, for stimuli more and less compliant than skin, respectively. These findings and detailed measurements seek to inform the design of haptic interfaces.

A Recognition Method for Soft Objects Based on the Fusion of Vision and Haptics.

For humans and animals to recognise an object, the integration of multiple sensing methods is essential when one sensing modality is only able to acquire limited information. Among the many sensing modalities, vision has been intensively studied and proven to have superior performance for many problems. Nevertheless, there are many problems which are difficult to solve by solitary vision, such as in a dark environment or for objects with a similar outlook but different inclusions. Haptic sensing is another commonly used means of perception, which can provide local contact information and physical features that are difficult to obtain by vision. Therefore, the fusion of vision and touch is beneficial to improve the robustness of object perception. To address this, an end-to-end visual-haptic fusion perceptual method has been proposed. In particular, the YOLO deep network is used to extract vision features, while haptic explorations are used to extract haptic features. Then, visual and haptic features are aggregated using a graph convolutional network, and the object is recognised based on a multi-layer perceptron. Experimental results show that the proposed method excels in distinguishing soft objects that have similar appearance but varied interior fillers, comparing a simple convolutional network and a Bayesian filter. The resultant average recognition accuracy was improved to 0.95 from vision only (mAP is 0.502). Moreover, the extracted physical features could be further used for manipulation tasks targeting soft objects.

Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach.

In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information via custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction. Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without ad-hoc sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user via vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees. The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).

Real-time vibrotactile pattern generation and identification using discrete event-driven feedback

This study assesses human identification of vibrotactile patterns by using real-time discrete event-driven feedback. Previously acquired force and bend sensor data from a robotic hand were used to predict movement-type (stationary, flexion, contact, extension, release) and object-type (no object, hard object, soft object) states by using decision tree (DT) algorithms implemented in a field-programmable gate array (FPGA). Six able-bodied humans performed a 2- and 3-step sequential pattern recognition task in which state transitions were signaled as vibrotactile feedback. The stimuli were generated according to predicted classes represented by two frequencies (F1: 80 Hz, F2: 180 Hz) and two magnitudes (M1: low, M2: high) calibrated psychophysically for each participant; and they were applied by two actuators (Haptuators) placed on upper arms. A soft/hard object was mapped to F1/F2; and manipulating it with low/high force was assigned to M1/M2 in the left actuator. On the other hand, flexion/extension movement was mapped to F1/F2 in the right actuator, with movement in air as M1 and during object manipulation as M2. DT algorithm performed better for the object-type (97%) than the movement-type (88%) classification in real time. Participants could recognize feedback associated with 14 discrete-event sequences with low-to-medium accuracy. The performance was higher (76 ± 9% recall, 76 ± 17% precision, 78 ± 4% accuracy) for recognizing any one event in the sequences. The results show that FPGA implementation of classification for discrete event-driven vibrotactile feedback can be feasible in haptic devices with additional cues in the physical context.

Stiffness-Estimation-Based Grasping Force Fuzzy Control for Underactuated Prosthetic Hands

Human hands can rapidly perceive the stiffness of an object and quickly adapt to different soft and hard objects by adjusting the grasp force of each finger. However, patients with upper-limb loss have difficulty in outputting accurate grasp force based on objects stiffness information, as the efferent and afferent feedback of nerve channels was losing. To realize stiffness-based force control on a prosthetic hand, it is necessary to know the stiffness of the object to be grasped in advance. This article proposed a novel method to estimate the stiffness of the grasped object based on a kinematics model of prosthetic fingers. Using underactuated mechanisms, the stiffness of the grasped object can be derived based on the contact stiffness information that is obtained upon contact between the prosthetic hand and the target object. Based on the estimation of the stiffness of the grasped object, we developed a control strategy to adjust the inappropriate surface electromyography decoding force of the prosthetic hand, which simulated the motion of a human hand. The selected objects in experiments were representative and common objects used in daily life, with stiffness ranging from soft to hard. The proposed method was proven to be effective and feasible during experimental tests, where such a method can be used to quickly differentiate objects with different stiffness values and thus replicate the controllable stiffness-based force of a human hand with a prosthetic hand.

Soft Hydroxyapatite Composites Based on Triazine-Trione Systems as Potential Biomedical Engineering Frameworks.

Composites of triazine-trione (TATO) thiol-ene networks and hydroxyapatite (HA) have shown great potential as topological fixation materials for complex bone fractures due to their high flexural modulus, biocompatibility, and insusceptibility to forming soft-tissue adhesions. However, the rigid mechanical properties of these composites make them unsuitable for applications requiring softness. The scope of these materials could therefore be widened by the design of new TATO monomers that would lead to composites with a range of mechanical properties. In this work, four novel TATO-based monomers, decorated with either ester or amide linkages as well as alkene or alkyne end groups, have been proposed and synthesized via fluoride-promoted esterification (FPE) chemistry. The ester-modified monomers were then successfully formulated along with the thiol TATO monomer tris [2-(3-mercaptopropionyloxy)ethyl] isocyanurate (TEMPIC) and HA to give soft composites, following the established photo-initiated thiol-ene coupling (TEC) or thiol-yne coupling (TYC) chemistry methodologies. The most promising composite shows excellent softness, with a flexural modulus of 57 (2) MPa and εf at maximum σf of 11.8 (0.3)%, which are 117 and 10 times softer than the previously developed system containing the commercially available tri-allyl TATO monomer (TATATO). Meanwhile, the surgically convenient viscosity of the composite resins and their excellent cytotoxicity profile allow them to be used in the construction of soft objects in a variety of shapes through drop-casting suitable for biomedical applications.

Shape Servoing of Deformable Objects using Adaptive Deformation Model Estimation

In this paper, we propose an adaptive shape servoing method to deform a soft object into a desired 3-D shape. The high dimensional representation and the unknown deformation properties of the soft object pose a challenge to actively manipulate its shape. To address this issue, we develop a method to compute the deformation Jacobian matrix in real-time. The Jacobian is estimated using a set of basis functions and its corresponding parameters to capture the dynamics of the system and relate the applied input motion to changes in the soft object's shape. An integral concurrent learning (ICL) based adaptive update law is derived using Lyapunov analysis to estimate the deformation parameters and prove its convergence. A physics-based simulation is used to validate the proposed method and controller by performing manipulation tasks with different desired configurations. The performance is compared with a standard gradient update law to demonstrate the accuracy and robustness of our approach.

Fluid-solid transitions in photonic crystals of soft, thermoresponsive microgels.

Microgels are often discussed as well-suited model system for soft colloids. In contrast to rigid spheres, the microgel volume and, coupled to this, the volume fraction in dispersion can be manipulated by external stimuli. This behavior is particularly interesting at high packings where phase transitions can be induced by external triggers such as temperature in the case of thermoresponsive microgels. A challenge, however, is the determination of the real volume occupied by these deformable, soft objects and consequently, to determine the boundaries of the phase transitions. Here we propose core-shell microgels with a rigid silica core and a crosslinked, thermoresponsive poly-N-isopropylacrylamide (PNIPAM) shell with a carefully chosen shell-to-core size ratio as ideal model colloids to study fluid-solid transitions that are inducible by millikelvin changes in temperature. Specifically, we identify the temperature ranges where crystallization and melting occur using absorbance spectroscopy in a range of concentrations. Slow annealing from the fluid to the crystalline state leads to photonic crystals with Bragg peaks in the visible wavelength range and very narrow linewidths. Small-angle X-ray scattering is then used to confirm the structure of the fluid phase as well as the long-range order, crystal structure and microgel volume fraction in the solid phase. Thanks to the scattering contrasts and volume ratio of the cores with respect to the shells, the scattering data do allow for form factor analysis revealing osmotic deswelling at volume fractions approaching and also exceeding the hard sphere packing limit.

A conformal van der Waals graphene coating enabled high-performance piezo-ionic sensor for spatial, gesture, and object recognition

Highly electrically conductive (16 ohm sq−1) and conformal van der Waals graphene electrodes were synthesized on piezo-ionic materials. Such electrodes enhanced the piezo-ionic output to 90.3 mV and distinguished soft and hard objects.

A Robot Hand Mechanism with Functions to Scoop and Pull-in a Soft Object

A Robot Hand Mechanism with Functions to Scoop and Pull-in a Soft Object

Vision-based Picking-Up Control by Robotic Grippers with Belts

This paper presents a vision-based control to pick up an object by a robotic hand. In-hand manipulation, i.e., changing the position and orientation of the grasped object without dropping it, is a challenging task. It is important and necessary for tasks like achieving a stable grasp by translating or rotating the grasped object even though only a part of the object is grasped. However, it is difficult for soft objects like food, because the grasping force needs to be just enough to hold and manipulate the object without crushing it. To tackle this problem, we propose a system which integrates the control of the hand and object detection via a camera. The target robotic hand is configured by two parallel grippers with conveyor belts for manipulation, and is equipped only with a stereo camera as a sensor. From the camera image frames, the three-dimensional position, orientation, and size of the object are calculated. While controlling the grippers and belts, the slippage between belts and the object is estimated based on the difference between their displacements. The validity of the proposed system is verified through the experiments wherein the hand is controlled to pick up objects of varying size and softness. By experimentally evaluating whether the objects have slipped, been crushed, or been manipulated to the target position, it was confirmed that all objects could be picked up with the appropriate force without dropping or crushing them.

Basic Experiments Toward Mixed Reality Dynamic Navigation for Laparoscopic Surgery

Laparoscopic surgery is a minimally invasive procedure that is performed by viewing endoscopic camera images. However, the limited field of view of endoscopic cameras makes laparoscopic surgery difficult. To provide more visual information during laparoscopic surgeries, augmented reality (AR) surgical navigation systems have been developed to visualize the positional relationship between the surgical field and organs based on preoperative medical images of a patient. However, since earlier studies used preoperative medical images, the navigation became inaccurate as the surgery progressed because the organs were displaced and deformed during surgery. To solve this problem, we propose a mixed reality (MR) surgery navigation system in which surgical instruments are tracked by a motion capture (Mocap) system; we also evaluated the contact between the instruments and organs and simulated and visualized the deformation of the organ caused by the contact. This paper describes a method for the numerical calculation of the deformation of a soft body. Then, the basic technology of MR and projection mapping is presented for MR surgical navigation. The accuracy of the simulated and visualized deformations is evaluated through basic experiments using a soft rectangular cuboid object.

МНОГОМОДОВЫЙ СКАНИРУЮЩИЙ МИКРОСКОП ИОННОЙ ПРОВОДИМОСТИ С СИСТЕМОЙ ПЬЕЗОИНЕРЦИАЛЬНОГО ПЕРЕМЕЩЕНИЯ

A scanning ion conductance microscope (SICM) has been developed, operating in several modes: DC mode, current modulation mode, and hopping mode. SICM employs a piezoelectric-inertial movement system. The nanoprobes, in the form of glass nanopipettes with an internal radius of r ~ 50 nm, have been created and tested. The current-voltage characteristics I (V) and current dependences on the distance between the probe and the sample I (z) (approach/withdrawal curves) were measured. Images of a polymeric test object with a periodic structure and a biological object (CHO cell) were obtained, their quality was assessed, and the features of the SICM operation in various modes are discussed. The multimode SICM provides non-destructive, non-contact visualization of soft objects in a liquid conducting medium with nanometer spatial resolution in various measuring modes and can be used in biology, cytology, electrochemistry, and medicine when studying inorganic soft objects, biological objects in buffer media, etc.

Human–robot handover with prior-to-pass soft/rigid object classification via tactile glove

Human–robot handovers constitute a challenging and fundamental aspect of physical human–robot interaction. This paper describes the design and implementation of a human–robot handover pipeline in the case in which both soft and rigid objects are passed by the human to the robot. These objects require different profiles of grasping torques by the robot hand fingers, so as to avoid damaging them. As a viable solution to this problem, a tactile glove worn by the human is used to provide real-time information to a deep neural network, which classifies each object as soft or rigid in the pre-handover phase: this information is passed to the robot, which applies the grasping torque profile suitable for the specific type of object. The proposed method is designed and validated based on experiments with eight human participants and 24 objects. The outcomes of these experiments regarding classification accuracy, force and torque profiles, and evaluation of the subjective experiences via questionnaires, are described and discussed.