Tactile 3D Research Articles

Enabling Microscopy for the Visually Impaired: Tactile 3D Printed Representations of Electron Micrographs

Enabling Microscopy for the Visually Impaired: Tactile 3D Printed Representations of Electron Micrographs

Evidence for tactile 3D shape discrimination by octopus.

Octopuses integrate visual, chemical and tactile sensory information while foraging and feeding in complex marine habitats. The respective roles of these modes are of interest ecologically, neurobiologically, and for development of engineered soft robotic arms. While vision guides their foraging path, benthic octopuses primarily search "blindly" with their arms to find visually hidden prey amidst rocks, crevices and coral heads. Each octopus arm is lined with hundreds of suckers that possess a combination of chemo- and mechanoreceptors to distinguish prey. Contact chemoreception has been demonstrated in lab tests, but mechanotactile sensing is less well characterized. We designed a non-invasive live animal behavioral assay that isolated mechanosensory capabilities of Octopus bimaculoides arms and suckers to discriminate among five resin 3D-printed prey and non-prey shapes (all with identical chemical signatures). Each shape was introduced inside a rock dome and was only accessible to the octopus' arms. Octopuses' responses were variable. Young octopuses discriminated the crab prey shape from the control, whereas older octopuses did not. These experiments suggest that mechanotactile sensing of 3D shapes may aid in prey discrimination; however, (i) chemo-tactile information may be prioritized over mechanotactile information in prey discrimination, and (ii) mechanosensory capability may decline with age.

Using micro spheres as reference artifacts for the in-situ characterization of tactile 3D micro probes along the probing sphere’s equator

Abstract Nano and micro coordinate measuring machines (CMMs) have been developed for the characterization of small dimensional features. They require a procedure which enables a traceable and precise characterization of probing spheres. In this contribution we explore the use of well characterized micro spheres as reference artifacts for the in-situ characterization of probing spheres along the probing sphere’s equator. The spheres are characterized using a strategy which is based on a set of tactile surface scans in conjunction with a stitching-algorithm. These micro spheres serve as a reference for the in-situ characterization of a tactile 3D micro probe on a nano measuring machine (NMM-1). Our investigations are based on a sample of eight spheres sourced from two different suppliers. Although the sample is small, we could already observe characteristics which seem to be typical for spheres of a certain type (i.e. nominal radius and material). The experiments indicate that micro spheres are a suitable reference artifact for tactile 3D micro probes. We were able to reproduce the measured mean radius of the probing sphere with a standard deviation of 31 nm using reference spheres whose nominal radius covers a range of 89 µm (61 µm up to 150 µm).

Accessible Point-and-Tap Interaction for Acquiring Detailed Information about Tactile Graphics and 3D Models.

We have devised a novel "Point-and-Tap" interface that enables people who are blind or visually impaired (BVI) to easily acquire multiple levels of information about tactile graphics and 3D models. The interface uses an iPhone's depth and color cameras to track the user's hands while they interact with a model. When the user points to a feature of interest on the model with their index finger, the system reads aloud basic information about that feature. For additional information, the user lifts their index finger and taps the feature again. This process can be repeated multiple times to access additional levels of information. For instance, tapping once on a region in a tactile map could trigger the name of the region, with subsequent taps eliciting the population, area, climate, etc. No audio labels are triggered unless the user makes a pointing gesture, which allows the user to explore the model freely with one or both hands. Multiple taps can be used to skip through information levels quickly, with each tap interrupting the current utterance. This allows users to reach the desired level of information more quickly than listening to all levels in sequence. Experiments with six BVI participants demonstrate that the approach is practical, easy to learn and effective.

Augmented Reality as a Tool for Enhancing Metacognitive Knowledge in Chemistry

This study developed a Teaching and Learning Material (TLM) with Augmented Reality (AR) to enhance the students’ metacognitive knowledge, namely declarative, procedural, and conditional knowledge in General Chemistry. The developed TLM has three components: the printed learning material that contains the AR trackers; a database of tactile 3D objects and animations that served as metacognitive scaffolds; and a dedicated smartphone software application specifically created for the retrieval, display and control of AR objects. The design-based research paradigm was used in the development of the Teaching and Learning Material with Augmented Reality, while the pre-experimental single group research design was employed in the assessment of the effect of scaffolds using AR on the students’ metacognitive knowledge. A Metacognitive Knowledge Test (MKT) was administered to the student participants before and after using the AR learning material system. Using the Hake factor test, the study found that the student participants achieved high learning gains for declarative and procedural metacognitive knowledge as signified by g values of 0.73 and 0.79, respectively. In terms of conditional metacognitive knowledge, the students registered a medium learning gain given by g = 0.68. It is recommended that teachers use innovative teaching and learning materials embedded with AR to enhance students’ learning attributes like metacognitive knowledge.

Advancements in Oral Maxillofacial Surgery: A Comprehensive Review on 3D Printing and Virtual Surgical Planning

This comprehensive review explores the advancements in Orthognathic and Oral Maxillofacial Surgery, focusing on the integration of 3D Printing and Virtual Surgical Planning (VSP). Traditional surgical methods, while effective, come with inherent risks and complications, and can lead to variability in outcomes due to the reliance on the surgeon’s skill and experience. The shift towards patient-centric care necessitates personalized surgical methods, which can be achieved through advanced technology. The amalgamation of 3D printing and VSP revolutionizes surgical planning and implementation by providing tactile 3D models for visualization and planning, and accurately designed surgical guides for execution. This convergence of digital planning and physical modeling facilitates a more predictable, personalized, and precise surgical process. However, the adoption of these technologies presents challenges, including the need for extensive software training and the steep learning curve associated with computer-aided design programs. Despite these challenges, the integration of 3D printing and VSP paves the way for advanced patient care in orthognathic and oral maxillofacial surgery.

Cognitive map formation in the blind is enhanced by three-dimensional tactile information

For blind individuals, tactile maps are useful tools to form cognitive maps through touch. However, they still experience challenges in cognitive map formation and independent navigation. Three-dimensional (3D) tactile information is thus increasingly being considered to convey enriched spatial information, but it remains unclear if it can facilitate cognitive map formation compared to traditional two-dimensional (2D) tactile information. Consequently, the present study investigated the impact of the type of sensory input (tactile 2D vs. tactile 3D vs. a visual control condition) on cognitive map formation. To do so, early blind (EB, n = 13), late blind (LB, n = 12), and sighted control (SC, n = 14) participants were tasked to learn the layouts of mazes produced with different sensory information (tactile 2D vs. tactile 3D vs. visual control) and to infer routes from memory. Results show that EB manifested stronger cognitive map formation with 3D mazes, LB performed equally well with 2D and 3D tactile mazes, and SC manifested equivalent cognitive map formation with visual and 3D tactile mazes but were negatively impacted by 2D tactile mazes. 3D tactile maps therefore have the potential to improve spatial learning for EB and newly blind individuals through a reduction of cognitive overload. Installation of 3D tactile maps in public spaces should be considered to promote universal accessibility and reduce blind individuals’ wayfinding deficits related to the inaccessibility of spatial information through non-visual means.

High-Precision 3D Reconstruction Study with Emphasis on Refractive Calibration of GelStereo-Type Sensors.

GelStereo sensing technology is capable of performing three-dimensional (3D) contact shape measurement under various contact structures such as bionic curved surfaces, which has promising advantages in the field of visuotactile sensing. However, due to multi-medium ray refraction in the imaging system, robust and high-precision tactile 3D reconstruction remains a challenging problem for GelStereo-type sensors with different structures. In this paper, we first propose a universal Refractive Stereo Ray Tracing (RSRT) model for GelStereo-type sensing systems to realize 3D reconstruction of the contact surface. Moreover, a relative geometry-based optimization method is presented to calibrate multiple parameters of the proposed RSRT model, such as the refractive indices and structural dimensions. Furthermore, extensive quantitative calibration experiments are performed on four different GelStereo sensing platforms; the experimental results show that the proposed calibration pipeline can achieve less than 0.35 mm in Euclidean distance error, based on which we believe that the proposed refractive calibration method can be further applied in more complex GelStereo-type and other similar visuotactile sensing systems. Such high-precision visuotactile sensors can facilitate the study of robotic dexterous manipulation.

Using tactile imagery to teach geometry to students with visual impairments in the United Arab Emirates

Learning geometry is difficult for students with visual impairments (VIs). However, tactile imagery, the process of constructing mental images of physical objects with certain shapes, may help them make sense of geometrical shapes. Thus, discussions have centred on interventions to promote the participation of students with VIs in the learning of shapes. This study explored tactile imagery connecting physical touch to memory as an approach to teaching shapes to students with VIs. Eleven students, five with VIs and six sighted students took part in this experimental design study. A tactile imagery test and intervention lessons were developed for this study. Four tactile imagery domains (tactile discrimination, tactile memory 2D, tactile memory 3D and pattern recall), each made up of 10 tests, guided the design of the tests and training lessons. The students’ scores from the pre-test and post-test were subjected to mean computations, Mann-Whitney U tests and the Wilcoxon signed-rank test. The post-test results revealed that students with VIs performed better than their sighted peers. The study concludes with a discussion of the need for teacher educators to consider using tactile imagery as a way of teaching geometry to students with VIs.

3D printing ophthalmology related models for enhancing learning through the concept of puzzle assembly - A comprehensive self-learning tactile tool kit.

Practical sessions facilitate teaching, critical thinking, and coping skills, especially among medical students and professionals. Currently, in ophthalmology, virtual and augmented reality are employed for surgical training by using three-dimensional (3D) eyeball models. These 3D models when printed can be used not only for surgical training but also in teaching ophthalmic residents and fellows for concept learning through tactile 3D puzzle assembly. 3D printing is perfectly suited for the creation of complex bespoke items in a cost-effective manner, making it ideal for rapid prototyping. Puzzle making, when combined with 3D printing can evolve into a different level of learning in the field of ophthalmology. Though various 3D eyeball models are currently available, complex structures such as the cerebral venous system and the circle of Willis have never been 3D printed and presented as 3D puzzles for assembling and learning. According to our knowledge, this concept of ophthalmic pedagogy has never been reported. In this manuscript, we discuss in detail the 3D models created by us (patent pending), for printing into multiple puzzle pieces for effective tactile learning by cognitive assembling.

Open-Source, Tactile 3D Printed Interlockable Tiles Incorporating Valency, Bonding, and Hybridization for Molecular Representation for Sighted and Visually Impaired Students

Learning to write chemical formulas of compounds is a basic and indispensable part of understanding and studying chemistry. However, it is hard for students with visual impairment to assess and learn molecular arrangements and formulas. For the convenience of such students with special needs, it is necessary to come up with easy, comprehensive, and efficient solutions to make them independent, self-reliant, and be able to control their progress at their own pace. This can be achieved with the aid of emerging new technologies such as 3D printing. In the current research work, a 3D printed model was designed and developed. The open-source repository of 3D printable model design files for the basic building blocks to represent chemical formulas of different elements such as carbon, hydrogen, other elements, and related organic as well as inorganic compounds are being shared freely. These models feature a lock and key design, like jigsaw puzzle pieces, along with Braille and print notation that allow learning to be more interactive, engaging, productive, and effective as the sense of touch (tactile) is one of the primary ways through which visually impaired people perceive physical entities. Each elemental piece in the model is designed specifically as per its available bonding electrons, i.e., valency, allowed only to connect with another complementary element in a unique configuration to ensure their correct arrangement and representation using the lock-and-key feature. The concept of hybridization is also included for enhanced learning.

Soft touch between a highly flexible bio‐inspired tactile sensor and 3D objects

AbstractScanning and reconstructing the environment using tactile sensors alongside optical sensors is a promising approach in mobile robotics. Within the present paper, we take advantage of a recently presented vibrissa‐inspired tactile sensor concept for 3D object scanning and reconstruction, broadening our previous studies. The sensor consists of a slender, cylindrical, highly flexible probe, one‐sided clamped to some force‐torque measuring device. The probe is shifted relatively to an object of interest by displacing its clamping position quasi‐statically. Consequently, the probe gets bent, sweeps over the object and transmits mechanical signals (observables) to its support. The focus of the present investigation is on how the probe sweeps over a new type of object (paraboloid), verifying a necessary condition for optional contact points. Finally, this condition allows to find multiple equilibrium states for a single clamping position.

Creating Tactile Educational Materials for the Visually Impaired and Blind Students Using AI Cloud Computing

There are 24.5 million visually impaired and blind (VIB) students who have limited access to educational materials due to cost or availability. Although advancement in technology is prevalent, providing individualized learning using technology remains a challenge without the proper tools or experience. The TacPic system was developed as an online platform to create tactile educational materials (TEM) based on the image inputs of users who do not have prior experience in tactile photo development or 3D printing. The TacPic system allows the users to simply upload images to a website and uses AI cloud computing on the Amazon Web Services platform. First, it segments and labels the images. Then, the text label is converted into braille words. Subsequently, surface rendering and consolidation of the image and text is performed, before it is converted into a single file that is ready for 3D printing. Currently, the types of TEM that can be created are tactile flashcards, tactile maps, and tactile peg puzzles, which can be developed within a few hours. This is in contrast to a development period of weeks using traditional methods. Furthermore, the tactile educational materials were tested by two VIB teachers and six VIB students. It was found that those who are congenitally blind need more time to identify the object and rely more on the braille labels compared to students who became blind at a later age. Teachers also suggested producing TEM that use simpler images, and TEM that are suitable for both sighted and VIB students. In conclusion, the researchers successfully developed a platform that allows more educators or parents to develop personalized and individualized TEM. In the future, further optimization of the algorithms to improve segmentation and the inclusion of other features, such as color, could be undertaken. Finally, new printing materials and methods are needed to improve printing efficiency.

Surface integrity in machining of AZ91/SiC composites

Use of metal matrix composites (MMC) is growing due to their high strength-to-weight ratio, resistance to wear, creep, etc. Machining of metal-matrix composites (MMC) faces many challenges, especially with regard to obtaining a finished surface with high quality. In this research, AZ91/SiC samples with different volume fractions are machined at different cutting conditions with respect to feed rate, cutting speed, and depth of cut. Surface integrity of the machined samples is analyzed by different methods such as tactile profilometer and 3D surface topography to investigate the SiC effects on the finished surface. Additionally, sample surfaces are evaluated by scanning electron microscope (SEM) and with energy-dispersive X-ray (EDS) to assess the surface defects formed around reinforcement materials. Results indicate SiC particles decline the surface quality and uniformity due to the formation of some defects such as micro cracks, holes, and undesired deformations when the cutting process. Also, subsurface SiC particles close to the machined surface are cracked after machining.

Transfer of Learning from Vision to Touch: A Hybrid Deep Convolutional Neural Network for Visuo-Tactile 3D Object Recognition.

Transfer of learning or leveraging a pre-trained network and fine-tuning it to perform new tasks has been successfully applied in a variety of machine intelligence fields, including computer vision, natural language processing and audio/speech recognition. Drawing inspiration from neuroscience research that suggests that both visual and tactile stimuli rouse similar neural networks in the human brain, in this work, we explore the idea of transferring learning from vision to touch in the context of 3D object recognition. In particular, deep convolutional neural networks (CNN) pre-trained on visual images are adapted and evaluated for the classification of tactile data sets. To do so, we ran experiments with five different pre-trained CNN architectures and on five different datasets acquired with different technologies of tactile sensors including BathTip, Gelsight, force-sensing resistor (FSR) array, a high-resolution virtual FSR sensor, and tactile sensors on the Barrett robotic hand. The results obtained confirm the transferability of learning from vision to touch to interpret 3D models. Due to its higher resolution, tactile data from optical tactile sensors was demonstrated to achieve higher classification rates based on visual features compared to other technologies relying on pressure measurements. Further analysis of the weight updates in the convolutional layer is performed to measure the similarity between visual and tactile features for each technology of tactile sensing. Comparing the weight updates in different convolutional layers suggests that by updating a few convolutional layers of a pre-trained CNN on visual data, it can be efficiently used to classify tactile data. Accordingly, we propose a hybrid architecture performing both visual and tactile 3D object recognition with a MobileNetV2 backbone. MobileNetV2 is chosen due to its smaller size and thus its capability to be implemented on mobile devices, such that the network can classify both visual and tactile data. An accuracy of 100% for visual and 77.63% for tactile data are achieved by the proposed architecture.

The manufacturing procedure of 3D printed models for endoscopic endonasal transsphenoidal pituitary surgery.

BACKGROUND: Endoscopic endonasal transsphenoidal pituitary surgery is usually difficult and risky. With limited sources of cadaveric skulls, traditional methods of using virtual images to study the surgery are difficult for neurosurgeons and students because the surgery requires spatial imagination and good understanding of the patient’s conditions as well as practical experience. The three-dimensional (3D) printing technique has played an important role in clinical medicine due to its advantages of low cost, high-efficiency and customization.OBJECTIVE: CT images are used as the source data of 3D printing. The data obtained directly from the CT machine has limited accuracy, which cannot be printed without processing. Some commercial platforms can help build an accurate model but the cost and customization are not satisfactory. In this situation, a tactile, precise and low-cost 3D model is highly desirable.METHODS: Three pieces of computer software are used in the manufacturing of medical 3D models and the processing procedure is easy to understand and operate.RESULTS: This study proposes a practical and cost-effective method to obtain the corrected digital model and produce the 3D printed skull with complete structures of the nasal cavity, sellar region and different levels of pituitary tumors. The model is used for the endoscopic endonasal transsphenoidal pituitary surgery preparation.CONCLUSION: The 3D printed medical model can directly help neurosurgeons and medical students to practice their surgery skills on both general and special cases with customized structures and different levels of tumors.

Touching the stars: improving NASA 3D printed data sets with blind and visually impaired audiences

Astronomy has been an inherently visual area of science for millenia, yet a majority of its significant discoveries take place in wavelengths beyond human vision. There are many people, including those with low or no vision, who cannot participate fully in such discoveries if visual media is the primary communication mechanism. Numerous efforts have worked to address equity of accessibility to such knowledge sharing, such as through the creation of three-dimensional (3D) printed data sets. This paper describes progress made through technological and programmatic developments in tactile 3D models using NASA's Chandra X-ray Observatory to improve access to data.

Augmenting Tactile 3D Data Navigation With Pressure Sensing

AbstractWe present a pressure‐augmented tactile 3D data navigation technique, specifically designed for small devices, motivated by the need to support the interactive visualization beyond traditional workstations. While touch input has been studied extensively on large screens, current techniques do not scale to small and portable devices. We use phone‐based pressure sensing with a binary mapping to separate interaction degrees of freedom (DOF) and thus allow users to easily select different manipulation schemes (e. g., users first perform only rotation and then with a simple pressure input to switch to translation). We compare our technique to traditional 3D‐RST (rotation, scaling, translation) using a docking task in a controlled experiment. The results show that our technique increases the accuracy of interaction, with limited impact on speed. We discuss the implications for 3D interaction design and verify that our results extend to older devices with pseudo pressure and are valid in realistic phone usage scenarios.

Improved Visualisation of Patient-Specific Heart Structure Using Three-Dimensional Printing Coupled with Image-Processing Techniques Inspired by Astrophysical Methods

The aim of this study is to use image-processing techniques developed in the field of astrophysics as inspiration for a novel approach to the three-dimensional (3D) imaging of periprocedural medical data, with the intention of providing improved visualisation of patient-specific heart structure and thereby allowing for an improved quality of procedural planning with regards to individualized cardiovascular healthcare. Using anonymized patient DICOM data for a cardiac computed tomography (CT) angiography, two-dimensional slices of the patient's heart were processed using a series of software packages in order to produce an accurate 3D representation of the patient's heart tissue as a computer-generated stereolithography (STL) file, followed by the creation of a tactile 3D printout. We find that the models produced provide clear definition of heart structure, in particular in the left atrium, left ventricle and aorta. This level of clarity also allows for the aortic valve to be observed and 3D printed. This study provides a step-by-step blueprint of how this can be achieved using open source software, specifically Slicer 4.8.1, MeshLab and AutoDesk Netfabb. In addition, the implementation of astrophysical image-processing techniques shows an improvement in modelling of the heart based on the CT data, in particular in the case of small-scale features where echocardiography has previously been required for more reliable results.

Pictures in Your Mind

Tactile reliefs offer many benefits over the more classic raised line drawings or tactile diagrams, as depth, 3D shape, and surface textures are directly perceivable. Although often created for blind and visually impaired (BVI) people, a wider range of people may benefit from such multimodal material. However, some reliefs are still difficult to understand without proper guidance or accompanying verbal descriptions, hindering autonomous exploration. In this work, we present a gesture-controlled interactive audio guide (IAG) based on recent low-cost depth cameras that can be operated directly with the hands on relief surfaces during tactile exploration. The interactively explorable, location-dependent verbal and captioned descriptions promise rapid tactile accessibility to 2.5D spatial information in a home or education setting, to online resources, or as a kiosk installation at public places. We present a working prototype, discuss design decisions, and present the results of two evaluation studies: the first with 13 BVI test users and the second follow-up study with 14 test users across a wide range of people with differences and difficulties associated with perception, memory, cognition, and communication. The participant-led research method of this latter study prompted new, significant and innovative developments.