Visual Depth Research Articles

Research on abnormal object detection network of computer room inspection robot based on depth vision

Abstract This paper investigates a deep learning-based anomaly object detection network for identifying and alerting on abnormal items within computer room. First, the framework of the data center inspection robot system is outlined, and the anomaly detection task is decomposed. Next, a dataset of abnormal objects based on data center environmental information is established, and augmentation operations are performed on the created dataset. Subsequently, a SqueezeNet network model based on Residual Squeeze Excitation and Atrous Spatial Pyramid Pooling (RSE-ASPP) is proposed to optimize and improve the SqueezeNet network model. Finally, this paper employs transfer learning to address the issue of insufficient data volume. By pre-training on a large-scale dataset and fine-tuning on the constructed dataset, the accuracy and stability of abnormal object recognition can be significantly enhanced. Ultimately, the proposed RSE-ASPP-SqueezeNet network achieves high-precision detection of abnormal items in the data center inspection robot's anomaly detection task.

An effective picking point localization method for multi-posture lotus pods based on three-view depth vision observation

Different lotus pod individuals have different growth postures, and conventional single-view observation makes it difficult to ensure the visibility of the stalk’s picking segment of all lotus pods, thus affecting the efficiency and quality of automatic picking. To solve this problem, this study proposes a picking point localization method for multi-posture lotus pods based on three-view depth vision observation. It includes three steps: first, the RGB-D (red–green–blue–depth) images of lotus pods are collected from three viewing directions based on the designed three-view depth vision observation scheme. Second, instance segmentation on the lotus pods and stalks in the collected images is performed to obtain mask data. Third, the final lotus pod picking point coordinates and picking vector data are obtained using the proposed lotus pod picking information calculation algorithm. The algorithm comprises the two-dimensional image feature point calculation, single-view picking information calculation, and three-view picking information data synthesis. The three-view observation test of lotus pods in a planting environment shows that the proposed three-view observation scheme can meet the picking segment observation needs of lotus pods in any posture. In addition, lotus pod localization tests were conducted based on a specifically designed localization test bench. The proposed localization method achieved a localization success rate of 98.00 % and 90.86 % for single and multiple lotus pods, respectively. The results verify the feasibility of the proposed localization method for calculating the picking information of arbitrary lotus pods, which can provide essential support for the practical development of automatic lotus pod harvesting technology.

Effects of scenery frame on visual depth perception in classical Chinese gardens: A case study of the Lvyin Pavilion in Lingering Garden

Visual depth (distance) perception is a fundamental aspect of environmental cognition, as it allows people to judge the spatial scale of their surroundings. However, estimating the depth of classical Chinese gardens is challenging, especially from static viewpoints that frame the scenery. Previous studies have examined how the internal components of the scenery frame affect depth perception. Still, the role of the frame and its peripheral information as environmental background have been largely overlooked. This study investigates how depth perception at viewpoints is influenced by viewing position displacement, frame geometry, and environmental context. The authors created nine stimulus materials in a cave virtual reality environment (three image treatments × three positions). Seventy-one participants were asked to evaluate depth perception using the magnitude estimation and adjustment methods. Their eye movement behavior was also recorded using an eye-movement instrument (SensoMotoric Instruments (SMI) eye-tracking glasses, 120 Hz). The results showed that participants could perceive spatial depth differences between viewing positions even when the internal viewpoint displacement was small; frame shape did not significantly affect depth perception and gaze behavior; and peripheral visual information of the frame enhanced depth perception significantly. Moreover, the form of the environmental background, especially the position of the scenery window, strongly guided the participants’ gaze. These findings suggest that ambient visual information significantly impacts environmental experience, which landscape designers should consider.

Prediction of abnormal gait behavior of lower limbs based on depth vision

Abstract As a kind of lower-limb motor assistance device, the intelligent walking aid robot plays an essential role in helping people with lower-limb diseases to carry out rehabilitation walking training. In order to enhance the safety performance of the lower-limb walking aid robot, this study proposes a deep vision-based abnormal lower-limb gait prediction model construction method for the problem of abnormal gait prediction of patients’ lower limbs. The point cloud depth vision technique is used to acquire lower-limb motion data, and a multi-posture angular prediction model is trained using long and short-term memory networks to build a model of the user’s lower-limb posture characteristics during normal walking as well as a real-time lower-limb motion prediction model. The experimental results indicate that the proposed lower-limb abnormal behavior prediction model is able to achieve a 97.4% prediction rate of abnormal lower-limb movements within 150 ms. Additionally, the model demonstrates strong generalization ability in practical applications. This paper proposes further ideas to enhance the safety performance of lower-limb rehabilitation robot use for patients with lower-limb disabilities.

A novel concurrent learning-based fixed-time convergent visual depth observer for weakly persistently exciting perspective dynamical systems

The problem of synthesizing a fixed-time depth observer based on monocular image feedback is considered in this study. The key challenge lies in the fact that the perspective dynamical system used to model visual motion is typically characterized as being weakly persistently exciting, which complicates observer synthesis. Furthermore, the key to achieving the task objective (safe obstacle avoidance, for instance) lies in the synthesis of a depth observer that achieves rapid convergence of the (static) obstacle depth estimate to the ground truth in a known fixed-time. To address these challenges, and in contrast with prior schemes that rely on a motion-restrictive persistency of excitation (PE) condition for ensuring exponential convergence, a novel adaptive observer framework is considered in this study that incorporates a concurrent learning (CL) term for ensuring fixed-time observer convergence. In particular, the use of concurrent learning allows for the synthesis of a relaxed finite-time excitation condition that relies on historical data recorded over a dynamic sliding window in the recent past that the proposed observer relies on to ensure fixed-time convergence. Thus, a continuous-time reduced-order observer formulation is presented that relies on camera motion data to achieve fixed-time convergence to a uniform ultimate bound for a suitably large choice of the observer gains. Experimental results are used to demonstrate the efficacy of the proposed scheme in the presence of significant measurement noise. A performance comparison study is also undertaken to demonstrate superior performance of the proposed scheme compared to leading alternative designs. Finally, the practical applicability of the proposed scheme is verified by incorporating the proposed scheme within a reactive navigation scheme to accomplish obstacle avoidance. By incorporating a suitably informative CL term within the observer framework, the proposed scheme eliminates the need to rely on a difficult-to-verify PE condition, thus rendering it more suitable for practical applications like visual target-tracking and visual servo control.

Cardboard packaging minimization: Autonomous box selection by RGB-depth vision system and complementary deep learning classification

Amidst the dramatic expansion of online shopping, wasted cardboard packaging has become a significant challenge in modern-day society. In this paper, we propose an innovative approach to address the issue of box packaging waste by efficiently selecting an optimal cardboard box for a set of items. Employing depth vision and deep learning classification techniques, the proposed system analyzes depth images to compute an optimal cuboid for unpackaged items while accounting for fragility characteristics. The system, facilitated by two Intel RealSense RGB-D cameras, first captures the overhead and sideview images to determine the dimensions of an unknown object. Then, the developed vision system utilizes averaging and morphological opening filters to denoise the acquired images, calculates the difference between the background image and object image, and employs thresholding by Otsu’s algorithm to find the minimum cuboid of the irregular object. After, a two-step deep learning model is used to determine the general category and characteristics of an object, considering factors, such as fragility, that may necessitate wrapping or separate packaging. Finally, a modified largest area first fit algorithm uses the outputted dimensions for irregular-shaped objects and packaged objects to determine the optimal fitting box from the standard Amazon box catalog. The computer simulation shows that the proposed method can be implemented in a cardboard packaging warehouse for optimal box selection, effectively minimizing cardboard waste.

Visual information and appearance: The impact of visual attributes of user-generated photos on review helpfulness

Images have become integral to consumers’ sharing of consumption experiences due to their abilities of carrying rich and vivid information. Drawing from the perspective of information theory and the elaboration likelihood model (ELM), this study investigates impacts of visual information and visual appearance of user-generated photos (UGPs) on customers’ perceived review helpfulness. We utilize deep learning techniques to calculate the breadth and depth of photos (visual information), and evaluate the aesthetic value of the photos (visual appearance). By collecting a substantial amount of review, reviewer and restaurant information from the Yelp platform, our results demonstrate that the visual breadth and visual depth of review photos have a significant positive impact on review helpfulness. Notably, reviewer status and review length moderate these effects. These insights offer valuable strategies for both restaurant managers and online restaurant platforms regarding UGP management.

Multisensory Extended Reality Applications Offer Benefits for Volumetric Biomedical Image Analysis in Research and Medicine.

3D data from high-resolution volumetric imaging is a central resource for diagnosis and treatment in modern medicine. While the fast development of AI enhances imaging and analysis, commonly used visualization methods lag far behind. Recent research used extended reality (XR) for perceiving 3D images with visual depth perception and touch but used restrictive haptic devices. While unrestricted touch benefits volumetric data examination, implementing natural haptic interaction with XR is challenging. The research question is whether a multisensory XR application with intuitive haptic interaction adds value and should be pursued. In a study, 24 experts for biomedical images in research and medicine explored 3D medical shapes with 3 applications: a multisensory virtual reality (VR) prototype using haptic gloves, a simple VR prototype using controllers, and a standard PC application. Results of standardized questionnaires showed no significant differences between all application types regarding usability and no significant difference between both VR applications regarding presence. Participants agreed to statements that VR visualizations provide better depth information, using the hands instead of controllers simplifies data exploration, the multisensory VR prototype allows intuitive data exploration, and it is beneficial over traditional data examination methods. While most participants mentioned manual interaction as the best aspect, they also found it the most improvable. We conclude that a multisensory XR application with improved manual interaction adds value for volumetric biomedical data examination. We will proceed with our open-source research project ISH3DE(Intuitive Stereoptic Haptic 3D Data Exploration) to serve medical education, therapeutic decisions, surgery preparations, or research data analysis.

Binocular receptive-field construction in the primary visual cortex

ON and OFF thalamic afferents from the two eyes converge in the primary visual cortex to form binocular receptive fields. The receptive fields need to be diverse to sample our visual world but also similar across eyes to achieve binocular fusion. It is currently unknown how the cortex balances these competing needs between receptive-field diversity and similarity. Our results demonstrate that receptive fields in the cat visual cortex are binocularly matched with exquisite precision for retinotopy, orientation/direction preference, orientation/direction selectivity, response latency, and ON-OFF polarity/structure. Specifically, the average binocular mismatches in retinotopy and ON-OFF structure are tightly restricted to 1/20 and 1/5 of the average receptive-field size but are still large enough to generate all types of binocular disparity tuning. Based on these results, we conclude that cortical receptive fields are binocularly matched with the high precision needed to facilitate binocular fusion while allowing restricted mismatches to process visual depth.

Flexible delivery by an autonomous robot in a secure building

In industrial settings, autonomous robotic systems have become more common. Companies use autonomous robots to optimise repetitive, time-consuming tasks and improve efficiency. Strategically integrating robots frees up human resources from manual tasks to boost workplace efficiency and productivity. As they pursue Industry 4.0 goals, companies need autonomous robots to meet strict security standards. This depends on preserving confidentiality regarding sensitive information and protecting personnel health and safety. The mobile robot in this experiment aimed to simplify the transportation of standard load carriers, such as plastic boxes used in Kanban storage racks. A controlled lab environment was needed to recreate the robot’s working environment. To streamline delivery, an innovative optical camera and depth vision were added. With this technology configuration, the robot could perform tasks independently. Limiting data retrieval helped the depth camera distinguish conventional boxes. The main project achievement was proving that an autonomous guided vehicle could transport plastic boxes while protecting company data. This method replaced manual part retrieval from secure locations, improving operational efficiency and reinforcing data security.

Structure-Function Relationship in Keratoconus: Spatial and Depth Vision.

The purpose of this study was to determine changes in spatial and depth vision with increasing severity of keratoconus and to model the structure-function relationship to identify distinct phases of loss in visual function with disease severity. Best-spectacle corrected, monocular high-contrast visual acuity, contrast sensitivity function (CSF) and stereoacuity of 155 cases (16-31years) with mild to advanced bilateral keratoconus was determined using standard psychophysical tests. Disease severity was quantified using the multimetric D-index. The structure-function relationship was modeled using linear, positive exponential, negative exponential, and logistic nonlinear regression equations. The logistic regression model explained the highest proportion of variance for spatial vision, without bias in the residual plots (R2 ≥ 66%, P < 0.001). Visual acuity showed a distinct ceiling phase and a steeper loss rate with increasing D-index (1.8units/D-index) in this model. The area under the CSF lacked this ceiling phase and had a shallower loss rate (0.28units/D-index). Stereoacuity loss with D-index was poorly explained by all models tested (P ≤ 0.2). Cases with lower and bilaterally symmetric D-index had better stereoacuity (181.6-376 arc seconds) than those with higher D-index (>400 arc second); both were significantly poorer than controls (approximately 30 arc second). Vision loss in keratoconus varies with the visual function parameter tested. Contrast sensitivity may be an earlier indicator of spatial vision loss than visual acuity. Depth perception is significantly deteriorated from very early stages of the disease. The study outcomes may be used to forecast longitudinal vision loss in keratoconus and to apply appropriate interventions for timely preservation/enhancement of vulnerable visual functions.

Multimodal feedback systems for smart laser osteotomy: Depth control and tissue differentiation.

The study aimed to improve the safety and accuracy of laser osteotomy (bone surgery) by integrating optical feedback systems with an Er:YAG laser. Optical feedback consists of a real-time visual feedback system that monitors and controls the depth of laser-induced cuts and a tissue sensor differentiating tissue types based on their chemical composition. The developed multimodal feedback systems demonstrated the potential to enhance the safety and accuracy of laser surgery. The proposed method utilizes a laser-induced breakdown spectroscopy (LIBS) system and long-range Bessel-like beam optical coherence tomography (OCT) for tissue-specific laser surgery. The LIBS system detects tissue types by analyzing the plasma generated on the tissue by a nanosecond Nd:YAG laser, while OCT provides real-time monitoring and control of the laser-induced cut depth. The OCT system operates at a wavelength of 1288 ± 30 nm and has an A-scan rate of 104.17 kHz, enabling accurate depth control. Optical shutters are used to facilitate the integration of these multimodal feedback systems. The proposed system was tested on five specimens of pig femur bone to evaluate its functionality. Tissue differentiation and visual depth feedback were used to achieve high precision both on the surface and in-depth. The results showed successful real-time tissue differentiation and visualization without any visible thermal damage or carbonization. The accuracy of the tissue differentiation was evaluated, with a mean absolute error of 330.4 μm and a standard deviation of ±248.9 μm, indicating that bone ablation was typically stopped before reaching the bone marrow. The depth control of the laser cut had a mean accuracy of 65.9 μm with a standard deviation of ±45 μm, demonstrating the system's ability to achieve the pre-planned cutting depth. The integrated approach of combining an ablative laser, visual feedback (OCT), and tissue sensor (LIBS) has significant potential for enhancing minimally invasive surgery and warrants further investigation and development.

Self-supervised deep monocular visual odometry and depth estimation with observation variation

Recent research in deep learning has greatly advanced the development of monocular visual depth estimation and odometry. These learning-based methods predominantly leverage neural networks to extract prevalent high-dimensional features for regression-based estimation. However, the consistency of observations is still regarded as a core challenge that limits the generalization ability. We attribute this challenge to the significant inconsistency in feature levels due to changes in the environment or motion, collectively referred to as observation variation. To this end, we propose a novel monocular depth estimation and a pose estimation network with attention mechanism to recalibrate features for different environment. The depth and ego-motion estimation tasks are formulated by coupling together in an end-to-end reconstruction problem, and attention modules adaptively recalibrate specific feature responses. Furthermore, we propose a simple sliding window optimization without RNN-based architecture, which is introduced to overcome the movement speed variation and error accumulation. Finally, we collect an observation variation dataset with environmental and motion changes. Evaluations of depth and pose estimation demonstrate that our methods effectively overcomes these observation variation challenges and achieves better performance. Our project will be available at https://github.com/alikesierzhao/OV-SfmLearner.

Activation of p38 MAPK hinders the reactivation of visual cortical plasticity in adult amblyopic mice

ObjectiveTo investigate the impact of p38 mitogen-activated protein kinase (MAPK) signaling on reactivating visual cortical plasticity in adult amblyopic mice. Materials and methodsReverse suture (RS), environment enrichment (EE), and combined with left intracerebroventricular injection of p38 MAPK inhibitor (SB203580, SB) or p38 MAPK agonist (dehydrocorydaline hydrochloride, DHC) were utilized to treat adult amblyopic mice with monocular deprivation (MD). The visual water task, visual cliff test, and Flash visual-evoked potential were used to measure the visual function. Then, Golgi staining and transmission electron microscopy were used to assess the reactivation of structural plasticity in adult amblyopic mice. Western blot and immunohistochemistry detected the expression of ATF2, PSD-95, p38 MAPK, and phospho-p38 MAPK in the left visual cortex. ResultsNo statistically significant difference was observed in the visual function in each pre-intervention group. Compared to pre-intervention, the visual acuity of deprived eyes was improved significantly, the impairment of visual depth perception was alleviated, and the P wave amplitude and C/I ratio were increased in the EE + RS, the EE + RS + SB, and the EE + RS + DMSO groups, but no significant difference was detected in the EE + RS + DHC group. Compared to EE + RS + DHC group, the density of dendritic spines was significantly higher, the synaptic density of the left visual cortex increased significantly, the length of the active synaptic zone increased, and the thickness of post-synaptic density (PSD) thickened in the left visual cortex of EE + RS, EE + RS + SB, and EE + RS + DMSO groups. And that, the protein expression of p-p38 MAPK increased while that of PSD-95 and ATF2 decreased significantly in the left visual cortex of the EE + RS + DHC group mice. ConclusionRS and EE intervention improved the visual function and synaptic plasticity of the visual cortex in adult amblyopic mice. However, activating p38 MAPK hinders the recovery of visual function by upregulating the phosphorylation of p38 MAPK and decreasing the ATF2 protein expression.

Research on Road Sign Detection and Visual Depth Perception Technology for Mobile Robots

To accomplish the task of detecting and avoiding road signs by mobile robots for autonomous running, in this paper, we propose a method of road sign detection and visual depth perception based on improved Yolov5 and improved centroid depth value filtering. First, the Yolov5 model has a large number of parameters, a large computational volume, and a large model size, which is difficult to deploy to the CPU side (industrial control computer) of the robot mobile platform. To solve this problem, the study proposes a lightweight Yolov5-SC3FB model. Compared with the original Yolov5n model, the Yolov5-SC3FB model only loses lower detection accuracy, the parameter volume is reduced to 0.19 M, the computational volume is reduced to 0.5 GFLOPS, and the model size is only 0.72 MB, making it easy to deploy on mobile robot platforms. Secondly, the obtained depth value of the center point of the bounding box is 0 due to the influence of noise. To solve this problem, we proposed an improved filtering method for the depth value of the center point in the study, and the relative error of its depth measurement is only 2%. Finally, the improved Yolov5-SC3FB model is fused with the improved filtering method for acquiring centroid depth values and the fused algorithm is deployed to the mobile robot platform. We verified the effectiveness of this fusion algorithm for the detection and avoidance of road signs of the robot. Thus, it can enable the mobile robot to correctly perceive the environment and achieve autonomous running.

Regulatory role of the p38 MAPK/ATF2 signaling pathway in visual function and visual cortical plasticity in mice with monocular deprivation

BackgroundThis study aimed to examine the role of the p38 mitogen-activated protein kinase (MAPK)/ activating transcription factor 2 (ATF2) signaling in visual function impairment and visual cortical plasticity in mice with monocular deprivation (MD). MethodsVisual behavioral tests, including visual water task, visual cliff test, and flash visual evoked potential, were performed on each group. We studied the density of dendritic spines and the synaptic ultrastructure by Golgi staining and transmission electron microscope. We performed Western blot and immunohistochemistry and detected the expression of ATF2, PSD-95, p38 MAPK, and phosphor-p38 MAPK in the left visual cortex. ResultsIn the MD + SB group, the visual acuity in deprived eyes substantially improved, the impairment of visual depth perception was alleviated, and the P wave amplitude and C/I ratio increased. The density of dendritic spines and the numerical density of synapses increased significantly, the width of the synaptic cleft decreased significantly, and the length of the active synaptic zone and the thickness of post-synaptic density (PSD) increased substantially. The protein expression of phosphor-p38 MAPK decreased, whereas that of PSD-95 and ATF2 increased significantly. ConclusionsInhibiting the phosphorylation of p38 MAPK and negative feedback upregulated ATF2 expression, alleviated damage to visual function, and protected against synaptic plasticity in mice with MD.

Targeted reaching with monocular depth information and haptic feedback: Comparing between monocular patients and normally sighted observers

Monocular blindness impairs visual depth perception, yet patients seldom report difficulties in targeted actions like reaching, walking, or driving. We hypothesized that by utilizing monocular depth information and calibrating actions with haptic feedback, monocular patients can perceive egocentric distance and perform targeted actions. We compared targeted reaching in monocular patients, monocular-viewing, and binocular-viewing normal controls. Sixty observers reached either a far or a near target, calibrating reaches to the near target with accurate or false feedback while leaving reaches to the far target uncalibrated. Reaching accuracy and precision were analyzed. Results indicated no difference in reaching accuracy between monocular patients and normal controls; all groups initially underestimated distances before until calibration. Monocular patients responded to calibration sensitively, achieving accuracy in calibrated reaches and generalizing this effect to uncalibrated distances. Thus, with monocular depth information and haptic feedback, monocular patients could perceive distance and accomplish targeted reaching.

What Makes the Detection of Movement Different Within the Autistic Traits Spectrum? Evidence From the Audiovisual Depth Paradigm.

Atypical sensory processing is now considered a diagnostic feature of autism. Although multisensory integration (MSI) may have cascading effects on the development of higher-level skills such as socio-communicative functioning, there is a clear lack of understanding of how autistic individuals integrate multiple sensory inputs. Multisensory dynamic information is a more ecological construct than static stimuli, reflecting naturalistic sensory experiences given that our environment involves moving stimulation of more than one sensory modality at a time. In particular, depth movement informs about crucial social (approaching to interact) and non-social (avoiding threats/collisions) information. As autistic characteristics are distributed on a spectrum over clinical and general populations, our work aimed to explore the multisensory integration of depth cues in the autistic personality spectrum, using a go/no-go detection task. The autistic profile of 38 participants from the general population was assessed using questionnaires extensively used in the literature. Participants performed a detection task of auditory and/or visual depth moving stimuli compared to static stimuli. We found that subjects with high-autistic traits overreacted to depth movement and exhibited faster reaction times to audiovisual cues, particularly when the audiovisual stimuli were looming and/or were presented at a fast speed. These results provide evidence of sensory particularities in people with high-autistic traits and suggest that low-level stages of multisensory integration could operate differently all along the autistic personality spectrum.

Egyptian Lullaby by Zeena M. Pliska (review)

Reviewed by: Egyptian Lullaby by Zeena M. Pliska Kara Forde Pliska, Zeena M. Egyptian Lullaby; illus. by Hatem Aly. Roaring Brook, 2023 [40p] Trade ed. ISBN 9781250222497 $18.99 Reviewed from digital galleys R 3-6 yrs See this month's Big Picture, p. 245, for review. While our book's young narrator always loves visits from great Auntie Fatma, their time together is also a reminder of the home the child left behind. The sounds of Arabic conversation filling the house bring both a sense of comfort and melancholy, and even the steaming cups of sahlab the family drinks together taste just a bit different. While Auntie avows that "once you drink from the Nile, you'll always return," the young one's family has yet to go back. Thankfully, Auntie soothes the little one's longing for their homeland by singing the sounds of Egypt every night before bed: "This is the boat/that glides on the Nile,/which flows through the city./Swish, swoosh, swish." Pliska's text begins softly and then steadily builds as Auntie Fatma's song relays the melodic din of Cairo; a chorus of honking horns, the muezzin's call to prayer from the mosque, a street vendor selling sweet watermelons, plus donkeys, dogs, and kids comprise the cacophony of this resonant story, and the compounding repetition of the stanzas peppered with onomatopoeic phrases gives the story the cadence of a lullaby. Stylistically unique illustrations bring the city to life, as digital drawings and scanned textures experiment with the visual depth of field. Notes from the author and illustrator illuminate the personal experiences that led them to create this love letter to everyday Egyptian life, and a glossary provides pronunciations and translations for the Arabic terms that are integrated into the narrative. Copyright © 2023 The Board of Trustees of the University of Illinois

A railway track reconstruction method using robotic vision on a mobile manipulator: A proposed strategy

Autonomous robot integration in railways infrastructure maintenance accelerates the digitization and intelligence of infrastructure survey & maintenance, providing high-efficiency and low-cost execution. This paper proposes a health assessment based on 3D reconstruction technology for railway track maintenance using a mobile robotic sensing platform. By combining multiple sensing and taking advantage of a robotic manipulator, a digital model of the target track components is built by a robot-actuated vision system which provides better 3D structural and surface condition reconstruction. Global geo-location and surrounding laser scanning are integrated to reinforce the digital completeness of the model for intelligent management. The new method consists of the following steps: First, according to scheduled maintenance tasks, a Robotics Inspection and Repair System (RIRS) navigates to the task location and uses the onboard depth camera for positioning. Then robot-mounted vision system is guided with an automated trajectory to build the 3D reconstruction of the track or repair object using the vision modeling technique. Finally, the 3D reconstructed model is fused with surrounding mapping of depth vision and Lidar scanning. Both laboratory tests and a realistic track test validated the feasibility of the proposed method by creating an accurate 3D reconstructed model. The modeled rail steel section size is quantitively compared with the ground truth in dimension, demonstrating good accuracy with a size error of less than 0.3 cm. The main contribution includes: (1) unmanned automatic 3D reconstruction by a robotic mobile manipulator, (2) the technique trims the reconstruction details & data to the specific maintenance goal or components, which supports the infrastructure maintenance towards the high-detailed & target-oriented digital management. This combination strategy of robotic automation and sensor fusion lies down a promising foundation for automated digital twin establishment for railway maintenance with autonomous RIRS, and upgrades technology readiness and digital intelligence for maintenance management