3D Surface Generation Research Articles

An Artificial Intelligence-Based System to Assess Nutrient Intake for Hospitalized Patients

Reducing the risk of disease-related malnutrition requires routine monitoring of nutritional intake in hospitalised patients. There is still a definite need for a more dependable and totally automated way to estimate nutrient intake, since this might increase data accuracy and lessen the burden on participants as well as health expenditures. In this research, we offer an innovative artificial intelligence (AI) method that uses simple RGB Depth (RGBD) picture pair processing to estimate nutritional intake effectively. The system contains a unique multi-task contextual network for food segmentation, an algorithm for 3D surface generation, and a few-shot learning-based classifier for food identification. This enables entirely automated calculation of the nutritional intake for each meal via the sequential segmentation, identification, and estimation of the eaten food volume. A new database specifically created for the system's development and testing, complete with 322 meals' nutritional information and photos of each, has also been built. The predicted nutritional intake outperforms currently available methodologies for nutrient intake evaluation, according to experimental data, which reveal that it is well linked (> 0.91) to the ground truth and exhibits extremely modest mean relative errors ( 20%).

Surface Reconstruction Assessment in Photogrammetric Applications

The image-based 3D reconstruction pipeline aims to generate complete digital representations of the recorded scene, often in the form of 3D surfaces. These surfaces or mesh models are required to be highly detailed as well as accurate enough, especially for metric applications. Surface generation can be considered as a problem integrated in the complete 3D reconstruction workflow and thus visibility information (pixel similarity and image orientation) is leveraged in the meshing procedure contributing to an optimal photo-consistent mesh. Other methods tackle the problem as an independent and subsequent step, generating a mesh model starting from a dense 3D point cloud or even using depth maps, discarding input image information. Out of the vast number of approaches for 3D surface generation, in this study, we considered three state of the art methods. Experiments were performed on benchmark and proprietary datasets of varying nature, scale, shape, image resolution and network designs. Several evaluation metrics were introduced and considered to present qualitative and quantitative assessment of the results.

A VERSATILE MULTI-CAMERA SYSTEM FOR 3D ACQUISITION AND MODELING

Abstract. Image-based 3D models generation typically involves three stages, namely: 2D image acquisition, data processing, and 3D surface generation and editing. The availability of different easy-to-use and low-cost image acquisition solutions, combined with open-source or commercial processing tools, has democratized the 3D reconstruction and digital twin generation. But high geometric and texture fidelity on small- to medium-scale objects as well as integrated commercial system for mass 3D digitization are not available. The paper presents our effort to build such a system, i.e. a market-ready multi-camera solution and a customized reconstruction process for mass 3D digitization of small to medium objects. The system is realized as a joint work between industrial and academic partners, in order to employ the latest technologies for the needs of the market. The proposed versatile image acquisition and processing system pushes to the limits the 3D digitization pipeline combining a rigid capturing system with photogrammetric reconstruction methods.

Modeling of residual tool mark formation and its influence on the optical performance of KH2PO4 optics repaired by micro-milling

In the micro-milling repair process of surface defects on KH2PO4 (KDP) optics, residual tool marks are inevitably introduced on repaired surfaces. These tool marks could present great potential risks in lowering laser-induced damage thresholds of repaired KDP optics, which has not been reported in the previous works. In this work, the formation process of micro-milled tool marks was initially modeled and then the effect of these residual marks on the optical performance of KDP optics was investigated theoretically and experimentally. A 3D surface generation model for repair contours is proposed to predict the profiles of residual tool marks on micro ball-end milled surfaces, and then a numerical FEM model adopting the calculated profiles of residual tool marks is established to investigate the light intensification inside KDP optics based on electromagnetic theory. The relationship between repair machining processes, tool marks profiles and light intensifications as well as laser damage resistance of repaired KDP optics, has been discussed comprehensively and systematically. Practical repair experiments and laser damage tests verified the theoretical results very well. It revealed that the residual height of tool marks can induce apparent diffraction effect, consequently causing severe light intensification inside KDP optics. The micro milling strategies (e.g. a combination of layer-milling and spiral milling paths) can also exert a positive role in improving the laser damage resistance of KDP optics. An optimized repair process flow, adopting layer-milling path as rough milling and employing spiral-milling with path intervals of 10∼15 µm as fine milling, is recommended for the future practical engineering repair of full-aperture KDP optics in ICF facilities.

A novel 3D surface generation model for micro milling based on homogeneous matrix transformation and dynamic regenerative effect

A novel 3D surface generation modelling method for micro milling which considers the effect of machining nonlinear dynamics is proposed based on homogeneous matrix transformation. Considering the effects of machining process kinematics, tool run-out and the nonlinear dynamic regenerative effect of the machining system, the relationship between machining process and surface topography is established. On this basis, three typical machining cases, namely static stable, dynamic stable and unstable machining, are investigated in this paper. The machining experiment results demonstrate that the proposed surface generation model can accurately predict surface topography generation and roughness values with different machining conditions. Thus, it can be used not only to optimize the machining parameters for improving the micro-milled surface quality but also to predict the surface topography under pre-defined machining parameters.

3D SURFACE GENERATION FROM AERIAL THERMAL IMAGERY

Abstract. Aerial thermal imagery has been recently applied to quantitative analysis of several scenes. For the mapping purpose based on aerial thermal imagery, high accuracy photogrammetric process is necessary. However, due to low geometric resolution and low contrast of thermal imaging sensors, there are some challenges in precise 3D measurement of objects. In this paper the potential of thermal video in 3D surface generation is evaluated. In the pre-processing step, thermal camera is geometrically calibrated using a calibration grid based on emissivity differences between the background and the targets. Then, Digital Surface Model (DSM) generation from thermal video imagery is performed in four steps. Initially, frames are extracted from video, then tie points are generated by Scale-Invariant Feature Transform (SIFT) algorithm. Bundle adjustment is then applied and the camera position and orientation parameters are determined. Finally, multi-resolution dense image matching algorithm is used to create 3D point cloud of the scene. Potential of the proposed method is evaluated based on thermal imaging cover an industrial area. The thermal camera has 640×480 Uncooled Focal Plane Array (UFPA) sensor, equipped with a 25 mm lens which mounted in the Unmanned Aerial Vehicle (UAV). The obtained results show the comparable accuracy of 3D model generated based on thermal images with respect to DSM generated from visible images, however thermal based DSM is somehow smoother with lower level of texture. Comparing the generated DSM with the 9 measured GCPs in the area shows the Root Mean Square Error (RMSE) value is smaller than 5 decimetres in both X and Y directions and 1.6 meters for the Z direction.

Accurate and occlusion-robust multi-view stereo

This paper proposes an accurate multi-view stereo method for image-based 3D reconstruction that features robustness in the presence of occlusions. The new method offers improvements in dealing with two fundamental image matching problems. The first concerns the selection of the support window model, while the second centers upon accurate visibility estimation for each pixel. The support window model is based on an approximate 3D support plane described by a depth and two per-pixel depth offsets. For the visibility estimation, the multi-view constraint is initially relaxed by generating separate support plane maps for each support image using a modified PatchMatch algorithm. Then the most likely visible support image, which represents the minimum visibility of each pixel, is extracted via a discrete Markov Random Field model and it is further augmented by parameter clustering. Once the visibility is estimated, multi-view optimization taking into account all redundant observations is conducted to achieve optimal accuracy in the 3D surface generation for both depth and surface normal estimates. Finally, multi-view consistency is utilized to eliminate any remaining observational outliers. The proposed method is experimentally evaluated using well-known Middlebury datasets, and results obtained demonstrate that it is amongst the most accurate of the methods thus far reported via the Middlebury MVS website. Moreover, the new method exhibits a high completeness rate.

Fast tool servo diamond turning of optical freeform surfaces for rear-view mirrors

Optical freeform surfaces have been applied in optoelectronic systems more and more widely, but it is difficult to fabricate this kind of freeform surface optics with high quality and high efficiency. Ultra-precision diamond turning machining technology based on a fast tool servo system is an effective and precise process for fabricating optical freeform surfaces. This paper develops a novel fast tool servo system, which has a total stroke of 30 mm and a maximum acceleration of 920 m/s2. In this paper, we develop a theoretical and experimental investigation on the 3D surface generation of a rear-view mirror which is a typical optical freeform surface in single-point diamond tuning using a fast tool servo system. The experimental results are tested and analyzed, which indicate that the performances are satisfied. The field of view obtained by the designed rear-view mirror is much larger than the flat mirror, while the distortion is minimal.

Fabrication of off-axis aspheric surfaces using a slow tool servo

Off-axis aspheric surfaces have been widely applied in optical systems, but it is difficult to fabricate this kind of optical surface with high quality. Slow tool servo is one of the methods for turning off-axis aspheric surfaces on-axis, which have a large travel range without an additional device. This paper presents a theoretical and experimental investigation on the 3D surface generation in single point diamond tuning using slow tool servo. The form error of off-axis paraboloid caused by tool centering error is analyzed and verified by cutting experiment.

A computer vision system for diagnosing scoliosis using moiré images

For young people, scoliosis deformities are an evolving process which must be detected and treated as early as possible. The moiré technique is simple, inexpensive, not aggressive and especially convenient for detecting spinal deformations. Doctors make their diagnosis by analysing the symmetry of fringes obtained by such techniques. In this paper, we present a computer vision system for help diagnosing spinal deformations using noisy moiré images of the human back. The approach adopted in this paper consists of extracting fringe contours from moiré images, then localizing some anatomical features (the spinal column, lumbar hollow and shoulder blades) which are crucial for 3D surface generation carried out using Mota's relaxation operator. Finally, rules furnished by doctors are used to derive the kind of spinal deformation and to yield the diagnosis. The proposed system has been tested on a set of noisy moiré imagcs, and the experimental result have shown its robustness and reliability for the recognition of most scoliosis deformities.