Image Registration Techniques Research Articles

Combining convolution neural networks with long‐short term memory layers to predict Parkinson's disease progression

AbstractParkinson's disease (PD) is a neurological condition that occurs in nearly 1% of the world's population. The disease is manifested by a sharp drop in dopamine production, resulting from the death of the related producing cells in an area of the midbrain called the substantia nigra. Early diagnosis and accurate staging of the disease are essential to apply the appropriate therapeutic approaches to slow cognitive and motor decline. At present, there is not a singular blood test or biomarker accessible for diagnosing PD or monitoring the progression of its symptoms. Clinical professionals identify the disease by assessing the symptoms, which, however, may vary from case to case, as well as their progression speed. Magnetic resonance imaging (MRIs) have been used for the past three decades to diagnose and distinguish between PD and other neurological conditions.However, to the best of our knowledge, no neural network models have been designed to identify the disease stage. This paper aims to fill this gap. Using the “Parkinson's Progression Markers Initiative” dataset, which reports the patient's MRI and an indication of the disease stage, we developed a model to identify the level of progression. The images and the associated scores were used for training and assessing different deep learning models. Our analysis distinguished four distinct disease progression levels based on a standard scale (Hoehn and Yah scale). The final architecture consists of the cascading of a 3D‐CNN network, adopted to reduce and extract the spatial characteristics of the MRI for efficient training of the successive LSTM layers, aiming at modeling the temporal dependencies among the data. Before training the model, the patient's MRI is preprocessed to correct acquisition errors by applying image registration techniques, to extract irrelevant content from the image, such as nonbrain tissue (e.g., skull, neck, fat). We also adopted template‐based data augmentation techniques to obtain a balanced dataset about progression classes. Our results show that the proposed 3D‐CNN + LSTM model achieves state‐of‐the‐art results by classifying the elements with 91.90 as macro averaged OVR AUC on four classes.

Virtual reassembling of 3D fragments for the data-driven analysis of fracture mechanisms in multi-component materials

This paper introduces a novel method for characterizing fracture mechanisms in multi-component materials using 3D image data gained by X-ray computed tomography (CT) measurements. In mineral liberation, the understanding of these mechanisms is crucial, particularly whether fractures occur along the boundaries of mineral phases (intergranular fracture) and/or within mineral phases (transgranular fracture). Conventional techniques for analyzing fracture mechanisms are focused on globally comparing the surface exposure of mineral phases extracted from image measurements before and after fracture. Instead, we present a virtual reassembling algorithm based on image registration techniques, which is applied to 3D data of multi-component materials before and after fracture in order to determine and characterize the individual fracture surfaces. This enables us to conduct a local quantitative analysis of fracture mechanisms by voxelwise comparing adjacent regions at fracture surfaces. A quantitative analysis of fracture mechanisms is especially important in the context of recycling processes. As primary deposits are decreasing worldwide and the focus is shifting on reducing wast, the interest in secondary raw materials has increased. However, lower-concentrated, but valuable materials are often overlooked in recycling routes designed for higher-concentration materials and dissipate. Therefore, efforts are being made to enrich valuable elements, such as lithium, as engineered artificial minerals through pyrometallurgical processes. The subsequent liberation through comminution processes, such as crushing, is essential for the recovery of valuable minerals. A better understanding of crushing processes, especially fracture mechanisms in slags, is crucial for the success of recycling. The reassembling algorithm presented in this paper is evaluated through a simulation study, followed by an application to a naturally occurring ore and a slag resulting from a recycling process.

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

Tracking Fat Grafts by Magnetic Resonance Imaging: A Comparative Study of Adolescent and Adult Patients with Stable Localized Scleroderma.

The optimal timing of reconstruction for patients with facial localized scleroderma is uncertain. The purpose of this study was to compare the outcomes of autologous fat transplantation in adolescent and adult patients with stable localized scleroderma. Adolescent (age 10-19 years) and adult (age >19 years) patients with no previous surgery were enrolled (n = 10, each group). Preoperative magnetic resonance imaging (MRI), blood tests and dermatological assessments were used for disease activity assessment. All patients underwent autologous fat transplantation for anatomic facial fat restoration with preoperative MRI planning. Preoperative, immediate and one-year postoperative 3D Dixon MRI scans with image registration and fusion techniques were used for fat graft tracking. Patient satisfaction was assessed with a 5-point Likert scale. There was no significant difference in sex, body mass index, disease severity, or volume of injected fat between the two groups (p > 0.05), except for age (p < 0.05). The one-year postoperative fat graft retention rate was not significantly different, with 36.6 ± 2.4% (ranging from 25.3 to 49.3%) in the adolescent group and 32.9 ± 1.7% (ranging from 27.3 to 40.1%) in the adult group (p > 0.05). Surgical outcomes were favorable in all patients, with satisfaction scores of 3.8 ± 0.2 points in the adolescent group and 3.6 ± 0.2 points in the adult group (p > 0.05). In stable localized scleroderma, the initial autologous fat transplantation was equally effective for facial contour deformity improvement, with no significant difference in fat graft retention or satisfaction.

Mobile medial pivot (lateral slide)-type total knee arthroplasty exhibited different motion patterns between under anaesthesia and weight-bearing condition.

Total knee arthroplasty (TKA), which has medial pivot and mobile-bearing mechanisms, has been developed and clinically used. However, the in vivo dynamic kinematics of the mobile medial pivot-type TKA (MMPTKA) is unclear. This study analysed the in vivo kinematics of MMPTKA in weight-bearing and nonweight-bearing conditions. The study included 10 knees that underwent primary TKA using MMPTKA. After TKA, lateral view radiographs of the knee in full extension, 90° of flexionand passive full flexion were taken under general anaesthesia in the nonweight-bearing condition. At least 6 months postoperatively, knee motion during squatting from a weight-bearing standing position was observed using a flat-panel detector and analysed using the three-dimensional-to-two-dimensional image registration technique. Under anaesthesia: in passive full flexion, the anteroposterior (AP) locations of the femoral component's medial and lateral distal points were 10.2 and 16.0 mm posterior, and the rotational angles of the femoral component's X-axis (FCX) and insert were 8.1° external rotation and 18.5° internal rotation to full extension, respectively. Squatting: the AP translations of the femoral component's medial and lateral most distal points were 2.2 and 6.4 mm, and the rotational angles of the FCX and insert were 5.7° and 1.6° external rotation, respectively. Significant differences were observed in the AP translation of the femoral component's medial and lateral most distal points and changes in the insert's rotational angle when comparing under anaesthesia and squatting. The kinematics of the insert in MMPTKA was significantly influenced by loading and muscle contraction. The femoral component exhibited substantial external rotation and posterior translation under anaesthesia, which may contribute to achieving an optimal range of motion. The insert remained relatively stable during squatting and minimal rotation was observed, indicating good stability. MMPTKA was expected to demonstrate rational kinematics by incorporating mobile and medial pivot mechanisms. Level IV,prospective biomechanical case series study.

Visual-motor integration in children with unilateral cerebral palsy: application of the computer-aided measure of visual-motor integration

BackgroundChildren with unilateral cerebral palsy (UCP) are encouraged to participate in the regular school curriculum. However, even when using the less-affected hand for handwriting, children with UCP still experience handwriting difficulties. Visual-motor integration (VMI) is a predictor of handwriting quality. Investigating VMI in children with UCP is important but still lacking. Conventional paper-based VMI assessments is subjective and use all-or-nothing scoring procedures, which may compromise the fidelity of VMI assessments. Moreover, identifying important shapes that are predictive of VMI performance might benefit clinical decision-making because different geometric shapes represent different developmental stepping stones of VMI. Therefore, a new computer-aided measure of VMI (the CAM-VMI) was developed to investigate VMI performance in children with UCP and to identify shapes important for predicting their VMI performance.MethodsTwenty-eight children with UCP and 28 typically-developing (TD) children were recruited. All participants were instructed to complete the CAM-VMI and Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery-VMI). The test items of the CAM-VMI consisted of nine simple geometric shapes related to writing readiness. Two scores of the CAM-VMI, namely, Error and Effort, were obtained by image registration technique. The performances on the Beery-VMI and the CAM-VMI of children with UCP and TD children were compared by independent t-test. A series of stepwise regression analyses were used to identify shapes important for predicting VMI performance in children with UCP.ResultsSignificant group differences were found in both the CAM-VMI and the Beery-VMI results. Furthermore, Error was identified as a significant aspect for predicting VMI performance in children with UCP. Specifically, the square item was the only significant predictor of VMI performance in children with UCP.ConclusionsThis study was a large-scale study that provided direct evidence of impaired VMI in school-aged children with UCP. Even when using the less-affected hand, children with UCP could not copy the geometric shapes as well as TD children did. The copied products of children with UCP demonstrated poor constructional accuracy and inappropriate alignment. Furthermore, the predictive model suggested that the constructional accuracy of a copied square is an important predictor of VMI performance in children with UCP.

Analysis on frost resistance and pore structure of phase change concrete modified by Nano-SiO2 under freeze-thaw cycles

To study the effect of nano-SiO2 on the frost resistance of phase change concrete, and to further explore the deterioration of its internal pore structure under the effect of freeze–thaw cycles, X-ray tomography was applied to the rapid freeze–thaw test for the study of frost resistance, and the pore structure of the modified concrete was reconstructed during the freeze–thaw cycles. By image processing and registration techniques, the change pattern of mesoscopic pore inside the phase change concrete modified by nano-SiO2 and the development and ductility characteristics of the pore structure in the interface transition zone were analyzed. The results showed that the admixture of nano-SiO2 and phase change materials could effectively improve the frost resistance of concrete. Frost resistance life was improved by at least 43 % and up to 71 %. Additionally, nanomaterials could constrain the change of pore structure inside the composite modified concrete and densify the interface transition zone. The ITZ porosity of ordinary concrete increased by 10.58 times after 150 freeze–thaw cycles, while the secondary modified phase change concrete with 10mPCMs/1.5NS only increased by 1.2 times.

Comprehensive Evaluation of Multispectral Image Registration Strategies in Heterogenous Agriculture Environment.

This article is focused on the comprehensive evaluation of alleyways to scale-invariant feature transform (SIFT) and random sample consensus (RANSAC) based multispectral (MS) image registration. In this paper, the idea is to extensively evaluate three such SIFT- and RANSAC-based registration approaches over a heterogenous mix containing Triticum aestivum crop and Raphanus raphanistrum weed. The first method is based on the application of a homography matrix, derived during the registration of MS images on spatial coordinates of individual annotations to achieve spatial realignment. The second method is based on the registration of binary masks derived from the ground truth of individual spectral channels. The third method is based on the registration of only the masked pixels of interest across the respective spectral channels. It was found that the MS image registration technique based on the registration of binary masks derived from the manually segmented images exhibited the highest accuracy, followed by the technique involving registration of masked pixels, and lastly, registration based on the spatial realignment of annotations. Among automatically segmented images, the technique based on the registration of automatically predicted mask instances exhibited higher accuracy than the technique based on the registration of masked pixels. In the ground truth images, the annotations performed through the near-infrared channel were found to have a higher accuracy, followed by green, blue, and red spectral channels. Among the automatically segmented images, the accuracy of the blue channel was observed to exhibit a higher accuracy, followed by the green, near-infrared, and red channels. At the individual instance level, the registration based on binary masks depicted the highest accuracy in the green channel, followed by the method based on the registration of masked pixels in the red channel, and lastly, the method based on the spatial realignment of annotations in the green channel. The instance detection of wild radish with YOLOv8l-seg was observed at a mAP@0.5 of 92.11% and a segmentation accuracy of 98% towards segmenting its binary mask instances.

Deep Learning-Based Medical Image Registration Algorithm: Enhancing Accuracy with Dense Connections and Channel Attention Mechanisms

In critical clinical medical image analysis applications, such as surgical navigation and tumor monitoring, image registration is crucial. Recognizing the potential for enhanced accuracy in existing unsupervised image registration techniques for single-modal imagery, this research introduces an innovative deep learning-based image registration algorithm. Its novelty resides in integrating short and long connections to create a densely connected structure, markedly refining the feature map interconnectivity within the U-Net architecture. This advancement addresses the significant semantic gap issues arising from disparities in feature map sampling depths. Moreover, the algorithm incorporates a channel attention mechanism within the U-shaped network's decoder, significantly mitigating image noise and facilitating the generation of smoother deformation fields. This enhancement not only boosts the model's detail sensitivity but also markedly increases image registration precision, particularly evident when processing single-modal brain MRI datasets, thereby proving the algorithm's efficacy and utility. Extensive clinical application-based training and testing have underscored this algorithm's substantial contributions to medical image registration accuracy enhancement. Overall, by leveraging deep learning technologies and innovative algorithmic structures, this study addresses pivotal challenges in medical image registration, offering more precise and dependable support for clinical applications like surgical navigation and tumor surveillance.

Imaging radar and LiDAR image translation for 3-DOF extrinsic calibration

The integration of sensor data is crucial in the field of robotics to take full advantage of the various sensors employed. One critical aspect of this integration is determining the extrinsic calibration parameters, such as the relative transformation, between each sensor. The use of data fusion between complementary sensors, such as radar and LiDAR, can provide significant benefits, particularly in harsh environments where accurate depth data is required. However, noise included in radar sensor data can make the estimation of extrinsic calibration challenging. To address this issue, we present a novel framework for the extrinsic calibration of radar and LiDAR sensors, utilizing CycleGAN as a method of image-to-image translation. Our proposed method employs translating radar bird-eye-view images into LiDAR-style images to estimate the 3-DOF extrinsic parameters. The use of image registration techniques, as well as deskewing based on sensor odometry and B-spline interpolation, is employed to address the rolling shutter effect commonly present in spinning sensors. Our method demonstrates a notable improvement in extrinsic calibration compared to filter-based methods using the MulRan dataset.

Image registration for zooming: A statistically consistent local feature mapping approach

AbstractImage registration is a widely used tool for matching two images of the same scene with one another. In the literature, several image registration techniques are available to register rigid‐body and non‐rigid‐body transformations. One such important transformation is zooming. There are very few feature‐based methods that address this particular problem. These methods fail miserably when there are only a limited number of point features available in the image. This paper proposes a feature‐based approach that works with a feature that is readily available in almost all images, for registering two images of the same image object where one is a zoomed‐in version of the other. In the proposed method, we first detect the possible edge points which we consider as features in both the reference and the zoomed image. Then, we map these features of the reference and the zoomed image with one another and find the relationship between them using a mathematical model. Finally, we use the relationship to register the zoomed‐in image. This method outperforms some of the state‐of‐the‐art methods in many occasions. Several numerical examples and some statistical properties justify that this method works well in many applications.

GeoAI Dataset for Training Deep Learning-Based Optical Satellite Image Matching Model

Satellite imagery is being used to monitor the Earth, as it allows for the continuous provision of multi-temporal observations with consistent quality. To analyze time series remote sensing data with high accuracy, the process of image registration must be conducted beforehand. Image registration techniques are mainly divided into region-based registration and feature-based registration, and both techniques extract the same points based on the similarity of spectral characteristics and object shapes between master and slave images. In addition, recently, deep learning-based siamese neural network and convolutional neural network models have been utilized to match images. This has high performance compared to previous non-deep learning algorithms, but a very large amount of data is required to train a deep learning-based image registration model. In this study, we aim to generate a dataset for training a deep learning-based optical image registration model. To build the data, we acquired Satellite Side-Looking (S2Looking) data, an open dataset, and performed preprocessing and data augmentation on the data to create input data. After that, we added offsets to the X and Y directions between the master and slave images to create label data. The preprocessed input data and labeled data were used to build a dataset suitable for image registration. The data is expected to be useful for training deep learning-based satellite image registration models.

LodgeNet: an automated framework for precise detection and classification of wheat lodging severity levels in precision farming

Wheat lodging is a serious problem affecting grain yield, plant health, and grain quality. Addressing the lodging issue in wheat is a desirable task in breeding programs. Precise detection of lodging levels during wheat screening can aid in selecting lines with resistance to lodging. Traditional approaches to phenotype lodging rely on manual data collection from field plots, which are slow and laborious, and can introduce errors and bias. This paper presents a framework called ‘LodgeNet,’ that facilitates wheat lodging detection. Using Unmanned Aerial Vehicles (UAVs) and Deep Learning (DL), LodgeNet improves traditional methods of detecting lodging with more precision and efficiency. Using a dataset of 2000 multi-spectral images of wheat plots, we have developed a novel image registration technique that aligns the different bands of multi-spectral images. This approach allows the creation of comprehensive RGB images, enhancing the detection and classification of wheat lodging. We have employed advanced image enhancement techniques to improve image quality, highlighting the important features of wheat lodging detection. We combined three color enhancement transformations into two presets for image refinement. The first preset, ‘Haze &amp;amp; Gamma Adjustment,’ minimize atmospheric haze and adjusts the gamma, while the second, ‘Stretching Contrast Limits,’ extends the contrast of the RGB image by calculating and applying the upper and lower limits of each band. LodgeNet, which relies on the state-of-the-art YOLOv8 deep learning algorithm, could detect and classify wheat lodging severity levels ranging from no lodging (Class 1) to severe lodging (Class 9). The results show the mean Average Precision (mAP) of 0.952% @0.5 and 0.641% @0.50-0.95 in classifying wheat lodging severity levels. LodgeNet promises an efficient and automated high-throughput solution for real-time crop monitoring of wheat lodging severity levels in the field.

Deep learning-based 3D brain multimodal medical image registration.

Medical image registration is a critical preprocessing step in medical image analysis. While traditional medical image registration techniques have matured, their registration speed and accuracy still fall short of clinical requirements. In this paper, we propose an improved VoxelMorph network incorporating ResNet modules and CBAM (RCV-Net), for 3D multimodal unsupervised registration. Unlike popular convolution-based U-shaped registration networks like VoxelMorph, RCV-Net incorporates the convolutional block attention module (CBAM) during the convolution process. This inclusion enhances the feature map information extraction capabilities during training and effectively prevents information loss. Additionally, we introduce a lightweight and residual network module at the network's base, which enhances learning ability without significantly increasing training parameters. To evaluate the superiority of our registration model, we utilize evaluation metrics such as structural similarity (SSIM), peak signal-to-noise ratio (PSNR), and mean square error (MSE). Experimental results demonstrate that our proposed network structure outperforms current state-of-the-art methods, yielding better performance in multimodal registration tasks. Furthermore, generalization testing on databases outside of the training set has confirmed the optimal registration effectiveness of our model.

Effect of TMJ disc position on condylar bone remodeling after arthroscopic disc repositioning surgery

PurposeThe objective of this study was to analyze the effect of TMJ disc position on condylar bone remodeling after arthroscopic disc repositioning surgery. MethodsNine patients with anterior disc displacement without reduction (ADDWoR, 15 sides) who underwent arthroscopic disc repositioning surgery were included. Three-dimensional (3D) reconstruction of the articular disc and the condyle in the closed-mouth position was performed using cone-beam computed tomography (CBCT) and magnetic resonance imaging (MRI) data. Then, the CBCT and MRI images were fused and displayed together by multimodal image registration techniques. Morphological changes in the articular disc and condyle, as well as changes in their spatial relationship, were studied by comparing preoperative and 3-month postoperative CBCT-MRI fused images. ResultsThe volume and superficial area of the articular disc, as well as the area of the articular disc surface in the subarticular cavity, were significantly increased compared to that before the surgical treatment（P < 0.01）. There was also a significant increase in the volume of the condyle (P < 0.001). All condyles showed bone remodeling after surgery that could be categorized as one of two types depending on the position of the articular disc, suggesting that the location of the articular disc was related to the new bone formation. ConclusionsThe morphology of the articular disc and condyle were significantly changed after arthroscopic disc repositioning surgery. The 3D changes in the position of the articular disc after surgery tended to have an effect on condylar bone remodeling and the location of new bone formation.

Visualisation of drug distribution in skin using correlative optical spectroscopy and mass spectrometry imaging

A correlative methodology for label-free chemical imaging of soft tissue has been developed, combining non-linear optical spectroscopies and mass spectrometry to achieve sub-micron spatial resolution and critically improved drug detection sensitivity. The approach was applied to visualise the kinetics of drug reservoir formation within human skin following in vitro topical treatment with a commercial diclofenac gel. Non-destructive optical spectroscopic techniques, namely stimulated Raman scattering, second harmonic generation and two photon fluorescence microscopies, were used to provide chemical and structural contrast. The same tissue sections were subsequently analysed by secondary ion mass spectrometry, which offered higher sensitivity for diclofenac detection throughout the epidermis and dermis. A method was developed to combine the optical and mass spectrometric datasets using image registration techniques. The label-free, high-resolution visualisation of tissue structure coupled with sensitive chemical detection offers a powerful method for drug biodistribution studies in the skin that impact directly on topical pharmaceutical product development.

Evaluating the Necessity of Adaptive RT and the Role of Deformable Image Registration in Lung Cancer with Different Pathologic Classifications.

This study aimed to analyze differential radiotherapy (RT) responses according to the pathological type of lung cancer to see the possibility of applying adaptive radiotherapy (ART). ART planning with resampled-computed tomography was conducted for a total of 30 patients (20 non-small-cell lung cancer patients and 10 small-cell lung cancer patients) using a deformable image registration technique to reveal gross tumor volume (GTV) changes according to the duration of RT. The small-cell lung cancer group demonstrated an average GTV reduction of 20.95% after the first week of initial treatment (p = 0.001), whereas the adenocarcinoma and squamous cell carcinoma groups showed an average volume reduction of 20.47% (p = 0.015) and 12.68% in the second week. The application of ART according to the timing of GTV reduction has been shown to affect changes in radiation dose irradiated to normal tissues. This suggests that ART applications may have to be different depending on pathological differences in lung cancer. Through these results, the present study proposes the possibility of personalized treatment options for individual patients by individualizing ART based on specific radiation responses by pathologic types of lung cancer.

MICDIR: Multi-scale inverse-consistent deformable image registration using UNetMSS with self-constructing graph latent.

Image registration is the process of bringing different images into a common coordinate system - a technique widely used in various applications of computer vision, such as remote sensing, image retrieval, and, most commonly, medical imaging. Deep learning based techniques have been applied successfully to tackle various complex medical image processing problems, including medical image registration. Over the years, several image registration techniques have been proposed using deep learning. Deformable image registration techniques such as Voxelmorph have been successful in capturing finer changes and providing smoother deformations. However, Voxelmorph, as well as ICNet and FIRE, do not explicitly encode global dependencies (i.e. the overall anatomical view of the supplied image) and, therefore, cannot track large deformations. In order to tackle the aforementioned problems, this paper extends the Voxelmorph approach in three different ways. To improve the performance in case of small as well as large deformations, supervision of the model at different resolutions has been integrated using a multi-scale UNet. To support the network to learn and encode the minute structural co-relations of the given image-pairs, a self-constructing graph network (SCGNet) has been used as the latent of the multi-scale UNet - which can improve the learning process of the model and help the model to generalise better. And finally, to make the deformations inverse-consistent, cycle consistency loss has been employed. On the task of registration of brain MRIs, the proposed method achieved significant improvements over ANTs and VoxelMorph, obtaining a Dice score of 0.8013 ± 0.0243 for intramodal and 0.6211 ± 0.0309 for intermodal, while VoxelMorph achieved 0.7747 ± 0.0260 and 0.6071 ± 0.0510, respectively.

A workflow to visualize vertebrate eyes in 3D.

To establish a workflow to visualize the surgical anatomy in 3D based on histological data of eyes of experimental animals for improving the planning of complex surgical procedures. Four C57BL/6J wild-type(wt) mouse eyes, three Brown Norway rat eyes and four Chinchilla Bastard rabbit eyes were enucleated and processed for standard histology with serial sections and hematoxylin and eosin staining. Image stacks were processed to obtain a representation of the eye anatomy in 3D. In addition, virtual image stacks and 3D point clouds were generated by processing sagittal sections of eyes with stepwise 180° rotation and projection around the eye axis to construct a rotationally symmetric 3D model from one single sagittal section. Serial sections of whole eyes of mice, rats and rabbits showed significant artifacts interfering with a practical image stack generation and straightforward 3D reconstruction despite the application of image registration techniques. A workflow was established to obtain a 3D image of the eye based on virtual image stacks and point cloud generation by rotation of a single sagittal section of the eye around the symmetry axis. By analyzing the tissue shrinkage during histological processing true biometric reconstructions of the eyes were feasible making the resulting model usable for 3D modeling and simulation, e.g. for planning of complex surgical procedures in different species. Because serial sections of the eye with standard histological protocols yielded too many artifacts for a straightforward 3D visualization we reconstructed a pseudorealistic 3D model based on virtual image stacks and point cloud generation calculated from a single sagittal section of the eye. Such a model detailing microscopic structures of the whole eye will allow for a specific planning of surgical procedures in small animal eyes in order to prevent surgical complications in a very early stage of an experiment and it will support the design and development of complex intraocular implants. It will therefore be helpful in surgical teaching and improve laboratory animal welfare by an expected reduction of experimental animal numbers. Further processing including integration of mechanical tissue properties is needed to convert these 3D models into a practical virtual reality teaching and simulation platform for eyes of several species.

A deep learning approach for anomaly identification in PZT sensors using point contact method

The implementation of piezoelectric sensors is degraded due to surface defects, delamination, and extreme weathering conditions, to mention a few. Hence, the sensor needs to be diagnosed before the efficacious implementation in the structural health monitoring (SHM) framework. To rescue the problem, a novel experimental method based on Coulomb coupling is utilised to visualise the evolution of elastic waves and interaction with the surface anomaly in the lead zirconate titanate (PZT) substrate. Recently, machine learning (ML) has been expeditiously becoming an essential technology for scientific computing, with several possibilities to advance the field of SHM. This study employs a deep learning-based autoencoder neural network in conjunction with image registration and peak signal-to-noise ratio (PSNR) to diagnose the surface anomaly in the PZT substrate. The autoencoder extracts the significant damage-sensitive features from the complex waveform big data. Further, it provides a nonlinear input–output model that is well suited for the non-linear interaction of the wave with the surface anomaly and boundary of the substrate. The measured time-series waveform data is provided as input into the autoencoder network. The mean absolute error (MAE) between the input and output of the deep learning model is evaluated to detect the anomaly. The MAEs are sensitive to the anomaly that lies in the PZT substrate. Further, the challenge arising from offset and distortion is addressed with ad hoc image registration technique. Finally, the localisation and quantification of the anomaly are performed by computing PSNR values. This work proposes an advanced, efficient damage detection algorithm in the scenario of big data that is ubiquitous in SHM.