Rigid Transformation Research Articles

Efficient and Robust Point Cloud Registration via Heuristics-Guided Parameter Search.

Estimating the rigid transformation with 6 degrees of freedom based on a putative 3D correspondence set is a crucial procedure in point cloud registration. Existing correspondence identification methods usually lead to large outlier ratios (>95% is common), underscoring the significance of robust registration methods. Many researchers turn to parameter search-based strategies (e.g., Branch-and-Bround) for robust registration. Although related methods show high robustness, their efficiency is limited to the high-dimensional search space. This paper proposes a heuristics-guided parameter search strategy to accelerate the search while maintaining high robustness. We first sample some correspondences (i.e., heuristics) and then just need to sequentially search the feasible regions that make each sample an inlier. Our strategy largely reduces the search space and can guarantee accuracy with only a few inlier samples, therefore enjoying an excellent trade-off between efficiency and robustness. Since directly parameterizing the 6-dimensional nonlinear feasible region for efficient search is intractable, we construct a three-stage decomposition pipeline to reparameterize the feasible region, resulting in three lower-dimensional sub-problems that are easily solvable via our strategy. Besides reducing the searching dimension, our decomposition enables the leverage of 1-dimensional interval stabbing at all three stages for searching acceleration. Moreover, we propose a valid sampling strategy to guarantee our sampling effectiveness, and a compatibility verification setup to further accelerate our search. Extensive experiments on both simulated and real-world datasets demonstrate that our approach exhibits comparable robustness with state-of-the-art methods while achieving a significant efficiency boost.

EGST: Enhanced Geometric Structure Transformer for Point Cloud Registration.

We explore the effect of geometric structure descriptors on extracting reliable correspondences and obtaining accurate registration for point cloud registration. The point cloud registration task involves the estimation of rigid transformation motion in unorganized point cloud, hence it is crucial to capture the contextual features of the geometric structure in point cloud. Recent coordinates-only methods ignore numerous geometric information in the point cloud which weaken ability to express the global context. We propose Enhanced Geometric Structure Transformer to learn enhanced contextual features of the geometric structure in point cloud and model the structure consistency between point clouds for extracting reliable correspondences, which encodes three explicit enhanced geometric structures and provides significant cues for point cloud registration. More importantly, we report empirical results that Enhanced Geometric Structure Transformer can learn meaningful geometric structure features using none of the following: (i) explicit positional embeddings, (ii) additional feature exchange module such as cross-attention, which can simplify network structure compared with plain Transformer. Extensive experiments on the synthetic dataset and real-world datasets illustrate that our method can achieve competitive results.

Deep-UEO: Empowering medical image registration with hybrid strategy based on deep learning and united equilibrium optimizer

Traditional optimization methods encounter challenges in determining the optimal geometric transformation parameters and often lacking to achieve global optimum which leads to extended computational time required for convergence. To address this challenge, a dual-phase strategy is proposed for multimodal biomedical image registration by amalgamating the united equilibrium optimizer (UEO) with Visual Geometry Group (VGG-19). In first phase, the input images are aligned by identifying the optimal values of rigid transformation parameters through UEO, with mutual information (MI) maximization serving as an objective function. In second phase, VGG19 is employed to enhance the reliability of resulting registered image obtained through UEO optimization for extracting both the low- and high-level features of reference and target images. Following this, dynamic inlier selection is employed to refine the matching process, integrating expectation maximization for iterative updating of inlier selection, thus improving registration accuracy. Finally, thin plate spline interpolation is utilized to calculate the transformation matrix, ensuring precise alignment between reference and UEO registered image. Extensive experiments are conducted on multimodal medical images sourced from publicly available repositories and clinical(in-house) datasets to validate the reliability of the proposed scheme. These experiments showcase substantial improvements in RMSE from 22.21094 to 10.983654, SSIM from 0.7190832 to 0.8409321, PSNR from 30.176044 to 34.095471, and CC from 0.9304633 to 0.9809432, outperforming state-of-the-art methods.

Non-coplanar CBCT image reconstruction using a generative adversarial network for non-coplanar radiotherapy.

To develop a non-coplanar cone-beam computed tomography (CBCT) image reconstruction method using projections within a limited angle range for non-coplanar radiotherapy. A generative adversarial network (GAN) was utilized to reconstruct non-coplanar CBCT images. Data from 40 patients with brain tumors and two head phantoms were used in this study. In the training stage, the generator of the GAN used coplanar CBCT and non-coplanar projections as the input, and an encoder with a dual-branch structure was utilized to extract features from the coplanar CBCT and non-coplanar projections separately. Non-coplanar CBCT images were then reconstructed using a decoder by combining the extracted features. To improve the reconstruction accuracy of the image details, the generator was adversarially trained using a patch-based convolutional neural network as the discriminator. A newly designed joint loss was used to improve the global structure consistency rather than the conventional GAN loss. The proposed model was evaluated using data from eight patients and two phantoms at four couch angles (±45°,±90°) that are most commonly used for brain non-coplanar radiotherapy in our department. The reconstructed accuracy was evaluated by calculating the root mean square error (RMSE) and an overall registration error ε, computed by integrating the rigid transformation parameters. In both patient data and phantom data studies, the qualitative and quantitative metrics results indicated that±45° couch angle models performed better than±90° couch angle models and had statistical differences. In the patient data study, the mean RMSE and ε values of couch angle at 45°, -45°, 90°, and -90° were 58.5 HU and 0.42mm, 56.8 HU and 0.41mm, 73.6 HU and 0.48mm, and 65.3 HU and 0.46mm, respectively. In the phantom data study, the mean RMSE and ε values of couch angle at 45°, -45°, 90°, and -90° were 91.2 HU and 0.46mm, 95.0 HU and 0.45mm, 114.6 HU and 0.58mm, and 102.9 HU and 0.52mm, respectively. The results show that the reconstructed non-coplanar CBCT images can potentially enable intra-treatment three-dimensional position verification for non-coplanar radiotherapy.

Camera LiDAR calibration: an automatic and accurate method with novel PLE metrics

Abstract Camera-LiDAR calibration is a crucial aspect of perception systems and sensor fusion in various fields, facilitating the fusion of data from cameras and LiDAR sensors for applications such as autonomous vehicles, robotics, and augmented reality. In this paper, we presented a novel multi-modal fusion method that introduces an efficient camera-LiDAR joint calibration technique using a simple checkerboard. Our method estimates the 3D rigid transformation of the camera system with respect to the LiDAR frame by establishing 2D-to-3D point correspondences for geometric calibration. We improved the idea of the traditional PNP (Perspective-N-Point) algorithm by employing point cloud segmentation and clustering to re-match the checkerboard. When compared with the calibration methods in the Autoware and MATLAB calibration toolbox, the reprojection error was improved by 38.13\% and 58.30\% respectively. The average translation error was improved by 78.26\% and 51.61\% respectively, while the average rotation error is improved by 75\% and 60.78\% respectively, which has a great improvement in all three areas. At the same time, we propose a new evaluation metric for joint calibration results based on reprojected images: PLE (Pixel-Level Evaluation) index to reflect the accuracy of the joint calibration of the camera and LiDAR. An automatic calibration software has been developed for the calibration of the camera's intrinsic parameters as well as for the joint calibration of the camera-LIDAR extrinsic parameters. We have extensively validated our algorithm using the Intel RealSense Depth D435 Camera and LeiShen C16-151B LiDAR.

Fast Robust Point Cloud Registration Based on Compatibility Graph and Accelerated Guided Sampling

Point cloud registration is a crucial technique in photogrammetry, remote sensing, etc. A generalized 3D point cloud registration framework has been developed to estimate the optimal rigid transformation between two point clouds using 3D key point correspondences. However, challenges arise due to the uncertainty in 3D key point detection techniques and the similarity of local surface features. These factors often lead to feature descriptors establishing correspondences containing significant outliers. Current point cloud registration algorithms are typically hindered by these outliers, affecting both their efficiency and accuracy. In this paper, we propose a fast and robust point cloud registration method based on a compatibility graph and accelerated guided sampling. By constructing a compatible graph with correspondences, a minimum subset sampling method combining compatible edge sampling and compatible vertex sampling is proposed to reduce the influence of outliers on the estimation of the registration parameters. Additionally, an accelerated guided sampling strategy based on preference scores is presented, which effectively utilizes model parameters generated during the iterative process to guide the sampling toward inliers, thereby enhancing computational efficiency and the probability of estimating optimal parameters. Experiments are carried out on both synthetic and real-world data. The experimental results demonstrate that our proposed algorithm achieves a significant balance between registration accuracy and efficiency compared to state-of-the-art registration algorithms such as RANSIC and GROR. Even with up to 2000 initial correspondences and an outlier ratio of 99%, our algorithm achieves a minimum rotation error of 0.737° and a minimum translation error of 0.0201 m, completing the registration process within 1 s.

PRF-Net: A Progressive Remote Sensing Image Registration and Fusion Network.

Most of the existing fusion algorithms are not robust to unregistered input images. Even after image registration, nonlinear nonregistration may persist in the local areas of the images, leading to poor quality in the fused image. So, as to tackle these challenges, a progressive remote sensing image registration and fusion network is proposed in this article, and named PRF-Net, which is particularly useful when two images are from different platforms. First, a registration network is designed to register the input image patches, which includes a global spatial transform network (GSTN) and a local spatial warp network (LSWN). The GSTN is primarily used for coarse registration, applying rigid transformation to globally align the input images. After coarse registration, the preliminarily registered moving image is input into the LSWN for local fine-tuning to maximize correlation between the input image patches. Subsequently, the fine registered images are degraded and input into the fusion network to generate the fused image. To maintain sufficient spectral and spatial information of the fused image, a multiscale feature extraction (MSFE) block with a highly interpretable spatial details attention (SDA) block is designed, which can enhance the ability of fusion network to extract and preserve spatial details and spectral information. Three groups of experiments conducted on four types of remote sensing images give evidence of that the proposed PRF-Net exhibits excellent performance in both reduced and full resolutions, showcasing its outstanding registration and fusion quality.

Measurement Approach for the Pose of Flanges in Cabin Assemblies through Distributed Vision.

The relative rotation angle between two cabins should be automatically and precisely obtained during automated assembly processes for spacecraft and aircraft. This paper introduces a method to solve this problem based on distributed vision, where two groups of cameras are employed to take images of mating features, such as dowel pins and holes, in oblique directions. Then, the relative rotation between the mating flanges of two cabins is calculated. The key point is the registration of the distributed cameras; thus, a simple and practical registration process is designed. It is assumed that there are rigid and scaling transformations among the world coordinate systems (WCS) of each camera. Therefore, the rigid-correct and scaling-correct matrices are adopted to register the cameras. An auxiliary registration device with known features is designed and moved in the cameras' field of view (FOV) to obtain the matrix parameters so that each camera acquires traces of every feature. The parameters can be solved using a genetic algorithm based on the known geometric relationships between the trajectories on the registration devices. This paper designs a prototype to verify the method. The precision reaches 0.02° in the measuring space of 340 mm.

Sparse-to-Dense Point Cloud Registration Based on Rotation-Invariant Features

Point cloud registration is a critical problem because it is the basis of many 3D vision tasks. With the popularity of deep learning, many scholars have focused on leveraging deep neural networks to address the point cloud registration problem. However, many of these methods are still sensitive to partial overlap and differences in density distribution. For this reason, herein, we propose a method based on rotation-invariant features and using a sparse-to-dense matching strategy for robust point cloud registration. Firstly, we encode raw points as superpoints with a network combining KPConv and FPN, and their associated features are extracted. Then point pair features of these superpoints are computed and embedded into the transformer to learn the hybrid features, which makes the approach invariant to rigid transformation. Subsequently, a sparse-to-dense matching strategy is designed to address the registration problem. The correspondences of superpoints are obtained via sparse matching and then propagated to local dense points and, further, to global dense points, the byproduct of which is a series of transformation parameters. Finally, the enhanced features based on spatial consistency are repeatedly fed into the sparse-to-dense matching module to rebuild reliable correspondence, and the optimal transformation parameter is re-estimated for final alignment. Our experiments show that, with the proposed method, the inlier ratio and registration recall are effectively improved, and the performance is better than that of other point cloud registration methods on 3DMatch and ModelNet40.

Cultural relics fragment assembly based on fracture surface contour features

Addressing the issue of registration deviations caused by local fragment loss and shape diversity during the process of cultural relics fragment assembly, this paper proposes a method for assembling cultural relics fragments based on the contour features of fracture surfaces. Initially, the curvature of discrete point clouds is calculated using the quartic surface method and the Weingarten matrix, and curvature discontinuity points are identified using a proportional method as a candidate feature point set. Subsequently, the angle difference between a point and its neighboring points’ directed segments is defined as a neighborhood feature parameter, which further optimizes the feature point set and selects the optimal matching set. Then the 4-points congruent sets (4PCS) algorithm is utilized for the initial registration of the fragments, and the unit quaternion method is employed to calculate the rigid body transformation matrix during the iteration process. Finally, corresponding point pairs are determined by the intersection of a point’s normal with a set of points, improving the iterative closest point (ICP) algorithm to achieve precise assembly of the fragments. The experimental results show that, compared to other assembly algorithms, the proposed method is more accurate in feature point extraction, robust, and effectively enhances the accuracy and efficiency of fragment assembly.

Rigid point cloud registration based on correspondence cloud for image-to-patient registration in image-guided surgery.

Image-to-patient registration aligns preoperative images to intra-operative anatomical structures and it is a critical step in image-guided surgery (IGS). The accuracy and speed of this step significantly influence the performance of IGS systems. Rigid registration based on paired points has been widely used in IGS, but studies have shown its limitations in terms of cost, accuracy, and registration time. Therefore, rigid registration of point clouds representing the human anatomical surfaces has become an alternative way for image-to-patient registration in the IGSsystems. We propose a novel correspondence-based rigid point cloud registration method that can achieve global registration without the need for pose initialization. The proposed method is less sensitive to outliers compared to the widely used RANSAC-based registration methods and it achieves high accuracy at a high speed, which is particularly suitable for the image-to-patient registration inIGS. We use the rotation axis and angle to represent the rigid spatial transformation between two coordinate systems. Given a set of correspondences between two point clouds in two coordinate systems, we first construct a 3D correspondence cloud (CC) from the inlier correspondences and prove that the CC distributes on a plane, whose normal is the rotation axis between the two point clouds. Thus, the rotation axis can be estimated by fitting the CP. Then, we further show that when projecting the normals of a pair of corresponding points onto the CP, the angle between the projected normal pairs is equal to the rotation angle. Therefore, the rotation angle can be estimated from the angle histogram. Besides, this two-stage estimation also produces a high-quality correspondence subset with high inlier rate. With the estimated rotation axis, rotation angle, and the correspondence subset, the spatial transformation can be computed directly, or be estimated using RANSAC in a fast and robust way within only 100iterations. To validate the performance of the proposed registration method, we conducted experiments on the CT-Skull dataset. We first conducted a simulation experiment by controlling the initial inlier rate of the correspondence set, and the results showed that the proposed method can effectively obtain a correspondence subset with much higher inlier rate. We then compared our method with traditional approaches such as ICP, Go-ICP, and RANSAC, as well as recently proposed methods like TEASER, SC2-PCR, and MAC. Our method outperformed all traditional methods in terms of registration accuracy and speed. While achieving a registration accuracy comparable to the recently proposed methods, our method demonstrated superior speed, being almost three times faster thanTEASER. Experiments on the CT-Skull dataset demonstrate that the proposed method can effectively obtain a high-quality correspondence subset with high inlier rate, and a tiny RANSAC with 100 iterations is sufficient to estimate the optimal transformation for point cloud registration. Our method achieves higher registration accuracy and faster speed than existing widely used methods, demonstrating great potential for the image-to-patient registration, where a rigid spatial transformation is needed to align preoperative images to intra-operative patientanatomy.

Development and Evaluation of a Two-Staged 3D Keypoint Based Workflow for the Co-Registration of Unstructured Multi-Temporal and Multi-Modal 3D Point Clouds

Abstract. Robust and automated point cloud registration methods are required in many geoscience applications using multi-temporal and multi-modal 3D point clouds. Therefore, a 3D keypoint-based coarse registration workflow has been implemented, utilizing the ISS keypoint detector and 3DSmoothNet descriptor. This paper contributes to keypoint-based registration research through variations of the standard workflow proposed in the literature, applying a two-staged strategy of global and local keypoint matching as well as prototypical keypoint projection and fine registration based on ICP. Further, by testing the utilized detector and descriptor on unstructured, multi-temporal and multi-source point clouds with variations in point cloud density, generalization ability is tested outside benchmark data. Therefore, data of the Bøverbreen glacier in Jotunheimen, Norway has been acquired in 2022 and 2023, deploying UAV-based image matching and terrestrial laser scanning. The results show good performance of the implemented robust matching algorithm PROSAC, requiring fewer iterations than the well-known RANSAC approach, but solving the rigid body transformation with TEASER++ is faster and more robust to outliers without demanding pre-knowledge of the data. Further, the results identify the keypoint detection as most limiting factor in speed and accuracy. Summarizing, keypoint-based coarse registration on low density point clouds, applying a global and local matching strategy and transformation estimation using TEASER++ is recommended. Keypoint projection shows potential, increasing number and precision in low density clouds, but has to be more robust. Further research needs to be carried out, focusing on identifying a fast and robust keypoint detector.

Variational model-based reconstruction techniques for multi-patch data in Magnetic Particle Imaging

Magnetic Particle Imaging is an emerging imaging modality through which it is possible to detect tracers containing superparamagnetic nanoparticles. The exposure of the particles to dynamic magnetic fields generates a non-linear response that is used to locate the particles and produce an image of their distribution. The bounding box that can be covered by a single scan curve depends on the strength of the gradients of the magnetic fields applied, which is limited due to the risk of causing peripheral nerve stimulation (PNS) in the patients. To address this issue, multiple scans are performed. The scan data must be merged together to produce reconstructions of larger regions of interest. In this paper we propose a mathematical framework which can deal with rather general multi-patching scenarios including rigid transformations of the field of view (FoV), the specimen and of the scanner. We show the flexibility of this framework in a variety of different scanning scenarios. Moreover, we describe an iterative reconstruction algorithm that yields a reconstruction of the target distribution by minimizing a convex functional which includes positivity constraints and sparsity enforcing priors. We show its convergence to a minimizer and perform numerical experiments on simulated data.

Automatic image alignment and stitching for ultrasound-based robotic inspection of complex geometry components

Robotic inspection of complex geometry components using immersion ultrasonic techniques is relevant for many ultrasonic Non-Destructive Testing (NDT) applications in different industries. To image large component, ultrasonic probe is manipulated around the component immersed in water using a robotic guided system to create a tiled image of the entire component. Due to the mechanical vibrations of the high-speed robotic arm, the probe position coordinates provided by the robotic system are not precise enough to ensure an accurate reconstruction (stitching) of a composite image from all individual images. In this work, we propose a stitching method specifically designed to create a single 2D image of the surface of an inspected component from a stack of Total Focusing Method (TFM) images. This stitching method uses Iterative Closest Point (ICP) registration to estimate a 2D rigid transformation between two consecutive 2D images, by maximizing the overlap between the surface geometries extracted from each image. The capabilities of this imaging technique are illustrated by various simulated and experimental results carried out in a water tank. Significant improvements in surface image quality, leading to accurate surface reconstruction, are shown for vertical vibrations with displacements of more than two operating wavelengths and for rotational vibrations with deviation angles of less than 1°. The results also show that the resolving power of the ICP algorithm decreases for strong rotational vibrations.

Extrinsic calibration method for integrating infrared thermal imaging camera and 3D LiDAR

PurposeIn the context of fire incidents within buildings, efficient scene perception by firefighting robots is particularly crucial. Although individual sensors can provide specific types of data, achieving deep data correlation among multiple sensors poses challenges. To address this issue, this study aims to explore a fusion approach integrating thermal imaging cameras and LiDAR sensors to enhance the perception capabilities of firefighting robots in fire environments.Design/methodology/approachPrior to sensor fusion, accurate calibration of the sensors is essential. This paper proposes an extrinsic calibration method based on rigid body transformation. The collected data is optimized using the Ceres optimization algorithm to obtain precise calibration parameters. Building upon this calibration, a sensor fusion method based on coordinate projection transformation is proposed, enabling real-time mapping between images and point clouds. In addition, the effectiveness of the proposed fusion device data collection is validated in experimental smoke-filled fire environments.FindingsThe average reprojection error obtained by the extrinsic calibration method based on rigid body transformation is 1.02 pixels, indicating good accuracy. The fused data combines the advantages of thermal imaging cameras and LiDAR, overcoming the limitations of individual sensors.Originality/valueThis paper introduces an extrinsic calibration method based on rigid body transformation, along with a sensor fusion approach based on coordinate projection transformation. The effectiveness of this fusion strategy is validated in simulated fire environments.

The SVD-enhanced bees algorithm, a novel procedure for point cloud registration

Point cloud registration entails the estimation of the rigid transformation which best aligns two point sets. At present, Iterative Closest Point is arguably the best-known and one of the most effective algorithms for point cloud registration. Iterative Closest Point uses singular value decomposition to optimise the alignment of two point sets via least squares fit. However, the algorithm is liable to converge to sub-optimal solutions due to its greedy search strategy. In this study, the problem of point cloud registration is addressed using the popular Bees Algorithm metaheuristic. Thanks to its global search strategy, the Bees Algorithm is known to be impervious to sub-optimal convergence. Singular value decomposition is used as a problem-specific operator within the Bees Algorithm to improve the efficiency of the search. Experimental tests showed that the standard Bees Algorithm outperformed Iterative Closest Point, an Evolutionary Algorithm, and Particle Swarm Optimisation in terms of consistency and precision of the point cloud registration results. The Bees Algorithm also excelled for its robustness to noisy data. When enhanced by singular value decomposition, the Bees Algorithm outperformed the standard Bees Algorithm, and similarly enhanced evolutionary and particle swarm optimisers. The running times of the proposed method were compatible with online implementation. It is concluded that the proposed combination of bees-inspired optimisation and least squares fitting allows accurate, efficient, and consistent registration of point cloud models.

Hand-eye calibration of line structured-light sensor based on double-sphere constraints

Aiming at the complex problem of solving the positional relationship between the line laser sensor at the end of the robot wrist and the coordinate system of the end flange, a line laser hand-eye calibration method based on a double ball fixed target is proposed. Firstly, a double standard ball with known radius and center distance is used as a reference tool, and the point cloud data of a single contour line is collected several times, and the coordinates of the two ball centers under the camera coordinate system can be obtained according to the center coordinates of the circle obtained from the fitted circle combined with the Pythagoras theorem. Then, according to the principle that the two spherical centers are fixed under the robot base coordinate system, the rigid transformation matrix between the line laser sensor and the robot is successively solved. When solving the rotation matrix, the constraint of the distance between the spherical centers of the two balls is introduced to obtain a solution with better accuracy. Experimental results show that the method has better overall performance compared with the traditional single-ball hand-eye calibration method.

HALNet: Partial Point Cloud Registration Based on Hybrid Attention and Deep Local Features.

Point cloud registration is an important task in computer vision and robotics which is widely used in 3D reconstruction, target recognition, and other fields. At present, many registration methods based on deep learning have better registration accuracy in complete point cloud registration, but partial registration accuracy is poor. Therefore, a partial point cloud registration network, HALNet, is proposed. Firstly, a feature extraction network consisting mainly of adaptive graph convolution (AGConv), two-dimensional convolution, and convolution block attention (CBAM) is used to learn the features of the initial point cloud. Then the overlapping estimation is used to remove the non-overlapping points of the two point clouds, and the hybrid attention mechanism composed of self-attention and cross-attention is used to fuse the geometric information of the two point clouds. Finally, the rigid transformation is obtained by using the fully connected layer. Five methods with excellent registration performance were selected for comparison. Compared with SCANet, which has the best registration performance among the five methods, the RMSE(R) and MAE(R) of HALNet are reduced by 10.67% and 12.05%. In addition, the results of the ablation experiment verify that the hybrid attention mechanism and fully connected layer are conducive to improving registration performance.

Non-symmetric over-time pooling using pseudo-grouping functions for convolutional neural networks

Convolutional Neural Networks (CNNs) are a family of networks that have become state-of-the-art in several fields of artificial intelligence due to their ability to extract spatial features. In the context of natural language processing, they can be used to build text classification models based on textual features between words. These networks fuse local features to generate global features in their over-time pooling layers. These layers have been traditionally built using the maximum function or other symmetric functions such as the arithmetic mean. It is important to note that the order of input local features is significant (i.e. the symmetry is not an inherent characteristic of the model). While this characteristic is appropriate for image-oriented CNNs, where symmetry might make the network robust to image rigid transformations, it seems counter-productive for text processing, where the order of the words is certainly important. Our proposal is, hence, to use non-symmetric pooling operators to replace the maximum or average functions. Specifically, we propose to perform over-time pooling using pseudo-grouping functions, a family of non-symmetric aggregation operators that generalize the maximum function. We present a construction method for pseudo-grouping functions and apply different examples of this family to over-time pooling layers in text-oriented CNNs. Our proposal is tested on seven different models and six different datasets in the context of engineering applications, e.g. text classification. The results show an overall improvement of the models when using non-symmetric pseudo-grouping functions over the traditional pooling function.

Detection of maize stem diameter by using RGB-D cameras' depth information under selected field condition.

Stem diameter is a critical phenotypic parameter for maize, integral to yield prediction and lodging resistance assessment. Traditionally, the quantification of this parameter through manual measurement has been the norm, notwithstanding its tedious and laborious nature. To address these challenges, this study introduces a non-invasive field-based system utilizing depth information from RGB-D cameras to measure maize stem diameter. This technology offers a practical solution for conducting rapid and non-destructive phenotyping. Firstly, RGB images, depth images, and 3D point clouds of maize stems were captured using an RGB-D camera, and precise alignment between the RGB and depth images was achieved. Subsequently, the contours of maize stems were delineated using 2D image processing techniques, followed by the extraction of the stem's skeletal structure employing a thinning-based skeletonization algorithm. Furthermore, within the areas of interest on the maize stems, horizontal lines were constructed using points on the skeletal structure, resulting in 2D pixel coordinates at the intersections of these horizontal lines with the maize stem contours. Subsequently, a back-projection transformation from 2D pixel coordinates to 3D world coordinates was achieved by combining the depth data with the camera's intrinsic parameters. The 3D world coordinates were then precisely mapped onto the 3D point cloud using rigid transformation techniques. Finally, the maize stem diameter was sensed and determined by calculating the Euclidean distance between pairs of 3D world coordinate points. The method demonstrated a Mean Absolute Percentage Error (MAPE) of 3.01%, a Mean Absolute Error (MAE) of 0.75mm, a Root Mean Square Error (RMSE) of 1.07mm, and a coefficient of determination (R²) of 0.96, ensuring accurate measurement of maize stem diameter. This research not only provides a new method of precise and efficient crop phenotypic analysis but also offers theoretical knowledge for the advancement of precision agriculture.