Two-dimensional Pixel Research Articles

A full-morphology measurement method and system for tubing internal thread based on rotating-mirrored structured light vision

The major methods of tubing internal thread (TIT) measurement are low efficiency, poor accuracy and incomplete measurement. Therefore, a new full-morphology measurement method and system based on rotating-mirrored structured light vision (RMSLV) is proposed in this paper. Firstly, a measurement model of RMSLV is proposed, which establishes the mathematical relationship between two-dimensional (2D) pixel and three-dimensional (3D) topography data. Subsequently, a step-by-step calibration method for this model is put forward to accurately solve model parameters, so that the high-precision restoration from 2D images to intricate 3D full-morphology of TIT is realized. Finally, the measurement system is built, and the verification experiments are conducted. The results indicate that the system’s measurement accuracy of taper, pitch and tooth height is 6.61 times, 5.60 times and 4.48 times higher than traditional equipment, and the efficiency is exponentially improved. This study provides an effective means for highly precise and fast measurement of the TIT.

Rotating Machinery Fault Diagnosis with Limited Multisensor Fusion Samples by Fused Attention-Guided Wasserstein GAN

As vital equipment in modern industry, the health state of rotating machinery influences the production process and equipment safety. However, rotating machinery generally operates in a normal state most of the time, which results in limited fault data, thus greatly constraining the performance of intelligent fault diagnosis methods. To solve this problem, this paper proposes a novel fault diagnosis method for rotating machinery with limited multisensor fusion samples based on the fused attention-guided Wasserstein generative adversarial network (WGAN). Firstly, the dimensionality of collected multisensor data is reduced to three channels by principal component analysis, and then the one-dimensional data of each channel are converted into a two-dimensional pixel matrix, of which the RGB images are obtained by fusing the three-channel two-dimensional images. Subsequently, the limited RGB samples are augmented to obtain sufficient samples utilizing the fused attention-guided WGAN combined with the gradient penalty (FAWGAN-GP) method. Lastly, the augmented samples are applied to train a residual convolutional neural network for fault diagnosis. The effectiveness of the proposed method is demonstrated by two case studies. When training samples per class are 50, 35, 25, and 15 on the KAT-bearing dataset, the average classification accuracy is 99.9%, 99.65%, 99.6%, and 98.7%, respectively. Meanwhile, the methods of multisensor fusion and the fused attention mechanism have an average improvement of 1.51% and 1.09%, respectively, by ablation experiments on the WT gearbox dataset.

Real-Time Recognition and Localization of Apples for Robotic Picking Based on Structural Light and Deep Learning

The apple is a delicious fruit with high nutritional value that is widely grown around the world. Apples are traditionally picked by hand, which is very inefficient. The development of advanced fruit-picking robots has great potential to replace manual labor. A major prerequisite for a robot to successfully pick fruits the accurate identification and positioning of the target fruit. The active laser vision systems based on structured algorithms can achieve higher recognition rates by quickly capturing the three-dimensional information of objects. This study proposes to combine the laser active vision system with the YOLOv5 neural network model to recognize and locate apples on trees. The method obtained accurate two-dimensional pixel coordinates, which, when combined with the active laser vision system, can be converted into three-dimensional world coordinates for apple recognition and positioning. On this basis, we built a picking robot platform equipped with this visual recognition system, and carried out a robot picking experiment. The experimental findings showcase the efficacy of the neural network recognition algorithm proposed in this study, which achieves a precision rate of 94%, an average precision mAP% of 92.86%, and a spatial localization accuracy of approximately 4 mm for the visual system. The implementation of this control method in simulated harvesting operations shows the promise of more precise and successful fruit positioning. In summary, the integration of the YOLOv5 neural network model with an active laser vision system presents a novel and effective approach for the accurate identification and positioning of apples. The achieved precision and spatial accuracy indicate the potential for enhanced fruit-harvesting operations, marking a significant step towards the automation of fruit-picking processes.

Hydrogel Micropillar Array for Temperature Sensing in Fluid.

Localized temperature sensing and control on a micron-scale have diverse applications in biological systems. We present a micron-sized hydrogel pillar array as potential temperature probes and actuators by exploiting sensitive temperature dependence of their volume change. Soft lithography-based molding processes were presented to fabricate poly N-isopropyl acrylamide (p-NIPAAm)-based hydrogel pillar array on a glass substrate. Au nanorods as light-induced heating elements were embedded within the hydrogel pillars, and near-infrared (NIR) light was used to modulate temperature in a local area. First, static responses of the micro-pillar array were characterized as a function of its temperature. It was shown that the hydrogel had a sensitive volume transition near its low critical solution temperature (LCST). Furthermore, we showed that LCST could be readily adjusted by utilizing copolymerizing with acrylamide (AAM). To demonstrate the feasibility of spatiotemporal temperature mapping and modulation using the presented pillar array, pulsed NIR light was illuminated on a local area of the hydrogel pillar array, and its responses were recorded. Dynamic temperature change in water was mapped based on the abrupt volume change characteristics of the hydrogel pillar, and its potential actuation using NIR light was successfully demonstrated. Considering that the structure can be arrayed in a two-dimensional pixel format with high spatial resolution and high sensitive temperature characteristics, the presented method and the device structure can have diverse applications to change and sense local temperatures in liquid. This is particularly useful in biological systems, where their physiological temperature can be modulated and mapped with high spatial resolution.

On a generalization of Tupper's formula for m colors and n dimensions

Tupper's formula 12<⌊mod(⌊y17⌋2−17⌊x⌋−mod(⌊y⌋,17),2)⌋ has an interesting property: that for any monochrome image that can be represented using a 106×17 array of two-dimensional pixels, there exists a natural number k such that the graph of the inequality over the range 0≤x<106 and k≤y<k+17 is that image. In this paper, we give a generalization for m colors and n dimensions. In this paper, by “natural numbers” we mean non-negative integers. We give m formulae employing n free variables with the property that, for any n-dimensional object of m colors C1,…,Cm that can be represented by hypervoxels (hypervoxels being the multidimensional analogue of pixels) in an n-dimensional array of dimensions A1×⋯×An, there exists a natural number k such that, when the first formula is graphed using color C1, the second formula is graphed using color C2, …, and the mth formula is graphed using color Cm over 0≤x1<A1, 0≤x2<A2,…,0≤xn−1<An−1, and k≤xn<k+An, the union of all colored graphs is that n-dimensional object.

3D pose estimation and localization of construction equipment from single camera images by virtual model integration

Computer vision technologies are receiving much attention for automatic site monitoring. However, previous approaches mainly focused on two-dimensional (2D) pixel information, and there was a limit to providing detailed three-dimensional (3D) pose information. Some studies have built a training database only using virtual models, but there were still limitations in representing the actual equipment. To address these limitations, the authors propose a method for estimating 3D poses of equipment from single camera images by integrating virtual models. The proposed method consists of three main processes: (1) 2D – 3D annotation using a virtual model; (2) preprocessing for pose estimation in two ways (i.e., image matching or key points detection); and (3) 3D pose localization. As a result, the root mean square error for the 3D pose of construction equipment was 1.49 m (using image matching) and 1.56 m (using key points detection). The results imply that the proposed method successfully estimated the 3D pose of the equipment with a single camera. The findings of this study can contribute to detailed productivity or safety analysis based on more diverse 3D information than 2D pixel-based analysis.

Efficient Reversible Data Hiding Using Two-Dimensional Pixel Clustering

Pixel clustering is a technique of content-adaptive data embedding in the area of high-performance reversible data hiding (RDH). Using pixel clustering, the pixels in a cover image can be classified into different groups based on a single factor, which is usually the local complexity. Since finer pixel clustering seems to improve the embedding performance, in this manuscript, we propose using two factors for two-dimensional pixel clustering to develop high-performance RDH. Firstly, in addition to the local complexity, a novel factor was designed as the second factor for pixel clustering. Specifically, the proposed factor was defined using the rotation-invariant code derived from pixel relationships in the four-neighborhood. Then, pixels were allocated to the two-dimensional clusters based on the two clustering factors, and cluster-based pixel prediction was realized. As a result, two-dimensional prediction-error histograms (2D-PEHs) were constructed, and performance optimization was based on the selection of expansion bins from the 2D-PEHs. Next, an algorithm for fast expansion-bin selection was introduced to reduce the time complexity. Lastly, data embedding was realized using the technique of prediction-error expansion according to the optimally selected expansion bins. Extensive experiments show that the embedding performance was significantly enhanced, particularly in terms of improved image quality and reduced time complexity, and embedding capacity also moderately improved.

Nondestructive thermographic detection of internal defects using pixel-pattern based laser excitation and photothermal super resolution reconstruction

In this work, we present a novel approach to photothermal super resolution based thermographic resolution of internal defects using two-dimensional pixel pattern-based active photothermal laser heating in conjunction with subsequent numerical reconstruction to achieve a high-resolution reconstruction of internal defect structures. With the proposed adoption of pixelated patterns generated using laser coupled high-power DLP projector technology the complexity for achieving true two-dimensional super resolution can be dramatically reduced taking a crucial step forward towards widespread practical viability. Furthermore, based on the latest developments in high-power DLP projectors, we present their first application for structured pulsed thermographic inspection of macroscopic metal samples. In addition, a forward solution to the underlying inverse problem is proposed along with an appropriate heuristic to find the regularization parameters necessary for the numerical inversion in a laboratory setting. This allows the generation of synthetic measurement data, opening the door for the application of machine learning based methods for future improvements towards full automation of the method. Finally, the proposed method is experimentally validated and shown to outperform several established conventional thermographic testing techniques while conservatively improving the required measurement times by a factor of 8 compared to currently available photothermal super resolution techniques.

Water Surface Targets Detection Based on the Fusion of Vision and LiDAR.

The use of vision for the recognition of water targets is easily influenced by reflections and ripples, resulting in misidentification. This paper proposed a detection method based on the fusion of 3D point clouds and visual information to detect and locate water surface targets. The point clouds help to reduce the impact of ripples and reflections, and the recognition accuracy is enhanced by visual information. This method consists of three steps: Firstly, the water surface target is detected using the CornerNet-Lite network, and then the candidate target box and camera detection confidence are determined. Secondly, the 3D point cloud is projected onto the two-dimensional pixel plane, and the confidence of LiDAR detection is calculated based on the ratio between the projected area of the point clouds and the pixel area of the bounding box. The target confidence is calculated with the camera detection and LiDAR detection confidence, and the water surface target is determined by combining the detection thresholds. Finally, the bounding box is used to determine the 3D point clouds of the target and estimate its 3D coordinates. The experiment results showed this method reduced the misidentification rate and had 15.5% higher accuracy compared with traditional CornerNet-Lite network. By combining the depth information from LiDAR, the position of the target relative to the detection coordinate system origin could be accurately estimated.

3D Echocardiogram Reconstruction Employing a Flip Directional Texture Pyramid

Three dimensional (3D) echocardiogram enables cardiologists to visualize suspicious cardiac structures in detail. In recent years, this three-dimensional echocardiogram carries important clinical value in virtual surgical simulation. However, this 3D echocardiogram involves a trade-off difficulty between accuracy and efficient computation in clinical diagnosis. This paper presents a novel Flip Directional 3D Volume Reconstruction (FD-3DVR) method for the reconstruction of echocardiogram images. The proposed method consists of two main steps: multiplanar volumetric imaging and 3D volume reconstruction. In the creation of multiplanar volumetric imaging, two-dimensional (2D) image pixels are mapped into voxels of the volumetric grid. As the obtained slices are discontinuous, there are some missing voxels in the volume data. To restore the structural and textural information of 3D ultrasound volume, the proposed method creates a volume pyramid in parallel with the flip directional texture pyramid. Initially, the nearest neighbors of missing voxels in the multiplanar volumetric imaging are identified by 3D ANN (Approximate Nearest Neighbor) patch matching method. Furthermore, a flip directional texture pyramid is proposed and aggregated with distance in patch matching to find out the most similar neighbors. In the reconstruction step, structural and textural information obtained from different flip angle directions can reconstruct 3D volume well with the desired accuracy. Compared with existing 3D reconstruction methods, the proposed Flip Directional 3D Volume Reconstruction (FD-3DVR) method provides superior performance for the mean peak signal-to-noise ratio (40.538 for the proposed method I and 39.626 for the proposed method II). Experimental results performed on the cardiac datasets demonstrate the efficiency of the proposed method for the reconstruction of echocardiogram images.

Feature Extraction Method of EEG Signals Evaluating Spatial Cognition of Community Elderly with Permutation Conditional Mutual Information Common Space Model.

In order to improve the traditional common space pattern (CSP) algorithm pattern in EEG feature extraction, this study proposes a feature extraction method of EEG signals based on permutation conditional mutual information common space pattern (PCMICSP), which used the sum of the permutation condition mutual information matrices of each lead to replacing the mixed spatial covariance matrix in the traditional CSP algorithm, and its eigenvectors and eigenvalues are used to construct a new spatial filter. Then the spatial features in the different time domains and frequency domains are combined to construct the two-dimensional pixel map, Finally, a convolutional neural network (CNN) is used for binary classification. The EEG signals of 7 community elderly before and after spatial cognitive training in virtual reality (VR) scenes were used as the test data set. The average classification accuracy of the PCMICSP algorithm for pre-test and post-test EEG signals is 98%, which was higher than that of CSP based on CMI (conditional mutual information), CSP based on MI (mutual information), and traditional CSP in the combination of four frequency bands. Compared with the traditional CSP method, PCMICSP can be used as a more effective method to extract the spatial features of EEG signals. Therefore, this paper provides a new approach to solving the strict linear hypothesis of CSP and can be used as a valuable biomarker for the spatial cognitive evaluation of the elderly in the community.

Wear patterns in knee OA correlate with native limb geometry.

Background: To date, the amount of cartilage loss is graded by means of discrete scoring systems on artificially divided regions of interest (ROI). However, optimal statistical comparison between and within populations requires anatomically standardized cartilage thickness assessment. Providing anatomical standardization relying on non-rigid registration, we aim to compare morphotypes of a healthy control cohort and virtual reconstructed twins of end-stage knee OA subjects to assess the shape-related knee OA risk and to evaluate possible correlations between phenotype and location of cartilage loss. Methods: Out of an anonymized dataset provided by the Medacta company (Medacta International SA, Castel S. Pietro, CH), 798 end-stage knee OA cases were extracted. Cartilage wear patterns were observed by computing joint space width. The three-dimensional joint space width data was translated into a two-dimensional pixel image, which served as the input for a principal polynomial autoencoder developed for non-linear encoding of wear patterns. Virtual healthy twin reconstruction enabled the investigation of the morphology-related risk for OA requiring joint arthroplasty. Results: The polynomial autoencoder revealed 4 dominant, orthogonal components, accounting for 94% of variance in the latent feature space. This could be interpreted as medial (54.8%), bicompartmental (25.2%) and lateral (9.1%) wear. Medial wear was subdivided into anteromedial (11.3%) and posteromedial (10.4%) wear. Pre-diseased limb geometry had a positive predictive value of 0.80 in the prediction of OA incidence (r 0.58, p < 0.001). Conclusion: An innovative methodological workflow is presented to correlate cartilage wear patterns with knee joint phenotype and to assess the distinct knee OA risk based on pre-diseased lower limb morphology. Confirming previous research, both alignment and joint geometry are of importance in knee OA disease onset and progression.

Hotspot Detection with Machine Learning Based on Pixel-Based Feature Extraction

The complexity of physical verification increases rapidly with fast shrinking technology nodes. Considering only design rule checking (DRC) constraints or lithography models cannot capture the side physical effects in the fabrication process well. Thus, it is desirable to consider a more general physical verification problem with various types of hotspots. In this paper, we apply machine learning which is based on pixel-based feature extraction to deal with the generalized hotspot detection problem. First, a two-dimensional discrete Fourier transformation-based pixel extraction method is proposed to alleviate the shifting effect and produce stable hotspot features. Then, a pattern-based layout scanning approach is developed to enhance the program efficiency while preserving good detection accuracy. Finally, we design two false alarm reduction strategies to effectively reduce the number of detected nonhotspots and further improve the accuracy of hotspot position. Experimental results based on the industrial benchmarks show that our algorithm outperforms three competitive works in terms of accuracy, false alarm rate, efficiency, and time.

An efficient approach for the elevator button manipulation using the visual-based self-driving mobile manipulator

PurposeBecause mobile manipulators are unable to climb stairs, the elevator operation is a crucial capacity to help those kinds of robot systems work in modern multifloor buildings. Here, the elevator button manipulation is considered as an efficient approach to fulfill that requirement. Previously, some studies presented elevator button recognition algorithms while some others designed schemes for the button manipulation work. However, the mobile robot, the manipulator and the camera in their robot systems are asynchronous. Besides, the time-consuming calibration for the camera is inevitable, especially in changeable environments. This paper aims to present an alternative method for the elevator button manipulation to overcome mentioned shortcomings.Design/methodology/approachIn this paper, the elevator button manipulation is conducted by using the visual-based self-driving mobile manipulator in which the autonomous mobile robot, the manipulator and the camera cooperate more efficiently. Namely, the mobile robot does not need to be located exactly in front of the elevator panel as the manipulator has the ability to adjust the initial frame of the camera based on the system kinematic synchronization. In addition, the proposed method does not require the real world coordinates of elevator buttons, but uniquely using their pixel positions. By doing this, not only is the projection from two-dimensional pixel coordinates to three-dimensional (3D) real world coordinates unnecessary, but also the calibration of the camera is not required.FindingsThe proposed method is experimentally verified by using a visual-based self-driving mobile manipulator. This robotic system is the integration of an autonomous mobile robot, a manipulator and a camera mounted on the end-effector of the manipulator.Research limitations/implicationsBecause the surface of the elevator button panel is usually mirror-like, the elevator button detection is easily affected by the glare and the brightness of the environmental light condition.Practical implicationsThis robot system can be used for the goods delivery or the patrol in modern multifloor buildings.Originality/valueThis paper includes three new features: simultaneously detecting and manipulating elevator buttons without the projection from pixel coordinates to 3D real world coordinates, a kinematic synchronization to help the robot system eliminate accumulated errors and a safe human-like elevator button manipulation.

Experimental investigation and numerical simulation for chloride diffusivity of cement paste with elliptical cement particles

The shape of cement particle affects the hydration process of cement through its specific surface area and dispersion morphology, and then exerts great effects on the pore structure characteristic and the chloride ingression of cement paste, while it is generally simplified as the sphere in most of the vector models of cement hydration. To reveal the influence of cement particle shape on chloride diffusivity, a numerical method for quantitative analysis was proposed in this work. According to the size distribution curve and periodic distributions of cement particles, a representative fresh cement paste with elliptical inclusions was firstly constructed. According to the regulations of uniform increase of the thickness of hydration layer around the surface of quasi-ellipse and the interference effects induced by hydration layer overlapping, a vector-based model of cement hydration was established and the influence of aspect ratio of cement particles on the hydration degree was discussed. By considering the diffusion properties of the multi-phase composite cement paste, the first-passage theory suited to two-dimensional pixel model was proposed, and the corresponding Brownian motion algorithm was applied. Using high power microscope, the average aspect ratio of cement particles was calculated. By full comparison of eight groups of experimental results, the effectiveness of this estimation method was verified. At last, the influence of the aspect ratio of elliptical cement particles on chloride diffusivity was drawn.

Two-dimensional beam steering with tunable metasurface in infrared regime

Abstract Tunable metasurfaces can change the optical properties of incident light at will such as amplitude, phase, and polarization in a time-dependent fashion. Ultrafast switching speed and the ability for the pixel size reduction of the tunable metasurface can allow various applications such as light detection and ranging, interferometric sensors, and free space optical communications, to name a few. Although there have been successful demonstrations of the wavefront shaping using the tunable metasurface, the implementation of the two-dimensional metasurface pixel array that can be individually addressed in the optical frequency regime still remains challenging. Here, we present the experimental demonstration of the two-dimensional beam steering with the metasurface array by the binary phase grating in the infrared regime. The metasurface unit cell is composed of metal–dielectric–oxide structure with the indium tin oxide as an active layer, which is modulated by using the top fan-out electrodes. The metasurface array is two-dimensionally pixelated and has the phase change above 137° in the infrared regime.

Untrained Graph Neural Networks for Denoising

A fundamental problem in signal processing is to denoise a signal. While there are many well-performing methods for denoising signals defined on regular domains, including images defined on a two-dimensional pixel grid, many important classes of signals are defined over <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">irregular</i> domains that can be conveniently represented by a graph. This paper introduces two untrained graph neural network architectures for graph signal denoising, develops theoretical guarantees for their denoising capabilities in a simple setup, and provides empirical evidence in more general scenarios. The two architectures differ on how they incorporate the information encoded in the graph, with one relying on graph convolutions and the other employing graph upsampling operators based on hierarchical clustering. Each architecture implements a different prior over the targeted signals. Finally, we provide numerical experiments with synthetic and real datasets that i) asses the denoising behavior predicted by our theoretical results and ii) compare the denoising performance of our architectures with that of existing alternatives.

Searching for metastable particles using graph computing

The reconstruction of charged particle trajectories at the Large Hadron Collider and future colliders relies on energy depositions in sensors placed at distances ranging from a centimeter to a meter from the colliding beams. We propose a method of detecting charged particles that decay invisibly after traversing a short distance of about 25 cm inside the experimental apparatus. One of the decay products may constitute the dark matter known to be 84% of all matter at galactic and cosmological distance scales. Our method uses graph computing to cluster spacepoints recorded by two-dimensional silicon pixel sensors into mathematically-defined patterns. The algorithm may be implemented on silicon-based integrated circuits using field-programmable gate array technology to augment or replace traditional computing platforms.

Development of a scalable tabletop display using projection-based light field technology

We present a light-field display optical system design, state-of-the-art hardware technology, and computational software that are readily scalable (due to their modular structure) and provide naturally immersive and volumetric display systems. We explain the integration of 72 microprojectors into the tabletop display system. In addition, we used 6 workstations for all images and 18 high-performance graphics processing units. Sophisticated image generation through author tooling and the pixel re-arrangement algorithm in the Unity engine enabled us to produce complete three-dimensional images from the radiant rays of two-dimensional pixels using diffuser screens and microlens arrays.

Combining a multi-analyzer stage with a two-dimensional detector for high-resolution powder X-ray diffraction: correcting the angular scale.

In a test experiment, a two-dimensional pixel detector was mounted on the nine-channel multi-analyzer stage of the high-resolution powder diffraction beamline ID22 at the ESRF. This detector replaces a bank of scintillation counters that detect the diffracted intensity passing via the analyzer crystals as the diffractometer arm is scanned. At each diffractometer detector arm angle 2Θ, a 2D image is recorded that displays nine distinct regions of interest corresponding to the diffraction signals transmitted by each of the analyzer crystals. Summing pixels from within each region of interest allows the diffracted intensity to be extracted for each channel. X-rays are diffracted from the sample at various angles, 2θ, into Debye-Scherrer cones. Depending on the azimuthal angle around the cone, diffracted photons satisfy the analyzer-crystal Bragg condition at different diffractometer 2Θ values and arrive on the detector at different horizontal (axial) positions. The more the azimuthal angle deviates from diffraction in the vertical plane, the lower the 2Θ angle at which it is transmitted by an analyzer crystal, and the greater the distance of the detecting pixel from the centerline of the detector. This paper illustrates how the axial resolution afforded by the pixel detector can be used to correct the apparent diffraction angle, 2Θ, given by the diffractometer arm to its true diffraction angle, 2θ. This allows a reduction in peak asymmetry at low angle, and even with a relatively small axial acceptance, the correction leads to narrower peaks than if no correction is applied. By varying axial acceptance with diffraction angle, it is possible to optimize angular resolution at low diffraction angles and counting statistics at high angles. In addition, there is an intrinsic peak broadening with increasing azimuthal angle, dependent on the axial beam and detector pixel sizes. This effect reduces with 2θ, as the curvature of the Debye-Scherrer cones decreases. This broadening can be estimated and used to help choose the axial range to include as a function of diffraction angle.