Pixel Displacement Research Articles

Cardiotoxicity assessment using 3D vascularized cardiac tissue consisting of human iPSC-derived cardiomyocytes and fibroblasts

Human induced pluripotent stem cell-derived cardiomyocytes (hiPSC-CMs) are used for cardiac safety assessment but have limitations for the evaluation of drug-induced contractility. Three-dimensional (3D) cardiac tissues are similar to native tissue and valuable for the assessment of contractility. However, a longer time and specialized equipment are required to generate 3D tissues. We previously developed a simple method to generate 3D tissue in a short period by coating the cell surfaces with extracellular matrix proteins. We hypothesized that this 3D cardiac tissue could be used for simultaneous evaluation of drug-induced repolarization and contractility. In the present work, we examined the effects of several compounds with different mechanisms of action by cell motion imaging. Consequently, human ether-a-go-go-related gene (HERG) channel blockers with high arrhythmogenic risk caused prolongation of contraction-relaxation duration and arrhythmia-like waveforms. Positive inotropic drugs, which increase intracellular Ca2+ levels or myocardial Ca2+ sensitivity, caused an increase in maximum contraction speed (MCS) or average deformation distance (ADD) (ouabain, 138% for MCS at 300 nM; pimobendane, 132% for ADD at 3 μM). For negative inotropic drugs, verapamil reduced both MCS and ADD (61% at 100 nM). Thus, this 3D cardiac tissue detected the expected effects of various cardiovascular drugs, suggesting its usefulness for cardiotoxicity evaluation.

Detection of Bolt Looseness Based on Average Autocorrelation Function

Bolted connections are widely used in multiple engineering fields including aerospace and mechanical engineering due to their numerous advantages like the ability to bear relatively heavy loads, low costs, easy installation, and implementation. Bolt looseness may lead to costly disasters in industries and some cases of injuries. A new method for bolt looseness detection based on average autocorrelation function is proposed. It does not usually directly extract the looseness damage feature from the original response signal to detect the bolt looseness, but it uses the average autocorrelation function value at time lag T = 0 of the vibration pixel displacement signal to establish the looseness damage index. In terms of structural arrangement of this paper, firstly, the theoretical background of the proposed method is given. Then, an experimental system for bolt looseness detection based on computer vision is designed, and a verification experiment is carried out with the bolted connection plate as the experimental object. The results show that the proposed method can effectively obtain the location of the looseness damage of the bolted connection plate, which provides a new technical reference for the online monitoring of the looseness damage of bolted connection plate.

Subpixel Matching Using Double-Precision Gradient-Based Method for Digital Image Correlation.

Digital image correlation (DIC) for displacement and strain measurement has flourished in recent years. There are integer pixel and subpixel matching steps to extract displacement from a series of images in the DIC approach, and identification accuracy mainly depends on the latter step. A subpixel displacement matching method, named the double-precision gradient-based algorithm (DPG), is proposed in this study. After, the integer pixel displacement is identified using the coarse-fine search algorithm. In order to improve the accuracy and anti-noise capability in the subpixel extraction step, the traditional gradient-based method is used to analyze the data on the speckle patterns using the computer, and the influence of noise is considered. These two nearest integer pixels in one direction are both utilized as an interpolation center. Then, two subpixel displacements are extracted by the five-point bicubic spline interpolation algorithm using these two interpolation centers. A novel combination coefficient considering contaminated noises is presented to merge these two subpixel displacements to obtain the final identification displacement. Results from a simulated speckle pattern and a painted beam bending test show that the accuracy of the proposed method can be improved by four times that of the traditional gradient-based method that reaches the same high accuracy as the Newton–Raphson method. The accuracy of the proposed method efficiently reaches at 92.67%, higher than the Newton-Raphon method, and it has better anti-noise performance and stability.

A virtual environment for evaluation of computer vision algorithms under general airborne camera imperfections

AbstractThis paper describes a camera simulation framework for validating machine vision algorithms under general airborne camera imperfections. Lens distortion, image delay, rolling shutter, motion blur, interlacing, vignetting, image noise, and light level are modelled. This is the first simulation that considers all temporal distortions jointly, along with static lens distortions in an online manner. Several innovations are proposed including a motion tracking system allowing the camera to follow the flight log with eligible derivatives. A reverse pipeline, relating each pixel in the output image to pixels in the ideal input image, is developed. It is shown that the inverse lens distortion model and the inverse temporal distortion models are decoupled in this way. A short-time pixel displacement model is proposed to solve for temporal distortions (i.e. delay, rolling shutter, motion blur, and interlacing). Evaluation is done by several means including regenerating an airborne dataset, regenerating the camera path on a calibration pattern, and evaluating the ability of the time displacement model to predict other frames. Qualitative evaluations are also made.

Evaluating the Near and Mid Infrared Bi-Spectral Space for Assessing Fire Severity and Comparison with the Differenced Normalized Burn Ratio

Fire severity, defined as the degree of environmental change caused by a fire, is a critical fire regime attribute of interest to fire emissions modelling and post-fire rehabilitation planning. Remotely sensed fire severity is traditionally assessed by the differenced normalized burn ratio (dNBR). This spectral index captures fire-induced reflectance changes in the near infrared (NIR) and short-wave infrared (SWIR) spectral regions. This study evaluates a spectral index based on a band combination including the NIR and mid infrared (MIR) spectral regions, the differenced normalized difference vegetation index with mid infrared (dNDVIMID), to assess fire severity. This evaluation capitalized upon the unique opportunity stemming from the pre- and post-fire airborne acquisitions over the Rim (2013) and King (2014) fires in California with the MODIS/ASTER Airborne Simulator (MASTER) instrument. The field data consist of 85 Geometrically structured Composite Burn Index (GeoCBI) plots. In addition, six different index combinations, respectively three with a NIR–SWIR combination and three with a NIR–MIR combination, were evaluated based on the optimality of fire-induced spectral displacements. The optimality statistic ranges between zero and one, with values of one representing pixel displacements that are unaffected by noise. The results show that the dNBR demonstrated a stronger relationship with GeoCBI field data when field measurements over the two fire scars were combined than the dNDVIMID approaches. The results yielded an R2 of 0.68 based on a saturated growth model for the best performing dNBR index, whereas the performance of the dNDVIMID indices was lower with an R2 = 0.61 for the best performing dNDVIMID index. The dNBR also outperformed the dNDVIMID in terms of spectral optimality across both fires. The best performing dNBR index yielded median optimality statistics of 0.56 over the Rim and 0.60 over the King fire. The best performing dNDVIMID index recorded optimality values of 0.49 over the Rim and 0.46 over the King fire. We also found that the dNBR approach led to considerable differences in the form of the relationship with the GeoCBI between the two fires, whereas the dNDVIMID approach yielded comparable relationships with the GeoCBI over the two fires. This suggests that the dNDVIMID approach, despite its slightly lower performance than the dNBR, may be a more robust method for estimating and comparing fire severity over large regions. This premise needs additional verification when more air- or spaceborne imagery with NIR and MIR bands will become available with a spatial resolution that allows ground truthing of fire severity.

Study on microvibration effect of an optical satellite based on the imaging absence method

High-accuracy optical satellites may be severely impacted by microvibrations. We propose a simulation and measurement method without an image criterion to evaluate microvibrations that is characterized by low cost and early intervention in a satellite project schedule. First, the relation between image shift value and image quality is identified. Second, frequency response analysis and an optical amplification coefficient are adopted to bridge the relation between mirror assembly displacement and the triggered pixel displacement of the focal plane. Then the close-loop simulation is carried out. Next, the image absence microvibration experiment is conducted. Finally, the disturbance effect is examined using an imaging method and on-orbit test as the contrast experiments. Results indicate that the proposed method has good precision and enables close-loop microvibration analysis without an image as a critical criterion.

Mast Arm Monitoring via Traffic Camera Footage: A Pixel-Based Modal Analysis Approach

Traffic signal mast arm structures must be regularly inspected for cracking, bolt loosening, and other signs of deterioration. Due to large inventories, physical inspections and/or dedicated monitoring systems can be prohibitively time-consuming and expensive to implement at a large scale. However, the growing use of vision-based methods for structural monitoring applications introduces the possibility of leveraging video footage from existing traffic cameras for this purpose. The extraction of dynamic properties (i.e., natural frequencies and damping) from this footage could be employed in detecting possible signs of deterioration. This study presents a vision-based monitoring method which uses a single traffic camera to identify the modal properties of the supporting traffic signal mast arm. This was achieved via operational modal analysis on pixel displacements obtained from a traffic camera mounted on a traffic signal mast arm in Norfolk, VA, monitored during July, 2019. First, sub-pixel displacements were extracted frame-by-frame using weighted centroid tracking of pavement markings. Then, covariance-driven stochastic subspace identification (SSI-Cov) was employed to extract the mast arm fundamental frequencies, damping ratios, and mode shapes. For validation of the vision-based results, SSI-Cov was also applied to acceleration data recorded by two high-sensitivity accelerometers mounted on the structure. In total, the processing was carried out on four different videos and ten acceleration datasets. The vision-based method was able to reliably identify the fundamental frequencies of the structure (Δf < 0.005 Hz mean difference). The associated damping ratios were consistently overestimated but still close in structural terms (Δζ < 0.7% mean difference).

Is Fidgety Philip's ground truth also ours? The creation and application of a machine learning algorithm

BackgroundBehavioral observations support clinical in-depth phenotyping but phenotyping and pattern recognition are affected by training background. As Attention Deficit Hyperactivity Disorder, Restless Legs syndrome/Willis Ekbom disease and medication induced activation syndromes (including increased irritability and/or akathisia), present with hyperactive-behaviors with hyper-arousability and/or hypermotor-restlessness (H-behaviors), we first developed a non-interpretative, neutral pictogram-guided phenotyping language (PG-PL) for describing body-segment movements during sitting. Methodology & resultsThe PG-PL was applied for annotating 12 1-min sitting-videos (inter-observer agreements >85%->97%) and these manual annotations were used as a ground truth to develop an automated algorithm using OpenPose, which locates skeletal landmarks in 2D video. We evaluated the algorithm's performance against the ground truth by computing the area under the receiver operator curve (>0.79 for the legs, arms, and feet, but 0.65 for the head). While our pixel displacement algorithm performed well for the legs, arms, and feet, it predicted head motion less well, indicating the need for further investigations. ConclusionThis first automated analysis algorithm allows to start the discussion about distinct phenotypical characteristics of H-behaviors during structured behavioral observations and may support differential diagnostic considerations via in-depth phenotyping of sitting behaviors and, in consequence, of better treatment concepts.

Phase detection limits in off-axis electron holography from pixelated detectors: gain variations, geometric distortion and failure of reference-hologram correction.

We investigate the effect that recording off-axis electron holograms on pixelated detectors, such as charge-coupled devices (CCD) and direct-detection devices (DDD), can have on measured amplitudes and phases. Theory will be developed for the case of perfectly uniform interference fringes illuminating an imperfect detector with gain variations and pixel displacements. We will show that both these types of defect produce a systematic noise in the phase images that depends on the position of the holographic fringes with respect to the detector. Subtracting a reference hologram from the object hologram will therefore not remove the phase noise if the initial phases of the two holograms do not coincide exactly. Another finding is that pi-shifted holograms are much less affected by gain variations but show no improvement concerning geometric distortions. The resulting phase errors will be estimated and simulations presented that confirm the theoretical developments.

MONITORING MOUNTAIN CRYOSPHERE DYNAMICS BY TIME LAPSE STEREO PHOTOGRAMMETRY

Abstract. In this paper, we propose a method to monitor surface dynamics in mountain cryosphere environments, based on a device with two fixed cameras at two different locations and with a convergent angle. Computer vision methods are used to derive pixel displacements between images taken at different times and to reconstruct the 3D geometry of the scene to convert pixel displacements into meter displacements. The proposed methods overcome the drawbacks of traditional methods such as lack of time repeatability or lack of spatial resolution. We present the results on two study sites instrumented with the device located in the French Alps: a temperate glacier and a rock glacier. We propose different ways to assess the accuracy of each processing steps solely based on stereo images. The method is validated using traditional measurements (GPS and LiDAR) and shows results comparable or even more robust than these traditional methods.

CrossNet++: Cross-Scale Large-Parallax Warping for Reference-Based Super-Resolution.

The ability of camera arrays to efficiently capture higher space-bandwidth product than single cameras has led to various multiscale and hybrid systems. These systems play vital roles in computational photography, including light field imaging, 360 VR camera, gigapixel videography, etc. One of the critical tasks in multiscale hybrid imaging is matching and fusing cross-resolution images from different cameras under perspective parallax. In this paper, we investigate the reference-based super-resolution (RefSR) problem associated with dual-camera or multi-camera systems. RefSR consists of super-resolving a low-resolution (LR) image given an external high-resolution (HR) reference image, where they suffer both a significant resolution gap ( 8×) and large parallax ( ∼10% pixel displacement). We present CrossNet++, an end-to-end network containing novel two-stage cross-scale warping modules, image encoder and fusion decoder. The stage I learns to narrow down the parallax distinctively with the strong guidance of landmarks and intensity distribution consensus. Then the stage II operates more fine-grained alignment and aggregation in feature domain to synthesize the final super-resolved image. To further address the large parallax, new hybrid loss functions comprising warping loss, landmark loss and super-resolution loss are proposed to regularize training and enable better convergence. CrossNet++ significantly outperforms the state-of-art on light field datasets as well as real dual-camera data. We further demonstrate the generalization of our framework by transferring it to video super-resolution and video denoising.

Research on Visual Control System of Inverted Pendulum Based on Pixel Displacement

In inverted pendulum visual servo control system (IPVSCSs), how to achieve the fast and effective measurement of the state information of the inverted pendulum is an important task. In this paper, the characteristics of the inverted pendulum are considered and a series of image processing and positioning algorithm of the pendulum are proposed. Furthermore, the inverted pendulum control system model based on the pixel displacement is established and the LQR controller is designed. Finally, the feasibility and superiority of the proposed method are verified by real-time control experiments.

Building Extraction from High-Resolution Remote Sensing Images Based on GrabCut with Automatic Selection of Foreground and Background Samples

This article proposes a new building extraction method from high-resolution remote sensing images, based on GrabCut, which can automatically select foreground and background samples under the constraints of building elevation contour lines. First the image is rotated according to the direction of pixel displacement calculated by the rational function Model. Second, the Canny operator, combined with morphology and the Hough transform, is used to extract the building's elevation contour lines. Third, seed points and interesting points of the building are selected under the constraint of the contour line and the geodesic distance. Then foreground and background samples are obtained according to these points. Fourth, GrabCut and geometric features are used to carry out image segmentation and extract buildings. Finally, WorldView satellite images are used to verify the proposed method. Experimental results show that the average accuracy can reach 86.34%, which is 15.12% higher than other building extraction methods.

Mover Position Detection for PMSLM Based on Line-Scanning Fence Pattern and Subpixel Polynomial Fitting Algorithm

The article presents a rapid and precision position detection method for permanent magnet synchronous linear motors (PMSLM) based on digital image measurement and line-scanning photography. An exclusive fence pattern is designed as the target image for image measurement method, according to the motion feature of PMSLM. A line scan camera is installed on the mover to record image sequences instantaneously with the movement of linear motor, and from which the pixel displacement can be obtained using an image matching algorithm. To further improve the measurement accuracy, a subpixel polynomial fitting algorithm is proposed based on the line-scanning fence patterns. Finally, the displacement can be calculated according to system magnification parameters. Compared with the traditional image measurement method based on the area-array speckle pattern, both the cost and time delay of the proposed method are reduced greatly by employing line-scanning photography. The detection accuracy is also improved by four times. To improve the measurement robustness, an assessment index of fence pattern is proposed to optimize the fence image. The measurement error and robustness are analyzed in theory and simulation, and it demonstrates that the fence pattern shows stronger resistance to scanning deviation than speckle pattern. A mover position detection platform is established, and the experimental results show that the proposed method can achieve position estimation with average error of 0.005 mm under different working conditions.

Precision Position Measurement of Linear Motors Mover Based on Temporal Image Correlation

High-precision position measurement is very important for precision motion control of a linear motor. This paper presents an accurate mover position detection method with high precision stability based on temporal image correlation and fence image for the linear motors. First, sequence fence images with the information of mover position are recorded by a position measurement system for linear motor, and the integer pixel displacement of each image is obtained by image correlation method. Then, according to the continuity of the motion, the subpixel displacement can be extracted by fitting the integer pixel displacement. Finally, the displacement and velocity curves can be calculated according to system calibration. Compared to digital image measurement used in position detection of linear motors, the proposed method can detect mover position accurately with longer measurement range and smaller uncertainty. The moving least square (MLS) algorithm is used to ensure the stability of measurement uncertainty under different motion modes. Simulations and mover position detection experiments with different motion conditions were performed to demonstrate the effectiveness of the proposed method in linear motors. In this paper, the average measurement error of displacement is 0.006 mm with the maximum displacement of 120 mm. The maximum and average relative errors are less than 0.06% and 0.017%, respectively.

Attitude-Trajectory Estimation for Forward-Looking Multibeam Sonar Based on Acoustic Image Registration

This work considers the processing of acoustic data from a multibeam forward-looking sonar (FLS) on a moving underwater platform to estimate the platform's attitude and trajectory. We propose an algorithm to produce an estimate of the attitude trajectory for an FLS based on the optical flow between consecutive sonar frames. The attitude trajectory can be used to locate an underwater platform, such as an autonomous underwater vehicle, to a degree of accuracy suitable for navigation. It can also be used to build a mosaic of the underwater scene. The estimation is performed in three steps. First, a selection of techniques based on the optical flow model is used to estimate a pixel displacement map (DM) between consecutive sonar frames represented in the native polar (range/bearing) format. The second step finds the best match between the estimated DM and DMs for a set of modeled sonar sensor motions. To reduce complexity, it is proposed to describe the DM with a small parameter vector derived from the displacement distribution. Thus, an estimate of the incremental sensor motion between frames is made. Finally, using a weighted regularized spline technique, the incremental interframe motions are integrated into an attitude trajectory for the sonar sensor. To assess the accuracy of the attitude-trajectory estimate, it is used to register FLS frames from a field experiment data set and build a high-quality mosaic of the underwater scene.

Synthetic Aperture Sonar Track Registration With Motion Compensation for Coherent Change Detection

The capability to detect changes in an underwater scene has many applications, including environmental monitoring, surveillance of strategic maritime waterways, and naval mine countermeasures. This paper examines the automated detection of changes in repeat-pass synthetic aperture sonar (SAS) imagery by exploiting the loss of phase coherence caused by differences in the scene. A renavigation method based on track registration is developed for performing the coregistration of images. The method first determines the corrections to be applied to the repeat-pass navigation data by optimizing a linear track model using the estimated across-track pixel displacements. A second optimization is then performed using the along-track pixel displacements to determine a velocity correction for each ping location of the repeat-pass image to compensate for residual motion errors. The corrected navigation solution is then used to rebeamform the repeat-pass image onto the same focal points as the reference image. The method is demonstrated on a pair of repeat-pass SAS images acquired using the AquaPix INSAS2 sonar, where it is shown that it is not necessary to have access to raw element data to coregister images accurately enough for coherent change detection requirements. This technique is then applied to the second data set obtained using the HISAS 1030 sonar for the purpose of comparing both coherent and noncoherent (amplitude-only) approaches to change detection in scenes with low and high clutters. False alarms caused by the acoustic shadows of objects in high-clutter areas are mitigated using the single-pass interferometric coherence as a reference mask. The results show the successful use of the repeat-pass coherence to detect changes between SAS images with temporal baselines of several days on scenes consisting of mud, gravel, and sand, and demonstrate that a coherent approach is capable of detecting large-scale differences (caused by the introduction or removal of targets) as well as subtle scene changes that are not detectable using noncoherent methods.

Delineation of a Cardiac Planning Organ-At-Risk Volume Using Real-Time Magnetic Resonance Imaging for Cardiac Protection in Thoracic and Breast Radiation Therapy

PurposeCardiac radiation is associated with cardiotoxicity in patients with thoracic and breast malignancies. We conducted a prospective study using cine magnetic resonance imaging (MRI) scans to evaluate heart motion. We hypothesized that cine MRI could be used to define population-based cardiac planning organ-at-risk volumes (PRV). Methods and MaterialsA total of 16 real-time acquisitions were obtained per subject on a 1.5 Tesla MRI (Philips Ingenia). Planar cine MRI was performed in 4 sequential sagittal and coronal planes at free-breathing (FB) and deep-inspiratory breath hold (DIBH). In-plane cardiac motion was assessed using a scale-invariant feature transformation–based algorithm. Subject-specific pixel motion ranges were defined in anteroposterior (AP), left-right (LR), and superoinferior (SI) planes. Averages of the 98% and 67% of the maximum ranges of pixel displacement were defined by subject, then averaged across the cohort to calculate PRV expansions at FB and DIBH. ResultsData from 20 subjects with a total of 3120 image frames collected per subject in coronal and sagittal planes at DIBH and FB, and 62,400 total frames were analyzed. Cohort averages of 98% of the maximum cardiac motion ranges comprised margin expansions of 12.5 ± 1.1 mm SI, 5.8 ± 1.2 mm AP, and 6.6 ± 1.0 mm LR at FB and 6.7 ± 1.5 mm SI, 4.7 ± 1.3 mm AP, and 5.3 ± 1.3 mm LR at DIBH. Margins for 67% of the maximum range comprised 7.7 ± 0.7 mm SI, 3.2 ± 0.6 mm AP, and 3.7 ± 0.6 mm LR at FB and 4.1 ± 0.9 mm SI, 2.7 ± 0.8 mm AP, and 3.2 ± 0.8 mm LR at DIBH. Subsequently, these margins were simplified to form PRVs for treatment planning. ConclusionsWe implemented scale-invariant feature transformation-based motion tracking for analysis of the cardiac cine MRI scans to quantify motion and create cohort-based cardiac PRVs to improve cardioprotection in breast and thoracic radiation.

A fast video super resolution for facial image

Multi-frame super resolution has found various applications in different domains of machine vision such as remote sensing, object recognition, and security applications for the last two decades. Classic super resolution methods are not able to handle real word videos where different parts of scene undergo different motions. Most of recent methods in the literature address this problem but they suffer from time complexity. In this paper, we propose a fast method for super resolution of facial videos. Our proposed method provides less computational complexity in addition to handling videos having general motion patterns. These two benefits make the proposed method suitable for security purposes. In the proposed method, first we extract a number of key points from face in each video frame. Then, for each pixel in the reference frame the corresponding pixels in other frames are determined using triangular patches. Subsequently, the obtained solution is improved by minimizing an energy function considering both appearance and pixel displacements. Super resolved facial image is finally obtained by using information available in a small window around approximated location of the low resolution frames. The effectiveness of the proposed method has been demonstrated for real video sequences.

The role of parallel imaging technique in decreasing geometric distortion artifact in diffusion weighted imaging of the brain

IntroductionThe purpose of this study was to evaluate the role of parallel imaging technique in obtaining a more accurate geometric representation of the studied anatomy and how far this technique can reduce the pixel displacement and increase resolution in echo planner imaging- diffusion weighted (EPI-DW) images of the brain. Patients & methodsThe study was done on 6 patients, they were subjected to routine brain MRI examination for different symptoms, examination was done on 1.5 T scanner, the degree of geometric distortion was measured in the phase encoding direction (from anterior to posterior (A/P) in the used sequence) in both DWI sequence acquired without and with parallel imaging technique respectively and was compared with T2-FSE sequence which has no geometric distortion as in EPI. ResultsParallel imaging technique reduced artifact (pixel displacement) in frontal and occipital lobes in addition to increasing resolution for EPI-DWI for the brain. This technique reduced pixel displacement in the frontal lobe by 47 ± 11.7% and 47.3 ± 11% reduction in the occipital lobe. ConclusionParallel imaging technique can be used with EPI-DWI to increase resolution and decrease geometric distortion artifact to acquire more accurate geometric representation of the acquired anatomy.