Motion Vectors Research Articles

Respiratory motion management using a single rapid MRI scan for a 0.35 T MRI-Linac system.

MRI has a rapidly growing role in radiation therapy (RT) for treatment planning, real-time image guidance, and beam gating (e.g., MRI-Linac). Free-breathing 4D-MRI is desirable in respiratory motion management for therapy. Moreover, high-quality 3D-MRIs without motion artifacts are needed to delineate lesions. Existing MRI methods require multiple scans with lengthy acquisition times or are limited by low spatial resolution, contrast, and signal-to-noise ratio. We developed a novel method to obtain motion-resolved 4D-MRIs and motion-integrated 3D-MRI reconstruction using a single rapid (35-45 s scan on a 0.35 T MRI-Linac. Golden-angle radial stack-of-stars MRI scans were acquired from a respiratory motion phantom and 12 healthy volunteers (n=12) on a 0.35 T MRI-Linac. A self-navigated method was employed to detect respiratory motion using 2000 (acquisition time=5-7 min) and the first 200 spokes (acquisition time=35-45 s). Multi-coil non-uniform fast Fourier transform (MCNUFFT), compressed sensing (CS), and deep-learning Phase2Phase (P2P) methods were employed to reconstruct motion-resolved 4D-MRI using 2000 spokes (MCNUFFT2000) and 200 spokes (CS200 and P2P200). Deformable motion vector fields (MVFs) were computed from the 4D-MRIs and used to reconstruct motion-corrected 3D-MRIs with the MOtion Transformation Integrated forward-Fourier (MOTIF) method. Image quality was evaluated quantitatively using the structural similarity index measure (SSIM) and the root mean square error (RMSE), and qualitatively in a blinded radiological review. Evaluation using the respiratory motion phantom experiment showed that the proposed method reversed the effects of motion blurring and restored edge sharpness. In the human study, P2P200 had smaller inaccuracy in MVFs estimation than CS200. P2P200 had significantly greater SSIMs (p<0.0001) and smaller RMSEs (p<0.001) than CS200 in motion-resolved 4D-MRI and motion-corrected 3D-MRI. The radiological review found that MOTIF 3D-MRIs using MCNUFFT2000 exhibited the highest image quality (scoring>8 out of 10), followed by P2P200 (scoring>5 out of 10), and then motion-uncorrected (scoring<3 out of 10) in sharpness, contrast, and artifact-freeness. We have successfully demonstrated a method for respiratory motion management for MRI-guided RT. The method integrated self-navigated respiratory motion detection, deep-learning P2P 4D-MRI reconstruction, and a motion integrated reconstruction (MOTIF) for 3D-MRI using a single rapid MRI scan (35-45 s) on a 0.35 T MRI-Linac system.

Detection of clouds in multiple wind velocity fields using ground-based infrared sky images

To improve the performance of an intra-hour global solar irradiance forecasting algorithm, it is important to detect multiple layers of clouds. Horizontal atmospheric wind shear causes wind velocity fields to have different directions and speeds. In images of clouds acquired using ground-based sky imagers, clouds may be moving in different wind layers. The information provided by a solar forecasting algorithm is necessary to optimize and schedule the solar generation resources and storage devices in a smart grid. This investigation studies the performance of unsupervised learning techniques when detecting the number of cloud layers in infrared sky images. The images are acquired using an innovative infrared sky imager mounted on a solar tracker. Different mixture models are used to infer the distribution of the cloud features. Multiple Bayesian metrics and a sequential hidden Markov model are implemented to find the optimal number of clusters in the mixture models, and their performances are compared. The motion vectors are computed using a probabilistic implementation of the Lucas-Kanade algorithm. The correlations between the cloud motion vectors and temperatures are analyzed to discover the method that leads to the most accurate results. The findings point that a sequential hidden Markov model outperforms the detection accuracy of standard Bayesian model selection metrics.

Faint but Not Forgotten. I. First Results from a Search for Astrospheres around AGB Stars in the Far-ultraviolet

Using the GALEX archive, we have discovered extended structures around ten asymptotic giant branch (AGB) stars (out of a total 92 searched) emitting in the far-ultraviolet (FUV) band. In all but one, we find the typical morphology expected for a spherical wind moving relative to, and interacting with, the interstellar medium (ISM) to produce an astrosphere. The exception is V Hya whose mass ejection is known to be highly aspherical, where we find evidence of its large parabolic outflows interacting with the ISM, and its collimated, extreme velocity outflows interacting with the circumstellar medium. For eight objects with relatively large proper motions, we find (as expected) that the termination-shock region lies in a hemisphere that contains the proper motion vector. Radial intensity cuts for each source have been used to locate the termination shock and the astropause’s outer edge. In a few objects, the cuts also reveal faint emission just outside the astropause that likely arises in shocked ISM material. We have used these data, together with published mass-loss rates and wind expansion velocities, to determine the total mass lost and duration for each source—we find that the duration of and total mass in the shocked wind are significantly larger than their corresponding values for the unshocked wind. The combination of FUV and far-IR data on AGB astrospheres provides a unique database for theoretical studies (numerical simulations) of wind–ISM interactions. We show that a Cyclical Spatial Heterodyne Spectrometer on a small space-based telescope can provide high-resolution spectra of astrospheres to confirm the emission mechanism.

Automated nonlinear registration of coronary PET to CT angiography using pseudo-CT generated from PET with generative adversarial networks

BackgroundCoronary 18F-sodium-fluoride (18F-NaF) positron emission tomography (PET) showed promise in imaging coronary artery disease activity. Currently image processing remains subjective due to the need for manual registration of PET and computed tomography (CT) angiography data. We aimed to develop a novel fully automated method to register coronary 18F-NaF PET to CT angiography using pseudo-CT generated by generative adversarial networks (GAN). MethodsA total of 169 patients, 139 in the training and 30 in the testing sets were considered for generation of pseudo-CT from non-attenuation corrected (NAC) PET using GAN. Non-rigid registration was used to register pseudo-CT to CT angiography and the resulting transformation was used to align PET with CT angiography. We compared translations, maximal standard uptake value (SUVmax) and target to background ratio (TBRmax) at the location of plaques, obtained after observer and automated alignment. ResultsAutomatic end-to-end registration was performed for 30 patients with 88 coronary vessels and took 27.5 seconds per patient. Difference in displacement motion vectors between GAN-based and observer-based registration in the x-, y-, and z-directions was 0.8 ± 3.0, 0.7 ± 3.0, and 1.7 ± 3.9 mm, respectively. TBRmax had a coefficient of repeatability (CR) of 0.31, mean bias of 0.03 and narrow limits of agreement (LOA) (95% LOA: − 0.29 to 0.33). SUVmax had CR of 0.26, mean bias of 0 and narrow LOA (95% LOA: − 0.26 to 0.26). ConclusionPseudo-CT generated by GAN are perfectly registered to PET can be used to facilitate quick and fully automated registration of PET and CT angiography.

Games of children of senior preschool age as early career guidance in activities

Early preparation of a child for the choice of a future profession is not a recognition of who the child should be, but an acquaintance of the child with various activities. This may make it easier to choose in the future. Therefore, acquaintance with the activities of adults should begin at preschool age, when children learn about various professions through accessible forms of knowledge. As a result, the child learns the values of labor, gets an idea about various professions, interests in certain types of professional activity begin to form in their mind. In order for a child to make a conscious choice in adult life, professional orientation should begin with a close environment, first of all, with the professions of parents. In this regard, the purpose of the article is to develop mechanisms for early professional orientation of preschoolers through play activities. For the study, methods of ascent from the abstract to the real were applied, the method of transforming real images of objects based on their abstract meaning. As a result of the study, conclusions were drawn that the personal potential of a person in relation to life and the world of activity is manifested in the positions of "individual", "child" and "subject"Confidence in thinking is given by the application of motion vectors to the known abstract, in the logic of ascent from the abstract to the concrete, as well as in their combinations. The cycle of revealing the potential of the individual and its transformation into the subject of educational activity is described. A sufficient prerequisite is the mastery of the subjects of reflexive-thinking abilities in accordance with self-determination, self-organization, self-regulation

A visual SLAM algorithm based on fuzzy clustering for removing dynamic features

Most visual simultaneous localization and mapping (SLAM) algorithms assume that no or only few moving objects occur in application environments. This assumption makes the algorithms vulnerable to the interference of moving objects in dynamic environments. To address the problem, a new visual SLAM method, which could eliminate dynamic features without any prior information, was proposed. By measuring the position of each feature point and its motion vector difference between image sequences, a two-stage clustering was performed on the feature points in the field of view. This method removed the features detected on moving objects, and used a static initialization technique to eliminate the dependence of SLAM on prior information. The proposed method intended to improve OV2SLAM (a fully online and versatile visual SLAM for real-time applications) algorithm, and the experimental verification was carried out. Our results show that while maintaining the real-time performance of the original OV2SLAM algorithm, the positioning accuracy and robustness of the proposed method is improved in a dynamic environment.

Performance of a Hybrid Gain Ensemble Data Assimilation Scheme in Tropical Cyclone Forecasting with the GRAPES Model

Hybrid data assimilation (DA) methods have received extensive attention in the field of numerical weather prediction. In this study, a hybrid gain data assimilation (HGDA) method that combined the gain matrices of ensemble and variational methods was first applied in the mesoscale version of the Global/Regional Assimilation and Prediction System (GRAPES_Meso). To evaluate the performance of the HGDA method in the GRAPES_Meso model, different DA schemes, including the three-dimensional variational (3DVAR), local ensemble transform Kalman filter (LETKF), and HGDA schemes, were compared across eight tropical cyclone (TC) cases, and FY-4A atmospheric motion vectors were assimilated. The results indicated that the HYBRID scheme outperformed the 3DVAR and LETKF schemes in TC position forecasting, and with ensemble forecasting techniques, the HYBRID scheme promoted the accuracy of the prediction TC intensity. The threat score (TS) values for the light and medium precipitation forecasts obtained in the HYBRID experiment were higher than those for the forecasts obtained in the 3DVAR and LETKF experiments, which may be attributed to the forecasting accuracy for the TC position. Regarding heavy and extreme rainfall, the HYBRID scheme achieved a more stable effect than those of the 3DVAR and LETKF schemes. The results demonstrated the superiority of the HGDA scheme in TC prediction with the GRAPES_Meso model.

Optimizing cloud motion estimation on the edge with phase correlation and optical flow

Abstract. Phase correlation (PC) is a well-known method for estimating cloud motion vectors (CMVs) from infrared and visible spectrum images. Commonly, phase shift is computed in the small blocks of the images using the fast Fourier transform. In this study, we investigate the performance and the stability of the blockwise PC method by changing the block size, the frame interval, and combinations of red, green, and blue (RGB) channels from the total sky imager (TSI) at the United States Atmospheric Radiation Measurement user facility's Southern Great Plains site. We find that shorter frame intervals, followed by larger block sizes, are responsible for stable estimates of the CMV, as suggested by the higher autocorrelations. The choice of RGB channels has a limited effect on the quality of CMVs, and the red and the grayscale images are marginally more reliable than the other combinations during rapidly evolving low-level clouds. The stability of CMVs was tested at different image resolutions with an implementation of the optimized algorithm on the Sage cyberinfrastructure test bed. We find that doubling the frame rate outperforms quadrupling the image resolution in achieving CMV stability. The correlations of CMVs with the wind data are significant in the range of 0.38–0.59 with a 95 % confidence interval, despite the uncertainties and limitations of both datasets. A comparison of the PC method with constructed data and the optical flow method suggests that the post-processing of the vector field has a significant effect on the quality of the CMV. The raindrop-contaminated images can be identified by the rotation of the TSI mirror in the motion field. The results of this study are critical to optimizing algorithms for edge-computing sensor systems.

Weakly supervised multi-class semantic video segmentation for road scenes

Weakly supervised multi-class video segmentation is one of the most challenging yet least studied research problems in computer vision. This study aims to investigate two main items: (1) effective feature update for temporal changes combined with feature reuse between temporal frames; and (2) learn object patterns in complex scenes specifically for videos under weak supervision. Associating image tags to visual appearance is not a straightforward learning task, especially for complex scenes. Therefore, in this paper, we present manifold augmentations to obtain reliable pixel labels from image tags. We propose a framework comprised of two key modules: a temporal split module for efficient video processing and a pseudo per-pixel seed generation module for precise pixel-level supervision. Particularly, in our model, we utilize and explore temporal correlations via temporal split module and temporal attention. To reuse the extracted features and incorporate temporal updates for precise and fast computation, a channel-wise temporal split mechanism between successive video frames is presented. Furthermore, we evaluated proposed modules in two additional settings: (1) fully or sparsely supervised road scene video segmentation; and (2) weakly supervised segmentation for complex road scene images. Experiments are conducted on the Cityscapes and CamVid datasets, using DeepLabv3 as segmentation network and LiteFlowNet to compute motion vectors.

Spatiotemporal directional coherence-based scrolling-text detection for frame rate up-conversion

This paper proposes a novel scrolling-text detection method that uses spatiotemporal directional coherence for frame rate up-conversion. Spatiotemporal directional coherence is defined as the coherence level of the spatiotemporal gradient distribution in consecutive frames. Existing scrolling-text detection methods usually detect text using the motion vectors of an image or the distributions of gradient components. The number of motion vectors to determine the scrolling-text decreases at the start and end points of the frame boundary. Therefore, they have difficulty accurately detecting the scrolling text from the start or end points of a frame boundary. The distributions of gradient components can be generated by non-scrolling-text components and they cannot consider the temporal information of consecutive frames, so they may erroneously detect non-scrolling-text regions as the scrolling-text. Unlike the previous methods, which are vulnerable to the presence of texture and noise, the core idea of the proposed method is to use the spatiotemporal directional coherence of the gradient components in the consecutive frames to detect the regions with a dominant edge orientation component for generation of the scrolling-text map and use bit codes from the luminance values to analyze the diversity of luminance patterns for refinement process. With these, the proposed method can further improve the detection accuracy compared to the benchmark methods. The experimental results showed that the proposed method enhanced the average F1 score by up to 0.5195 (a 137.69 % improvement) compared to the benchmark methods. The average computation time per pixel of the proposed method was also reduced by up to 16.205 μs (a 70.57 % reduction) compared to the benchmark methods.

Optical Flow-Based Algorithm Analysis to Detect Human Emotion from Eye Movement-Image Data

One of the popular methods for the recognition of human emotions such as happiness, sadness and shock is based on the movement of facial features. Motion vectors that show these movements can be calculated by using optical flow algorithms. In this method, for detecting emotions, the resulted set of motion vectors is compared with a standard facial movement template caused by human emotional changes. In this paper, a new method is introduced to compute the quantity of likeness towards a particular emotion to make decisions based on the importance of obtained vectors from an optical flow approach. The current study uses a feature point tracking technique separately applied to the five facial image regions (eyebrows, eyes, and mouth) to identify basic emotions. Primarily, this research will be focusing on eye movement regions. For finding the vectors, one of the efficient optical flow methods is using the pre-experiment as explained further below.

УЧЕТ СКОРРЕЛИРОВАННОСТИ ОШИБОК СПУТНИКОВЫХ ДАННЫХ НАБЛЮДЕНИЙ AMV В АНСАМБЛЕВОЙ СИСТЕМЕ УСВОЕНИЯ ДАННЫХ

The effect of considering correlated errors in AMV (Atmospheric Motion Vectors) satellite observations of wind in the local ensemble transform Kalman filter data assimilation system is studied. It is customary to use a diagonal covariance matrix of errors in observations taking part in assimilation. When assimilating satellite data with correlated errors, the data are thinned, and the values of diagonal elements of the error covariance matrix are often overestimated. This is accompanied by the loss of useful information about the correlation between the errors. The present study uses a different method: the elements of the error covariance matrix for AMV satellite observations are simulated using a second-order autoregressive function. It is shown that such approach reduces the root-mean-square error in initial data for a numerical weather prediction model, in particular on small scales, and improves the forecast quality. It is found that the application of the non-diagonal AMV observation error covariance matrix increases the accuracy of analysis and forecast fields.

Using Model Cloud Information to Reassign Low-Level Atmospheric Motion Vectors in the ECMWF Assimilation System

Abstract This paper investigates the use of model cloud information in the assimilation of low-level atmospheric motion vectors (AMVs) in the ECMWF global data assimilation system, with the aim to characterize and address issues encountered in the assimilation of these observations. An analysis of background departure statistics (comparison of observations with the model background) shows that AMVs placed above the model cloud show larger deviations from the model background relative to those placed unrealistically close to the surface. Reassigning the pressure of AMVs diagnosed above the model cloud layer to either the model cloud top, cloud base, or average pressure leads to improvements in root-mean-square vector difference (RMSVD) and speed bias against the background wind fields. In assimilation experiments, reassigning AMVs placed above the model cloud to the model cloud top, cloud base, or average pressure results overall in a positive impact on subsequent forecasts. The reassignment to an average model cloud pressure performs best in this respect, and this approach has been implemented in the operational ECMWF system in October 2021.

OAOF - Obstacle Avoidance using Optical Flow Algorithm for Unmanned Aerial Vehicles

A camera sensor may be a useful tool for a number of robotic and autonomous systems when used with MATLAB Simulink for obstacle avoidance. Using a camera sensor and MATLAB Simulink's optical flow algorithm, the obstacle avoidance system is shown in this work. For the purpose of identifying and following the motion of objects in the picture, the system analyses visual data from the camera. The motion vectors of the objects are computed using the optical flow technique, and the robot and obstacles' relative motion is calculated using those motion vectors. In order to provide safe and easy navigation, the system can recognise and avoid obstacles in real-time. The system is easily adjustable and adaptable to diverse applications thanks to the usage of MATLAB Simulink, which enables the integration of several sensor and control systems. The suggested system's findings demonstrate that using a camera sensor with the optical flow algorithm may be a useful tool for obstacle avoidance in a range of robotic and autonomous systems.

Improving the predictability of the Qendresa Medicane by the assimilation of conventional and atmospheric motion vector observations. Storm-scale analysis and short-range forecast

Abstract. The coastal population in the western Mediterranean Basin is frequently affected by high-impact weather events that produce huge economic and human losses. Among the wide spectrum of maritime severe weather events, tropical-like Mediterranean cyclones (a.k.a. medicanes) draw particular attention, specially due to their poor predictability. The accurate prediction of this kind of event still remains a key challenge to the weather forecast community, mainly because of (i) errors in the initial conditions, (ii) lack of accuracy of modeling micro-scale physics processes and (iii) chaotic behavior inherent to numerical weather prediction models. The 7 November 2014 Qendresa Medicane, that took place over the Sicilian channel affecting the islands of Lampedusa, Pantelleria and Malta, was selected for this study because of its extremely low predictability behavior in terms of its track and intensity. To enhance the prediction of Qendresa, a high-resolution (4 km) ensemble-based data assimilation technique, known as ensemble Kalman filter (EnKF), is used. In this study, both in situ conventional and satellite-derived observations are assimilated with the main objective of improving Qendresa's model initial conditions and thus its subsequent forecast. The performance of the EnKF system and its impact on the Qendresa forecast are quantitatively assessed using different deterministic and probabilistic verification methods. A discussion in terms of the relevant physical mechanisms adjusted by the EnKF is also provided. Results reveal that the assimilation of both conventional and satellite-derived observations improves the short-range forecasts of the trajectory and intensity of Qendresa. In this context, the relevance of assimilating satellite-derived observations to improve the pre-convective estimation of Qendresa's upper-level dynamics is shown, which is key to obtain a realistic track and intensity forecast of this event.

A fast vanishing point detection method based on row space features suitable for real driving scenarios

The vanishing point (VP) is particularly important road information, which provides an important judgment criterion for the autonomous driving system. Existing vanishing point detection methods lack speed and accuracy when dealing with real road environments. This paper proposes a fast vanishing point detection method based on row space features. By analyzing the row space features, clustering candidates for similar vanishing points in the row space are performed, and then motion vectors are screened for the vanishing points in the candidate lines. The experimental results show that the average error of the normalized Euclidean distance is 0.0023716 in driving scenes under various lighting conditions. The unique candidate row space greatly reduces the amount of calculation, making the real-time FPS up to 86. It can be concluded that the fast vanishing point detection proposed in this paper would be suitable for high-speed driving scenarios.

Fractal video compression for IOT-based smart cities applications using motion vector estimation

Recently, Internet of the things (IoTs) devices have been used highly in surveillance applications in smart environments, cities, and buildings. Most of the surveillance data recorded by the IoT devices is in the form of video and needs to be transmitted to the admin or server by video streaming technology. Fractal video compression uses a self-similarity concept for video compression, meaning the fractal image contains self-similarity of itself that is explained and denoted by change. Computational cost is more in fractal video coding; different methods have been developed to reduce this computational cost. The proposed work's main objective is to implement a video streaming approach by combining the Diamond-search-pattern- block-matching motion estimation algorithm and hash-based fractal video compression algorithm to reduce the encoding time. It improves both the delay and security of the block matching system and the developed approach measures based on evaluation parameters: Peak signal-to-noise ratio (PSNR), encoding time, decoding time, Mean squared error (MSE), and Compression ratio for Smart City Applications. Compared to other algorithms, the IoTs video data is streamed efficiently in a smart city to analyse the abnormalities.

Vision-based biomechanical markerless motion classification

This study used stick model augmentation on single-camera motion video to create a markerless motion classification model of manual operations. All videos were augmented with a stick model composed of keypoints and lines by using the programming model, which later incorporated the COCO dataset, OpenCV and OpenPose modules to estimate the coordinates and body joints. The stick model data included the initial velocity, cumulative velocity, and acceleration for each body joint. The extracted motion vector data were normalized using three different techniques, and the resulting datasets were subjected to eight classifiers. The experiment involved four distinct motion sequences performed by eight participants. The random forest classifier performed the best in terms of accuracy in recorded data classification in its min-max normalized dataset. This classifier also obtained a score of 81.80% for the dataset before random subsampling and a score of 92.37% for the resampled dataset. Meanwhile, the random subsampling method dramatically improved classification accuracy by removing noise data and replacing them with replicated instances to balance the class. This research advances methodological and applied knowledge on the capture and classification of human motion using a single camera view.

Efficient Video Steganalytic Feature Design by Exploiting Local Optimality and Lagrangian Cost Quotient

As the opponent of motion vector (MV)-based video steganography, the corresponding symmetric steganalysis has also developed a lot in recent years, among which the logic-based steganalytic schemes, e.g., AoSO, NPELO and MVC, are the most prevailing. Although currently achieving the best detection performance, these steganalytic schemes are less effective in detecting some logic-maintaining steganographic schemes. In view of the fact that the distributions of covers’ local Lagrangian cost quotients are normally more concentrated in the small value ranges than those of stegos and “spread” to the large values ranges after modifying the motion vector, the local Lagrangian cost quotient would thus be an efficient indicator to reflect the difference between cover videos and stego ones. In this regard, combining the logic-based (Lg) and local Lagrangian cost quotient (LLCQ)-based feature, we finally proposed a more effective and general steganalysis feature, i.e., Lg-LLCQ, which is composed of diverse subfeatures and performs much better than the corresponding single-type feature. Extensive experimental results show that the proposed method exhibits detection performance superior to other state-of-the-art schemes and even works well under cover sources and steganographic scheme mismatch scenes, which indicates our proposed feature is more conducive to real-world applications.

The Impacts of Assimilating Fengyun-4A Atmospheric Motion Vectors on Typhoon Forecasts

Atmospheric motion vectors (AMVs), known as cloud track winds, have positive impacts on global numerical weather forecasts (NWP). In this study, AMVs that were retrieved from Fengyun-2G and Fengyun-4A were compared in their data quality and impacts on the typhoon forecasts in order to investigate the differences between the first and second generation of the geostationary meteorological satellites of China. This report conducted data evaluation and assimilation-forecasting experiments on FY-2G and FY-4A atmospheric motion vectors (AMVs), respectively. The results showed that the AMVs data of FY-4A are of better quality than those of FY-2G and assimilating the AMVs of FY-2G and FY-4A have a neutral to slightly positive impacts on typhoon forecasts, which is quite encouraging for their operational use in the future.