Dense Flow Research Articles

Video Frame Prediction by Joint Optimization of Direct Frame Synthesis and Optical-Flow Estimation

Video prediction is the problem of generating future frames by exploiting the spatiotemporal correlation from the past frame sequence. It is one of the crucial issues in computer vision and has many real-world applications, mainly focused on predicting future scenarios to avoid undesirable outcomes. However, modeling future image content and object is challenging due to the dynamic evolution and complexity of the scene, such as occlusions, camera movements, delay and illumination. Direct frame synthesis or optical-flow estimation are common approaches used by researchers. However, researchers mainly focused on video prediction using one of the approaches. Both methods have limitations, such as direct frame synthesis, usually face blurry prediction due to complex pixel distributions in the scene, and optical-flow estimation, usually produce artifacts due to large object displacements or obstructions in the clip. In this paper, we constructed a deep neural network Frame Prediction Network (FPNet-OF) with multiple-branch inputs (optical flow and original frame) to predict the future video frame by adaptively fusing the future object-motion with the future frame generator. The key idea is to jointly optimize direct RGB frame synthesis and dense optical flow estimation to generate a superior video prediction network. Using various real-world datasets, we experimentally verify that our proposed framework can produce high-level video frame compared to other state-of-the-art framework.

Analysis report on the spatial planning of streets and lanes of Shu Han Street Community

Shu Han Street Community of Jiangxi Street is located in the centre of Wuhou District, Chengdu Province and there are many Tibetan resident agencies in this community. It is also close to Southwest Minzu University, People’s Government of Wuhou District, Customs Office, and Temple of Marquis Wu. Shu Han Street Community is richly populated by multiple ethnic groups and other social groups. As a community with a dense flow of people and complex population composition, the spatial planning structure of streets and lanes in residents’ lives should be analyzed in great detail. Through measured survey and questionnaire survey, we have collected objective information such as street structure and planning and subjective information such as residents’ wishes and expectations. After integrating the information obtained, we used professional knowledge of architecture, ekistics and other subjects to propose an optimization plan for three parts—streets, corners, and micro-landscapes.

PROTOTYPE SMART TIME SCHEDULER LAMPU LALU LINTAS MENGGUNAKAN ALGORITMA HAAR CASCADE

The traffic light functions as a traffic controller at crossroads and also as smoothness at intersections to avoid congestion. The current application of conventional traffic lights still has weaknesses in determining the duration of traffic lights which are not adjusted to the number of vehicle volumes which often change every time. Especially when there is a dense flow of vehicles at intersections, especially at certain times. Based on observations of traffic density at the intersection that connects Soekarno - Hatta road and Arifin Achmad road in Pekanbaru City. There is often a high density of vehicles from the direction of Jalan Soekarno-Hatta (Morning Market) to Jalan Jenderal Sudirman during working hours so that the duration of the traffic lights cannot reduce the amount of queue density in the light traffic. The implementation of the Smart Time Scheduler is a solution for adjusting the duration of traffic lights based on the level of traffic density, by building a tool to detect and calculate the number of vehicle queues at the Traffic Light and then enter the calculation results into 3 categories, namely Normal, Medium and Dense Density. The system uses Haar Cascade with the open cv library on the Raspberry pi, the results of the system testing that has been carried out can count the number of vehicles and instruct Arduino to set the duration of the traffic lights where in the category of normal vehicle queues, the green traffic light gets a waiting time of 10 seconds in countdown, Moderate category for 15 seconds, and Solid Category, for 20 seconds, while the calculation error value uses the Error Percentage equation formula, namely comparing the results of vehicle calculations by the system with Manual Calculation Results multiplied by 100% Then the System Calculation Error is 16% and the accuracy level of the tool is 100% - 16% = 84%, the size of the error value is influenced by the quality of the light intensity, and the distance of the camera's detection of the object when performing the detection.

Subgrid Variational Optimized Optical Flow Estimation Algorithm for Image Velocimetry

The variational optical flow model is used in this work to investigate a subgrid-scale optimization approach for modeling complex fluid flows in image sequences and estimating their two-dimensional velocity fields. To solve the problem of lack of sub-grid small-scale structure information in variational optical flow estimation, we combine the motion laws of incompressible fluids. Introducing the idea of large eddy simulation, the instantaneous motion can be decomposed into large-scale motion and a small-scale turbulence in the data term. The Smagorinsky model is used to model and solve the small-scale turbulence. The improved subgrid scale Horn-Schunck (SGS-HS) optical flow algorithm provides better results in velocity field estimation of turbulent image sequences than the traditional Farneback dense optical flow algorithm. To make the SGS-HS algorithm equally competent for the open channel flow measurement task, a velocity gradient constraint is chosen for the canonical term of the model, which is used to improve the accuracy of the SGS-HS algorithm in velocimetric experiments in the case of the relatively uniform flow direction of the open channel flow field. The experimental results show that our algorithm has better performance in open channel velocimetry compared with the conventional algorithm.

Modeling and Analysis of Discrete Facial Expressions with Dense Optical Flow-derived Features

Facial expressions (FEs) are one of the most preeminent means of conveying one’s emotions and are pivotal to nonverbal communication. Potential applications in a wide range of areas in computer vision have lent a strong impetus to research in the domain of automatic facial expression recognition. This work discusses the effectiveness of two optical flow-based features for modeling the FEs associated with prototypic emotions based on the pattern of nonrigid deformable motion of facial components occurring during their portrayal. The discernible motion patterns are categorized into distinct discrete classes with the descriptive features indicating the global spatial distribution of deformation derived from the dense optical flow field associated with emotional and neutral face images. Results obtained with evaluation on images and video clips taken from Extended Cohn-Kanade, Japanese Female Facial Expressions, and Dynamic Karolinska Directed Emotional Faces datasets with multi-class support vector machine and [Formula: see text]-nearest neighbor classifiers are competent with the state-of-the-art techniques and concordant with empirical psychological studies in emotion science.

The Dynamics of University-Industry Interactions in Peripheral Contexts: Evidence from Brazil

This research aims at addressing the factors that constrain the flow of knowledge between universities and industry when these players are embedded in peripheral contexts. A multiple-case study was carried out in order to describe and understand the limitations of universities as agents of innovation in peripheral ecosystems. Twenty-two semi-structured interviews were conducted with the coordinators of five Technological Innovation Centers (entities equivalent to TTO) of all Federal Institutes (five) located in the State of Minas Gerais, Brazil. The findings show that there are constraining elements associated with the socioeconomic environment (the lack of economic dynamism and low absorptive capacity at firms), with universities (a lack of infrastructure, resources, and available researchers) and intermediary agents (the lack of staff and institutional legitimacy). The observed conditions lead to challenges in fostering dense knowledge flows, thus perpetuating regional economic asymmetries and hindering the institutional evolution of academic institutions toward the notion of entrepreneurial universities. Our research contributes to literature by addressing in detail the limitations of universities in spurring dense innovation networks in laggard ecosystems. Instead, more complex co-evolutionary processes seem to be at play – and “silver bullet” policies are likely to offer disappointing results.

Numerical simulation of biomass gasification in fluidized bed gasifiers

Numerical simulation of biomass gasification in fluidized bed reactors faces several challenges including proper modeling of the physical and chemical processes involved in the gasification and numerical stiffness of the two-phase dense particle flow problem. In this paper, the multi-phase particle-in-cell (MP-PIC) model coupled with a recently developed distribution kernel method (DKM) and a new one-step pyrolysis model are employed to investigate biomass gasification in two lab-scale fluidized bed gasifiers (FBGs). The results are evaluated by comparing with the results from the Particle Centroid Method (PCM) and experimental measurements. The performance of DKM is shown to improve the robustness of the model and the new pyrolysis model is shown to improve the sensitivity of the yields of gasification products to operating temperature. The simulation results using the new pyrolysis model agree well with the experimental data under different gasification conditions. The model is shown to be able to capture the trend of gas products with respect to variations of steam/biomass ratio (SR) and operating temperature (Tr). The mechanisms of the formation of gas products are analyzed based on the numerical results. By increasing the SR and Tr, the production of H2 and CO2 is shown to increase while the production of CO and CH4 to decrease. It is shown that varying the steam/biomass ratio in the range of 0.8∼2.0 has a minor effect on the pyrolysis process and heterogeneous reactions, while homogeneous reactions are significantly affected, leading to changes in the final composition of the gas products. Varying the gasifier temperature Tr has on the other hand a crucial effect on the pyrolysis process and as such a significant impact on the gasification products and carbon conversion.

Development of a non-local partial Peridynamic explicit mesh-free incompressible method and its validation for simulating dry dense granular flows

Development of a non-local partial Peridynamic explicit mesh-free incompressible method and its validation for simulating dry dense granular flows

Violence 4D: Violence detection in surveillance using 4D convolutional neural networks

AbstractAs violence has increased around the world, surveillance cameras are everywhere, and they are only going to get more ubiquitous. Due to the massive volume of video footage, automatic activity detection systems must be used to create an online warning in the event of aberrant activity. A deep learning architecture is presented in this study using four‐dimensional video‐level convolution neural networks. The proposed architecture includes residual blocks that are used with three‐Dimensional Convolution Neural Networks 3D (CNNs) to learn long‐term and short‐term spatiotemporal representation from the video as well as record inter‐clip interaction. ResNet50 is used as the backbone for three‐dimensional convolution networks and dense optical flow for the region of interest. The proposed architecture is applied on four benchmarks for violence and non‐violence videos, which are commonly used for violent detection. It obtained test accuracies of 94.67% on RWF2000, 97.29% on Crowd violence, 100% on Movie fight and 100% on the Hockey Fight dataset. These results outperform the previous methods used on RWF2000 datasets.

A Hybrid Rule-Based and Data-Driven Approach to Illegal Transshipment Identification with Interpretable Behavior Features

Illegal transshipment of maritime ships is usually closely related to illegal activities such as smuggling, human trafficking, piracy plunder, and illegal fishing. Intelligent identification of illegal transshipment has become an important technical means to ensure the safety of maritime transport. However, due to different geographical environments, legal policies and regulatory requirements in each sea area, there are differences in the movement characteristics and geographical distribution of illegal transshipment behavior in different time and space. Moreover, in areas with dense traffic flow, normal navigation behavior can easily be identified as illegal transshipment, resulting in a high rate of misidentification. This paper proposes a hybrid rule-based and data-driven approach to solve the problem of missing identification in fixed threshold methods and introduces a traffic density feature to reduce the misidentification rate in dense traffic areas. The method is both interpretable and adaptable through unsupervised clustering to get suitable threshold distribution combination for regulatory sea areas. The evaluation results in two different sea areas show that the proposed method is applicable. Compared with other widely used identification methods, this method identifies more illegal transshipment events, which are highly suspicious, and gives warning much earlier. The proposed method can even filter out misidentification events from compared methods' results, which account for more than half of the total number.

Scale Invariant low frame rate tracking

The cameras used for road surveillance present low image quality and significant variability of objects’ scale, making real-time vehicle detection and tracking challenging. This paper proposes Scale Invariant Low Frame Rate Tracking, called SILFT, a new methodology to track vehicles in most surveillance systems whose frame rate and resolution are low. This architecture uses the fusion of a dense optical flow with object detection in a 6-step pipeline. The detections are masks for the optical flow that later will be used for finding the object correspondence between frames, making it effective in low frame rate sequences with large object scale variability. SILFT achieved average precision for detection of 65.97%, surpassing the YOLO and FASTER R-CNN models in the proposed database. For the tracking task, SILFT achieved a PR-MOTA of 0.45, whereas the YOLO model with intersection over union tracking achieved 0.13 in the proposed dataset.

Experimental characterization of extreme temperature granular flows for solar thermal energy transport and storage

High-temperature, dense granular flows along an inclined plane were considered for solar thermal energy transport and storage with sintered bauxite particles. A series of experiments was performed for particle inlet temperatures of ∼ 200, 400, 600, and 800 °C to understand the mechanisms of granular flows at extreme temperatures. Mass flow rates were measured using a load cell and free-surface velocities were measured and computed using particle image velocimetry. Surface temperatures were measured using infrared cameras. A significant decrease in steady-state particle mass flow rate was observed with increasing temperature due to changing flow properties. A decrease in bulk particle free-surface velocities was observed at higher temperatures. Free-surface velocity measurement error between experiments were within 20% of the average. The particle surface temperatures decreased from inlet to outlet with larger gradients at higher temperatures observed due to increasing convection and radiative heat losses. A decrease in temperature was observed along the side walls due to a decrease in particle velocities. • Refractory granular media enable high temperature solar thermal energy storage. • Granular flow experiments on inclined plane explored at elevated temperatures. • Significant changes in flow behavior observed at elevated temperatures. • Larger heat loss to surroundings observed at side walls due to decrease in velocity.

Study on energy transition of dense particulate flow in a horizontal agitator

Hammer mill cuttings cleaner is an efficient low-temperature drying cuttings equipment. In order to investigate the main components of energy dissipation and the flow characteristics of granular flow in grinding systems. The energy dissipation model of dense particle flow is established by numerical simulation and validated by macroscopic experiment. The results show that the particle flow mainly moves in a circle along the wall of cylindrical vessel. The main parts of energy dissipation are sliding friction consumption of particle, the inelastic collision between particles and sliding friction between particles and container; the input energy is a linear function of particle filling rate. The flat blade can provide higher efficiency at the same operational conditions. The study lays a theoretical foundation for the industrial application of Thermal-mechanical cuttings cleaner of drill cuttings and provides a reference for the optimization of blade configuration.

A phenomenological law for complex granular materials from Mohr-Coulomb theory

Manipulating powders still entails some clumsy and risky operations even now in the middle of the fourth industrial revolution. This is because there is a lack of well-understood theory about granular matter due to its ravelled complexity. However, granular matter is the second most handled material by man after water and is thus ubiquitous in daily life and industry only after water. Since the eighteenth century, mechanical and chemical engineers have been striving to manage the many difficulties of grain handling, most of which are related to flow problems. Many continuum models for dense granular flow have been proposed. Herein, we investigated Mohr–Coulomb failure analysis as it has been the cornerstone of stress distribution studies in industrial applications for decades. This research gathers over 130 granular materials from several industrial sectors, as varied as cement and flour, including raw materials, food, pharmaceuticals, and cosmetics. A phenomenological law derived from the yield locus and governed exclusively by one dimensionless number from adhesive interactions has been found. Surprisingly, and in contrast to the common perception, flow in the quasi-static regime is actually independent of the friction, the packing fraction and any other grains/bulk intrinsic properties. The simplicity and accuracy of the model are remarkable in light of the complex constitutive properties of granular matter.

Effect of N2 and CO2 bubbles on gas volume fraction and flow pattern in a quasi-2D bubble column by shadow imaging

This study reports the effect of N2 and CO2 bubbles on dilute to dense gas-liquid two-phase bubbly flow. A shadowgraph imaging technique captured bubble images at a high spatiotemporal resolution. The recordings of bubble images allow us to compute gas fraction distribution. It requires challenging segmentation and gas-liquid interface detection approaches in image processing. Hence a novel gas contour characterization technique has been introduced in this study that analyses light intensity per pixel for quantifying the effect of local gas volume fraction. The dominant gas structure and repetitive gas pattern have also been determined here using Fourier transform-based power spectral density and 2D cross-correlation functions, respectively. Gas-liquid flow regimes of dissolved CO2 bubbles are found quite different than that of N2 bubbles. The plausible reasons are that gas fraction distribution at the sparger region may inhibit bubble coalescence and the positive surface charge of CO2 bubbles acts as a barrier to the interface deformation.

Analyzing the Structure of Residence–Leisure Network in Shenyang City

Leisure is an important part of the daily activities of urban residents. A relatively dense flow of people will be generated between residential areas and supermarkets, as well as between residential areas and highly popular park facilities. These flows of people can reflect the characteristics of residents’ leisure activities and the spatial characteristics of urban residence–leisure functions, as opposed to static leisure facilities and places; it is a new perspective for the study of urban spatial structure. Network studies on the relationship between residential and leisure functions within cities are rarely seen. In this study, from the flow space perspective, based on the questionnaire data, points of interest data, and mobile phone signaling data, the actual leisure travel flows of residents with different travel purposes can be identified, including residence–shopping leisure flows and residence–park leisure flows, and the corresponding urban networks can be constructed from them. With the help of complex network analysis, this paper discusses different types of residence–leisure network structures and their influencing factors in terms of network characteristics, node strength, and QAP analysis. It deepens the understanding of the urban spatial structure and provides the theoretical basis and technical support for urban structure analysis, urban layout optimization, and urban planning and management. The results show that: ① Both residence–shopping leisure and residence–park leisure networks have the small-world characteristics and scale-free properties of complex networks. ② The characteristics of the nodes of the residence–leisure network for different leisure travel purposes indicate that residents go more to Taiyuan Street and the New North Station business circle for shopping activities, and the parks that attract residents to go out for walks are concentrated in the central part of the city. ③ Different types of network structures have a strong correlation with the number of residential functions and leisure facilities but have a weak correlation with the difference in the number of inhabitants and leisure travel distance. This study enriches the research cases of the urban residence–leisure network structure to a certain extent. Shenyang City has the same background of rapid expansion as other large cities in China, and this study has an important role in planning and inspiration for solving urban diseases and achieving the orderly and rational development of large cities.

Material removal mechanism and MR fluid for magnetorheological finishing of an RSA-6061 aluminum alloy mirror.

We sought to improve the surface quality and the removal rate of aluminum alloy mirrors in magnetorheological finishing (MRF). Based on dense granular flow theory, we investigated the removal of magnetorheological (MR) fluid in the polishing area and adopted a new removal model to express the removal rate of MRF. Then, according to the composition and physical properties of RAS-6061 aluminum alloy, we analyzed the reaction state of it in an acidic and alkaline environment. At the same time, we gave a reasonable range of the content of MR fluid by single-factor experiment and explained the influence of MR fluid on surface quality and removal rate. Lastly, we carried out the MRF experiment on an aluminum alloy mirror and got a high surface quality. The surface roughness reached R a=2.715n m, and the tool marks formed by turning were removed, which verified the reliability of the removal model.

Dynamic densification of wet granular materials in continuous dense flows under gravity

AbstractThe continuous granular flow under gravity extensively exists in chemical processes, and the dynamic behaviors of granular flow under different gas–liquid environments are investigated by synchronous slide‐type high‐speed photography. Results show that a hexagonal arrangement is observed after three‐stage densification in wet granular flow, while these structures are almost absent in the dry system. Further analysis shows that under the induction of liquid, the special cavity collapse in stage І is the pre‐condition of granular densification, and the subsequent densification in stage ІІ for the first time gives the experimental proof for the hypothesis that energy barriers exist and depend on the friction characteristics, and the cage effect in stage ІІІ provides a guarantee for the formation of a hexagonal contact network. Besides, three‐stage densification in the three‐phase system mainly occurs in the region with low gas and liquid velocities (G* < 0.716 and L* < 0.023 in this work).

Learning to See Through Obstructions With Layered Decomposition

We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in the flow estimation and brittle assumptions, such as brightness consistency. We show that the proposed approach learned from synthetically generated data performs well to real images. Experimental results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.

Micro-expression spotting based on optical flow features

• The nose tip location-based image alignment method improves the performance of Micro-expression spotting. • The optical flow method can accurately capture micro-expression movements. • When a micro-expression occurs, a peak appears on the time-domain change curve. • Get complete micro-expression intervals after fusing local movements. Expression is the changes of facial organs due to facial muscle movement and an important way of human emotional interaction. Neurophysiological studies show that micro-expressions (MEs) are not controlled by subjective consciousness and reflect people's real emotions. That is why MEs have significant value in public security applications. This paper proposes an automatic ME spotting method of high accuracy and interpretability. Firstly, we design the nose tip location-based image alignment method to remove global displacement caused by head shaking. Secondly, according to the action unit definition in the face coding system (FACS), we select fourteen regions of interest (ROI) to capture subtle facial movements. The dense optical flow is introduced to estimate local movements and time-domain variations of the ROIs. Thirdly, we design a peak detection method on the time-domain variation curves to locate the movement intervals precisely. Lastly, we propose an overlapping index to measure the consistency of changes in different organs. Evaluation on the CAS(ME) 2 and SAMM Long Video database shows that our ME spotting method may achieve better accuracy with a relatively lower computation cost and can be applied to similar facial image processing applications.