Realistic Edge Research Articles

Parallel Spatio-Temporal Attention Transformer for Video Frame Interpolation

Traditional video frame interpolation methods based on deep convolutional neural networks face challenges in handling large motions. Their performance is limited by the fact that convolutional operations cannot directly integrate the rich temporal and spatial information of inter-frame pixels, and these methods rely heavily on additional inputs such as optical flow to model motion. To address this issue, we develop a novel framework for video frame interpolation that uses Transformer to efficiently model the long-range similarity of inter-frame pixels. Furthermore, to effectively aggregate spatio-temporal features, we design a novel attention mechanism divided into temporal attention and spatial attention. Specifically, spatial attention is used to aggregate intra-frame information, integrating both attention and convolution paradigms through the simple mapping approach. Temporal attention is used to model the similarity of pixels on the timeline. This design achieves parallel processing of these two types of information without extra computational cost, aggregating information in the space–time dimension. In addition, we introduce a context extraction network and multi-scale prediction frame synthesis network to further optimize the performance of the Transformer. Our method and state-of-the-art methods are extensively quantitatively and qualitatively experimented on various benchmark datasets. On the Vimeo90K and UCF101 datasets, our model achieves improvements of 0.09 dB and 0.01 dB in the PSNR metrics over UPR-Net-large, respectively. On the Vimeo90K dataset, our model outperforms FLAVR by 0.07 dB, with only 40.56% of its parameters. The qualitative results show that for complex and large-motion scenes, our method generates sharper and more realistic edges and details.

On the impact of deep neural network calibration on adaptive edge offloading for image classification

Edge devices can offload deep neural network (DNN) inference to the cloud to overcome energy or processing constraints. Nevertheless, offloading adds communication delay, which increases the overall inference time. An alternative is to use adaptive offloading based on early-exit DNNs. Early-exit DNNs have branches inserted at the output of given intermediate layers. These side branches provide confidence estimates. If the confidence level of the decision produced is sufficient, the inference is made by the side branch. Otherwise, the edge offloads the inference decision to the cloud, which implements the remaining DNN layers. Thus, the offloading decision depends on reliable confidence levels provided by the side branches at the device. This article provides an extensive calibration study on different datasets and early-exit DNNs for the image classification task. Our study shows that early-exit DNNs are often miscalibrated, overestimating their prediction confidence and making unreliable offloading decisions. To evaluate the impact of calibration on accuracy and latency, we introduce two novel application-level metrics and evaluate well-known DNN models in a realistic edge computing scenario. The results demonstrated that calibrating early-exit DNNs improves the probabilities of meeting accuracy and latency requirements.

D2UNet: Dual Decoder U-Net for Seismic Image Super-Resolution Reconstruction

Super-resolution reconstruction is an essential task of seismic inversion due to the low resolution and strong noise of field data. Popular deep networks derived from U-Net lack the ability to recover detailed edge features and weak signals. In this paper, we propose a dual decoder U-Net (D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> UNet) to explore both detail and edge information of the data. The encoder inputs the low resolution image and the edge image obtained through the Canny algorithm. Edge image can provide rich shape and boundary information, which is helpful to generate more accurate and high-quality data. The dual decoder consists of a main decoder for high-resolution recovery and an edge decoder for edge contour detection. These two decoders interact with a texture warping module (TWM) with deformable convolution. TWM aims to distort realistic edge details to match the fidelity of low resolution inputs, especially the location of edges and weak signals. The loss function is a combination of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> loss and multi-scale structural similarity loss (MS-SSIM) to ensure perception quality. Results on synthetic and field seismic images show that D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> UNet not only improves the resolution of noisy seismic images, but also maintains the image fidelity.

Farmland quality assessment using deep fully convolutional neural networks

Farmland is the cornerstone of agriculture and is important for food security and social production. Farmland assessment is essential but traditional methods are usually expensive and slow. Deep learning methods have been developed and widely applied recently in image recognition, semantic understanding, and many other application domains. In this research, we used fully convolutional networks (FCN) as the deep learning model to evaluate farmland grades. Normalized difference vegetation index (NDVI) derived from Landsat images was used as the input data, and the China National Cultivated Land Grade Database within Jiangsu Province was used to train the model on cloud computing. We also applied an image segmentation method to improve the original results from the FCN and compared the results with classical machine learning (ML) methods. Our research found that the FCN can predict farmland grades with an overall F1 score (the harmonic mean of precision and recall) of 0.719 and F1 score of 0.909, 0.590, 0.740, 0.642, and 0.023 for non-farmland, level I, II, III, and IV farmland, respectively. Combining the FCN and image segmentation method can further improve prediction accuracy with results of fewer noise pixels and more realistic edges. Compared with conventional ML, at least in farmland evaluation, FCN provides better results with higher precision, recall, and F1 score. Our research indicates that by using remote sensing NDVI data, the deep learning method can provide acceptable farmland assessment without fieldwork and can be used as a novel supplement to traditional methods. The method used in this research will save a lot of time and cost compared with traditional means.

Assessment of advanced fluid neutral models for the neutral atoms in the plasma edge and application in ITER geometry

The neutral atoms in the plasma edge of nuclear fusion devices are typically modeled using either a fluid or kinetic approach. The kinetic approach is most accurate, but it has two main disadvantages. First, the usual solution of the high-dimensional kinetic equation using Monte Carlo techniques introduces statistical noise, which hampers the convergence of the coupled plasma-neutral model. Second, the computational time strongly increases for highly collisional regimes. For these reasons, deterministic fluid neutral models remain an attractive alternative, in particular for the highly collisional conditions where their accuracy is expected to be high. In recent years, efforts have been undertaken to improve the agreement between the fluid and kinetic approach by introducing consistent transport coefficients and consistent boundary conditions in the fluid models. In this work, these so-called advanced fluid neutral models are further enhanced by introducing different strategies to cope with the high heterogeneity of the ion-neutral collisionality encountered in realistic plasma-edge geometries, namely isotropic neutral flux limiters and an automated selection criterion for the optimal neutral boundary conditions. The validity of the resulting fluid neutral models is thoroughly assessed for various representative simulation cases with different geometries, divertor collisionalities, and wall materials, including, for the first time, simulations in a realistic ITER plasma edge geometry. Strong quantitative agreement between the fluid and kinetic models is achieved for cases with highest divertor collisionality.

A Comparative Evaluation of Edge Cloud Virtualization Technologies

Edge cloud computing is a crucial enabler for applications or network services with stringent communication requirements in the 5G networks and beyond era. Although edge compute resources contribute to a significant portion of end-to-end performance (e.g., delay), the involved performance trade-offs in relation to the utilized virtualization technologies and application functions are not yet thoroughly investigated. In this context, we study the performance dynamics of alternative edge cloud technologies, including different unikernel flavors, container builds, and implementations of exemplary Web services. Our experiments do not target identifying a single best option, rather than the involved performance trade-offs with respect to both service and infrastructure viewpoints as well as realistic edge cloud tasks, including on service operation, cloud resource or service elasticity, dynamic resource allocation and removal. We provide insights gained from a relevant extensive comparative evaluation based on a novel edge cloud experimentation infrastructure. Our experimental results inspired a conceptual edge cloud orchestration platform and its basic design guidelines, i.e., mixing and matching heterogeneous edge cloud virtual entities with particular operational or performance requirements.

Distributed Clustering-Based Cooperative Vehicular Edge Computing for Real-Time Offloading Requests

Mobile vehicles have been considered as potential edge servers to provide computation resources for the emerging Intelligent Transportation System (ITS) applications. However, how to fully utilize the mobile computation resources to satisfy the real-time arrived computation requests has not been explored yet. This work will address the critical challenges of limited computation resources, stringent computation delay and unknown requirement statistics of real-time tasks in realistic vehicular edge computing scenarios. Specifically, we design a distributed clustering strategy to classify vehicles into multiple cooperative edge servers according to the available computation resources, effective connection time and the distribution of tasks&#x2019; expected deadlines. Then, a &#x2018;Less than or Equal to&#x2019; Generalized Assignment Problem (i.e., LEGAP) is formulated to maximize the system service revenue, and on this basis, we propose an offline Bound-and-Bound based Optimal (BBO) algorithm to make periodical scheduling with a global view of tasks&#x2019; requirement statistics. The quick branching is conducted by following a greedy solution and the upper bound at each branch is derived by solving a multiple-choice knapsack problem. In addition, we present an online heuristic algorithm which makes real-time offloading decision with the guarantees that the resource capacities of all the computing servers are never exceeded with new tasks arriving. Through comparing with the other four online algorithms, the BBO algorithm achieves the highest service revenues by offloading tasks with shorter delays, and the online heuristic algorithm has the best performance in improving the service ratios.

Sketch to portrait generation with generative adversarial networks and edge constraint

A novel method for generating color portraits from sketch images using the edge constraint algorithm and generative adversarial networks (GANs) is proposed in this paper. In converting sketches into color portraits, the details of portrait output by GANs are often blurred and unrealistic. A new method with edge constraint is proposed in this work to address this issue. The image generated from generator network and its edge generated from the followed edge network are combined and provided to the discriminator for authenticity identification. Experiments show that the portrait output by the proposed method provides a more clear and realistic edge than a Pix2Pix model and has a better ability to generate color portraits from sketches compared with other common methods. The average structural similarity index measure (SSIM) value of the proposed method is 82.78%, while the values obtained by other methods and Pix2Pix are 42.99% and 78.60%, respectively.

On the Planning and Design Problem of Fog Computing Networks

This paper proposes an exact model for the planning and design problem of fog networks. More precisely, a mathematical model is proposed to simultaneously determine the optimal location, the capacity and the number of fog node(s) as well as the interconnection between the installed fog nodes and the cloud. The goal of the model is to minimize the delay in the network and the amount of traffic sent to the cloud data center. To address this multi-objective optimization problem, three optimization techniques are used: the weighted sum, the hierarchical and the trade-off methods. The weighted sum method aggregates all the lone objective functions into a single objective by applying a weighted vector. The hierarchical method takes a sequential approach by tightly constraining the more important objective function. The trade-off method solves a single objective function and translates all other objective functions into constraints. These methods are then compared in terms of average delay, amount of traffic sent to the cloud and amount of CPU time required to find optimal solution(s). Since we are dealing with a multi-objective optimization problem and that multiple optimal solutions can be found, the fuzzy-based mechanism and the hypervolume indicator have been used. Computational results show that as the problem size increases, the delay and the traffic also increase in a linear form; whereas, the solution time increases in non-polynomial time. The weighted sum method was able to achieve the best trade-off results for the delay and the traffic, whereas the hierarchical method was able to return minimum delay but with worse traffic going to the cloud. As the model considers realistic edge device traffic parameters, constraints, and various topology aspects, it can be helpful for the planning and deployment of fog networks and how they operate within a cloud infrastructure.

A new resolution enhancement method for sandstone thin-section images using perceptual GAN

The thin section is a sliver cut of rock sample and photographed by a microscope to help researchers investigate the mineral, structure, and geology details of a rock sample. For the purpose of photographing using the microsctope, only a small portion of the thin section can be visualized. The results, however, can hardly satisfy the requirement of a high resolution and larger scale visualization. The thin section is then trapped into a dilemma of contradiction between resolution and field of view. To solve this problem, we propose the Generative Adversarial Networks（GAN） based architectures representing the perceptual SR in this paper. Four perceptual super-resolution methods and two pixel-wise super-resolution methods are trained and tested for comparison with the dataset of sandstone thin-section. The perceptual GAN based super-resolution method is demonstrated to compensate for this limitation by enhancing the resolution of a large field of view image, which will provide a higher resolution detail recovery and larger field of view simultaneously. Based on the digital experiment analysis and comparison, thin-section perceptual loss with Residual-in-Residual Dense Block (RRDB) and Relativistic average GANs (RaGAN) shows a more realistic texture and evident edge for minerals and pores than other methods. With the new perceptual GAN based image enhancement method, the petroleum geologist can significantly improve the accuracy and reliability of all their thin section based research quality.

EBIT: Weakly-supervised image translation with edge and boundary enhancement

Abstract In this paper, targeting image translation between the thermal and visible domains, we propose a novel framework to enhance the edge and boundary features during translation. We tackle the unsupervised training task where sample image pairs from two domains are randomly chosen so the image content does not match, but we also take advantage of paired feature maps during the feature disentanglement process. This can be considered to be weakly-supervisedtraining. First, since thermal images usually have vague edges, we propose to apply a Canny operator to strengthen the edge features of the thermal images. Next, to define the correct object boundaries, we extract the boundary features from silhouette masks which are paired with the visible domain images. Then we disentangle the features of both domains into a domain-shared latent space and a domain-exclusive latent space. In the domain-shared latent space, the edge and boundary features extracted from the thermal and visible domains respectively act as domain-shared information which is used to render the edges and define the boundaries of the translated images. In the domain-exclusive latent space, domain-exclusive information such as colour is used to render the colour of objects of the translated images. In addition, we propose a pixel-wise adversarial loss rather than more traditional ones. The experimental results show that the proposed method has the ability to render realistic edge and colour features within the correct object boundaries and outperforms several state-of-the-art methods.

Energy-Aware Mobile Edge Computing and Routing for Low-Latency Visual Data Processing

New paradigms such as Mobile Edge Computing (MEC) are becoming feasible for use in, e.g., real-time decision-making during disaster incident response to handle the data deluge occurring in the network edge. However, MEC deployments today lack flexible IoT device data handling such as handling user preferences for real-time versus energy-efficient processing. Moreover, MEC can also benefit from a policy-based edge routing to handle sustained performance levels with efficient energy consumption. In this paper, we study the potential of MEC to address application issues related to energy management on constrained IoT devices with limited power sources, while also providing low-latency processing of visual data being generated at high resolutions. Using a facial recognition application that is important in disaster incident response scenarios, we propose a novel “offload decision-making” algorithm that analyzes the tradeoffs in computing policies to offload visual data processing (i.e., to an edge cloud or a core cloud) at low-to-high workloads. This algorithm also analyzes the impact on energy consumption in the decision-making under different visual data consumption requirements (i.e., users with thick clients or thin clients). To address the processing-throughput versus energy-efficiency tradeoffs, we propose a “Sustainable Policy-based Intelligence-Driven Edge Routing” algorithm that uses machine learning within Mobile Ad hoc Networks. This algorithm is energy aware and improves the geographic routing baseline performance (i.e., minimizes impact of local minima) for throughput performance sustainability, while also enabling flexible policy specification. We evaluate our proposed algorithms by conducting experiments on a realistic edge and core cloud testbed in the GENI Cloud infrastructure, and recreate disaster scenes of tornado damages within simulations. Our empirical results show how MEC can provide flexibility to users who desire energy conservation over low latency or vice versa in the visual data processing with a facial recognition application. In addition, our simulation results show that our routing approach outperforms existing solutions under diverse user preferences, node mobility, and severe node failure conditions.

Spontaneous Breakdown of Topological Protection in Two Dimensions.

Because of time-reversal symmetry, two-dimensional topological insulators support counterpropagating helical edge modes. Here we show that, unlike the infinitely sharp edge potential utilized in traditional calculations, an experimentally more realistic smooth edge potential gives rise to edge reconstruction and, consequently, spontaneous time-reversal symmetry breaking. Such edge reconstruction may lead to breaking of the expected perfect conductance quantization, to a finite Hall resistance at zero magnetic field, and to a spin current. This calculation underpins the fragility of the topological protection in realistic systems, which is of crucial importance in proposed applications.

Systematic studies of the electronic transport in realistic edge disorderd graphene nanoribbons

Systematic studies of the electronic transport in realistic edge disorderd graphene nanoribbons

On the global nonlinear instability of the rotating-disk flow over a finite domain

Direct numerical simulations based on the incompressible nonlinear Navier–Stokes equations of the flow over the surface of a rotating disk have been conducted. An impulsive disturbance was introduced and its development as it travelled radially outwards and ultimately transitioned to turbulence has been analysed. Of particular interest was whether the nonlinear stability is related to the linear stability properties. Specifically three disk-edge conditions were considered; (i) a sponge region forcing the flow back to laminar flow, (ii) a disk edge, where the disk was assumed to be infinitely thin and (iii) a physically realistic disk edge of finite thickness. This work expands on the linear simulations presented by Appelquist et al. (J. Fluid. Mech., vol. 765, 2015, pp. 612–631), where, for case (i), this configuration was shown to be globally linearly unstable when the sponge region effectively models the influence of the turbulence on the flow field. In contrast, case (ii) was mentioned there to be linearly globally stable, and here, where nonlinearity is included, it is shown that both cases (ii) and (iii) are nonlinearly globally unstable. The simulations show that the flow can be globally linearly stable if the linear wavepacket has a positive front velocity. However, in the same flow field, a nonlinear global instability can emerge, which is shown to depend on the outer turbulent region generating a linear inward-travelling mode that sustains a transition front within the domain. The results show that the front position does not approach the critical Reynolds number for the local absolute instability, $R=507$. Instead, the front approaches $R=583$ and both the temporal frequency and spatial growth rate correspond to a global mode originating at this position.

Non-straight cell edges are important to invasion and engulfment as demonstrated by cell mechanics model.

Computational models of cell–cell mechanical interactions typically simulate sorting and certain other motions well, but as demands on these models continue to grow, discrepancies between the cell shapes, contact angles and behaviours they predict and those that occur in real cells have come under increased scrutiny. To investigate whether these discrepancies are a direct result of the straight cell–cell edges generally assumed in these models, we developed a finite element model that approximates cell boundaries using polylines with an arbitrary number of segments. We then compared the predictions of otherwise identical polyline and monoline (straight-edge) models in a variety of scenarios, including annealing, single- and multi-cell engulfment, sorting, and two forms of mixing—invasion and checkerboard pattern formation. Keeping cell–cell edges straight influences cell motion, cell shape, contact angle, and boundary length, especially in cases where one cell type is pulled between or around cells of a different type, as in engulfment or invasion. These differences arise because monoline cells have restricted deformation modes. Polyline cells do not face these restrictions, and with as few as three segments per edge yielded realistic edge shapes and contact angle errors one-tenth of those produced by monoline models, making them considerably more suitable for situations where angles and shapes matter, such as validation of cellular force–inference techniques. The findings suggest that non-straight cell edges are important both in modelling and in nature.Electronic supplementary materialThe online version of this article (doi:10.1007/s10237-015-0697-6) contains supplementary material, which is available to authorized users.

Protected edge states in silicene antidots and dots in magnetic field

Silicene systems, due to the buckled structure of the lattice, manifest remarkable intrinsic spin-orbit interaction triggering a topological phase transition in the low-energy regime. Thus, we found that protected edge states are present in silicene antidots and dots, being polarized in valley-spin pairs. We have also studied the effect of the lattice termination on the properties of the single electron energy levels and electron density distribution of silicene antidots and dots situated in a perpendicular magnetic field. Our calculations confirmed that the topological edge states are propagating over the perimeter of the antidot/dot for both ideal or realistic edge termination containing roughness on the atomic length scale. The valley polarization and the slope of the energy line as a function of the magnetic field is, however, reduced when the antidot or dot has a rough edge.

Reflection and transmission of ocean wave spectra by a band of randomly distributed ice floes

AbstractA new ocean wave/sea-ice interaction model is proposed that simulates how a directional wave spectrum evolves as it travels through an arbitrary finite array of circular ice floes, where wave/ ice dynamics are entirely governed by wave-scattering effects. The model is applied to characterize the wave reflection and transmission properties of a strip of ice floes, such as an ice edge band. A method is devised to extract the reflected and transmitted directional wave spectra produced by the array. The method builds upon an integral mapping from polar to Cartesian coordinates of the scattered wave components. Sensitivity tests are conducted for a row of floes randomly perturbed from a regular arrangement. Results for random arrays are generated using ensemble averaging. A realistic ice edge band is then reconstructed from field experiment data. Simulations show good qualitative agreement with the data in terms of transmitted wave energy and directional spreading. In particular, it is observed that short waves become isotropic quickly after penetrating the ice field.

Polarized micro Raman scattering spectroscopy for curved edges of epitaxial graphene

This letter performed polarized microscopic laser Raman scattering spectroscopy on the curved edges of transferred epitaxial graphene on SiO2/Si. The intensity ratio between the parallel and perpendicular polarized D band is evolved, providing a spectroscopy-based technique to probe the atomic-scale edge structures in graphene. A detailed analysis procedure for non-ideal disordered curved edges of graphene is developed combining the atomic-scale zigzag and armchair edge structures along with some point defects. These results could provide valuable information of the realistic edges of graphene at the atomic-scale that can strongly influence the performance of graphene-based nanodevices.

Polarized microscopic laser Raman scattering spectroscopy for edge structure of epitaxial graphene and localized vibrational mode

Polarized microscopic laser Raman scattering spectroscopy accompanied with simulation model has been used to systematically analyze the details of the edge structures of the transferred epitaxial graphene. The evolution of the intensity ratio between the parallel polarized D band (VV) and the perpendicular polarized D band (VH) is determined, providing a spectroscopy-based method to probe the atomic scale edge structures in graphene. Based on the experimental Raman results, we develop a rather precise atomic scale edge model combining the armchair and zigzag edges along with some point defects. The calculated polarization dependent VH/VV ratios of the developed model are perfectly matched with the D band intensity ratios of the realistic graphene edges. The developed edge structures are also supported by the atomic force microscopy (AFM) measurements. Moreover, we calculate the vibrational eigenmodes of graphene with some realistic edge and disordered configurations. We find that the typical mode patterns for in-plane optical phonon modes (iTO) at the K point are localized near the armchair edges or the defects, which are conceptually well agreed with the high intensity D peak in the Raman spectra come from the iTO phonon near the K point activating by the armchair edges or the defects.