Grid Cloud Research Articles

ARPEGE Cloud Cover Forecast Postprocessing with Convolutional Neural Network

AbstractCloud cover provides crucial information for many applications such as planning land observation missions from space. It remains, however, a challenging variable to forecast, and numerical weather prediction (NWP) models suffer from significant biases, hence, justifying the use of statistical postprocessing techniques. In this study, ARPEGE (Météo-France global NWP) cloud cover is postprocessed using a convolutional neural network (CNN). CNN is the most popular machine learning tool to deal with images. In our case, CNN allows the integration of spatial information contained in NWP outputs. We use a gridded cloud cover product derived from satellite observations over Europe as ground truth, and predictors are spatial fields of various variables produced by ARPEGE at the corresponding lead time. We show that a simple U-Net architecture (a particular type of CNN) produces significant improvements over Europe. Moreover, the U-Net outclasses more traditional machine learning methods used operationally such as a random forest and a logistic quantile regression. When using a large number of predictors, a first step toward interpretation is to produce a ranking of predictors by importance. Traditional methods of ranking (permutation importance, sequential selection, etc.) need important computational resources. We introduced a weighting predictor layer prior to the traditional U-Net architecture in order to produce such a ranking. The small number of additional weights to train (the same as the number of predictors) does not impact the computational time, representing a huge advantage compared to traditional methods.

Making slope map of the third national land survey based on high-precision DEM of 2 m grid cloud data

Making slope map of the third national land survey based on high-precision DEM of 2 m grid cloud data

Fuzzy Weighted Clustering Method for Numerical Attributes of Communication Big Data Based on Cloud Computing

It is necessary to optimize clustering processing of communication big data numerical attribute feature information in order to improve the ability of numerical attribute mining of communication big data, and thus a big data clustering algorithm based on cloud computing was proposed. The cloud extended distributed feature fitting method was used to process the numerical attribute linear programming of communication big data, and the mutual information feature quantity of communication big data numerical attribute was extracted. Combined with fuzzy C-means clustering and linear regression analysis, the statistical analysis of big data numerical attribute feature information was carried out, and the associated attribute sample set of communication big data numerical attribute cloud grid distribution was constructed. Cloud computing and adaptive quantitative recurrent classifiers were used for data classification, and block template matching and multi-sensor information fusion were combined to search the clustering center automatically to improve the convergence of clustering. The simulation results show that, after the application of this method, the information fusion performance of the clustering process was better, the automatic searching ability of the data clustering center was stronger, the frequency domain equalization control effect was good, the bit error rate was low, the energy consumption was small, and the ability of fuzzy weighted clustering retrieval of numerical attributes of communication big data was effectively improved.

Resource scheduling of information platform for general grid computing framework

The corresponding concepts and calculation methods are very in line with the requirements of information platform resource scheduling. Based on this, this paper discusses the related concepts of information grid in detail, which includes the key elements of grid computing. At the same time, it analyses and studies the key technology of information grid, web services, and puts forward the framework of information grid for resource scheduling of information platform. In order to better realise the storage and management of information resources in the information platform, this paper innovatively proposes a meta database data storage and management algorithm for the integration of grid computing and cloud computing in the information grid environment, and designs the corresponding data management and storage framework. Furthermore, we compare it with the cloud platform resource scheduling platform in order to reflect the advantages of our proposed method.

High Performance Computing in Resource Poor Settings: An Approach based on Volunteer Computing

High Performance Computing (HPC) systems aim to solve complex computing problems (in a short amount of time) that are either too large for standard computers or would take too long. They are used to solve computational problems in many fields such as medical science (for drug discovery, breast cancer detection in images, etc.), climate science, physics, mathematical science, etc. Existing solutions such as HPC Supercomputer, HPC Cluster, HPC Cloud or HPC Grid are not adapted for resource poor settings (mainly for developing countries) because their fees are generally beyond the funding (particularly for academics) and the administrative complexity to access to HPC Grid creates a higher barrier. This paper presents an approach allowing to build a Volunteer Computing system for HPC in resource poor settings. This solution does not require any additional investment in hardware, but relies instead on voluntary machines already owned by the private users. The experiment has been made on the mathematical problem of solving the matrices multiplication using Volunteer Computing system. Given the success of this experiment, the enrollment of other volunteers has already started. The goal being to create a powerful Volunteer Computing system with the maximum number of computers.

A proposed computation, which benefits from the cooperation of dew, edge, and cloud computations

AbstractWe propose a computation that compensates the weakness of the processing power and storage capacity of the dew and edge servers. In the proposed computation, the nodes near to each other, which can be the dew, edge, or cloud servers, form a computational grid. The dew servers can be mobile devices, laptops, or computers. Their processing power and storage capacity can be used when they are idle. Our goal is to achieve load balancing of different nodes (servers). A server of the grid with proper processing power, storage capacity, remained energy, and low task load is selected as the grid head (each server of grid shares its specification with the other servers of grid periodically). The grid head (server) is responsible for grid communications. The grid head shares the specification of the grid, with the neighbor nodes (we suppose that the end device is aware of the specification of the nearest dew, edge, and cloud grid heads to itself). The end device decides about sending its created tasks to the grid head of a dew or edge or cloud grid, with attention to the distance between itself and the grid head, and the free processing power and storage capacity of the grid. The scheduling algorithm of a grid is the combination of first‐come–first‐served and priority scheduling algorithms. The simulation results show that the proposed computation has less delay and energy consumption and more efficiency than the cloud and edge computations.

Novel Distributed Dynamic Backbone-based Flooding in Unstructured Networks

Resource discovery on different unstructured and dynamic networks such as grid, peer-to-peer, and cloud networks is an inevitable challenging issue. The primary method for resource discovery on the unstructured networks is flooding a query on the network. All existing flooding algorithms for unstructured networks generate almost high additional duplicated queries. This high duplication of the unstructured networks causes a lot of network traffic. This paper, therefore, proposes a novel flexible Distributed Dynamic backbone-based Flooding (DDBF) algorithm for distributed unstructured networks. This paper explores Grid middleware, Peer-to-Peer (P2P) paradigm, and cloud networks resource discovery requirements and it proposes flooding algorithm based on the P2P networks using simulation. To evaluate and prove DDBF algorithm we, first, evaluated it on four fixed network topologies along with two different query flooder distributions, then we evaluated it with one dynamic network topology. The performance of the proposed DDBF algorithm was assessed with five different metrics. The result showed a dramatic decrease in the number of engaged flooder nodes, the number of duplicated queries and consequently, network delay compared with the state-of-the-art algorithms.

Implementation of full-color holographic system using non-uniformly sampled 2D images and compressed point cloud gridding.

A multiple-camera holographic system using non-uniformly sampled 2D images and compressed point cloud gridding (C-PCG) is suggested. High-quality, digital single-lens reflex cameras are used to acquire the depth and color information from real scenes; these are then virtually reconstructed by the uniform point cloud using a non-uniform sampling method. The C-PCG method is proposed to generate efficient depth grids by classifying groups of object points with the same depth values in the red, green, and blue channels. Holograms are obtained by applying fast Fourier transform diffraction calculations to the grids. Compared to wave-front recording plane methods, the quality of the reconstructed images is substantially better, and the computational complexity is dramatically reduced. The feasibility of our method is confirmed both numerically and optically.

Evaluation of WRF-DART (ARW v3.9.1.1 and DART Manhattan release) multiphase cloud water path assimilation for short-term solar irradiance forecasting in a tropical environment

Abstract. Numerical weather prediction models tend to underestimate cloud presence and therefore often overestimate global horizontal irradiance (GHI). The assimilation of cloud water path (CWP) retrievals from geostationary satellites using an ensemble Kalman filter (EnKF) led to improved short-term GHI forecasts of the Weather Research and Forecasting (WRF) model in midlatitudes in case studies. An evaluation of the method under tropical conditions and a quantification of this improvement for study periods of more than a few days are still missing. This paper focuses on the assimilation of CWP retrievals in three phases (ice, supercooled, and liquid) in a 6-hourly cycling procedure and on the impact of this method on short-term forecasts of GHI for Réunion Island, a tropical island in the southwest Indian Ocean. The multilayer gridded cloud properties of NASA Langley's Satellite ClOud and Radiation Property retrieval System (SatCORPS) are assimilated using the EnKF of the Data Assimilation Research Testbed (DART) Manhattan release (revision 12002) and the advanced research WRF (ARW) v3.9.1.1. The ability of the method to improve cloud analyses and GHI forecasts is demonstrated, and a comparison using independent radiosoundings shows a reduction of specific humidity bias in the WRF analyses, especially in the low and middle troposphere. Ground-based GHI observations at 12 sites on Réunion Island are used to quantify the impact of CWP DA. Over a total of 44 d during austral summertime, when averaged over all sites, CWP data assimilation has a positive impact on GHI forecasts for all lead times between 5 and 14 h. Root mean square error and mean absolute error are reduced by 4 % and 3 %, respectively.

Parameter Estimation of Smart Grid using PMU with Kalman Filter and Bayesian Prediction

Load flow is the main issue which occurs in power grid systems. To improve the performance, reduce the cost and enhance the reliability in power systems, smart grids have been proposed. In electricity distribution system, smart devices like smart meters are used for effective performance. The real concern in these devices is to protect the data from unauthorized parties and noise occurring in data. Smart device reader acts as the bridge which connects the smart grid devices with smart grid clouds. In many of the instances of circuit-based analysis, the network parts are restricted to the regarded value of impedances with voltage and current resource. But the issue of load flow is usually diverse in the sense that rather of impedances, the known amounts are active and reactive powers in most of the network buses, since the performance of most of the load in a great deal of instances are as continuous power loads, presuming that voltages used on them stay within suitable ranges. There are various methods which are used to solve these problems. Kalman filters are proposed to achieve the optimal performance on the smart grid devices. This filter identifies the device failures, unusual disturbance, and malicious data attacks. Kalman Filter is a dynamic state estimation method which is mainly used in this paper for noise variation estimation. The use of dynamic state estimation methods such as the Kalman filter provides an optimal solution to the process of real-time data prediction and reduces the problem based on non-linearity. The analysis of real-time data depends on Phasor Measuring Units (PMU) which plays a significant role in power transmission and distribution processes due to their ability to monitor the power flow within a network. The process of PMU-based monitoring improves the quality of the smart grid. Simultaneously, the implementation of PMU increases the dynamics of noise variance which further inflates the uncertainty in noise-based distribution. This paper presents a method to reduce the amount of uncertainty in noise by using a linear quadratic estimation method (LQE), usually known as Kalman filter along with Taylor expansion series but this process is time-consuming and is vulnerable to a large number of errors at the time of testing. The main reason behind this approach is the high complexity of the system which makes it very hard to derive the process. The proposed studies adopts a technique to work on covariance earlier based estimation using Bayesian method together with the estimation of dynamic polynomial prior by using Particle Swarm Optimization (PSO). The experimental evaluation compares the outcomes received from the primary Kalman filter, PSO optimized Kalman filter out and Kalman filter Covariance Bayesian method. Finally, the effects received from the analysis highlights the truth that the PSO optimized Kalman clear out to be more effective than the Kalman filter out with Covariance Bayesian approach

Multiple-camera holographic system featuring efficient depth grids for representation of real 3D objects.

Recently, computer-generated holograms (CGHs) of real three-dimensional (3D) objects have become widely used to support holographic displays. Here, a multiple-camera holographic system featuring an efficient depth grid is developed to provide the correct depth cue. Multidepth cameras are used to acquire depth and color information from real scenes, and then to virtually reconstruct point cloud models. Arranging the depth cameras in an inward-facing configuration allowed simultaneous capture of objects from different directions, facilitating rendering of the entire surface. The multiple relocated point cloud gridding method is proposed to generate efficient depth grids by classifying groups of object points with the same depth values in the red, green, and blue channels. CGHs are obtained by applying a fast Fourier transform diffraction calculation to the grids. Full-color reconstructed images were obtained flexibly and efficiently. The utility of our method was confirmed both numerically and optically.

MCTE: Minimizes Task Completion Time and Execution Cost to Optimize Scheduling Performance for Smart Grid Cloud

Cloud computing is popular in nowadays for its convenient and cheap. Grid provides services of available everywhere, and shares everyone. Therefore, smart grid cloud is a good way to manage data for sharing with all power supply stations. Grid cloud task scheduling is one of the key technologies that affect resource allocation efficiency in cloud computing environment. The advantages and disadvantages of scheduling algorithms will directly affect the scheduling performance of both cloud computing and the stability of the entire system platform. Cloud task scheduling problem has been proved to be a NP-hard problem, The traditional task scheduling algorithm can no longer meet the actual needs of cloud task scheduling, but the Heuristic algorithm is an effective method to solve this problem. This paper studies and analyzes the application of heuristic algorithms in cloud task scheduling problems, and proposes a cloud task scheduling strategy to minimize the task completion time and execution cost (MCTE) for the smart grid cloud. Then, carry out mathematical modeling on the grid cloud task scheduling problem. The experimental results show MCTE is well for the smart grid cloud.

Impacts of distribution patterns of cloud optical depth on the calculation of radiative forcing

The gridding process applied to satellite-retrieved cloud properties results in the loss of certain information. In this study, we analyzed the error associated with using gridded cloud optical depth (τ) in calculating radiative forcing from the perspective of the distribution pattern of τ. Utilizing the simulated results from SBDART (Santa Babara DISORT Atmospheric Radiative Transfer), we calculated this error in ideal probability distribution functions (PDFs) of τ while keeping the average τ constant, and then used the τ retrieved from MODIS (Moderate Resolution Imaging Spectroradiometer) pixel-level observations to simulate real case studies. The results from both the ideal experiments and real case studies indicate that there is a large dependence of the error caused by gridding process on the PDF of τ. The greatest relative error occurs in the cases when τ fits a two-point or uniform distribution, reaching 10–20%, while this error is below 5% when τ follows a binomial distribution. From the analysis of MODIS pixel-level data from June 2016, we found that the PDFs of τ within one grid point (1° × 1°) could not be simply described by a normal distribution. Although using the logarithmic mean of τ controls the error effectively, the error can still be up to 4%. Our study suggests that using gridded data (especially the arithmetic mean) to calculate radiative forcing may result in uncertainty to a certain extent, which depends strongly on the distribution pattern of cloud properties within the grid point. The PDF of cloud properties should be comprehensively considered in the gridding process in the future.

DEVELOPMENT OF INFORMATION AND COMMUNICATION INFRASTRUCTURE OF DISTRIBUTED COMPUTING SYSTEMS BASED ON THE CONCEPT "THE NETWORK IS THE COMPUTER"

The concept "network is a computer" that has been further developed in the form of a paradigm of cloud applications that have the properties of "multi-lease"and "live" database migration is considered. Of particular interest in this regard is the development of a middleware for large reconfigurable clustered server systems as part of the support of the "big data" concept. Therefore, an approach is being developed to design a reconfigurable and parametrically tuned system and functional architecture of distributed computing systems. In some cases, the implementation of this approach can provide increased efficiency and cost reduction of large software and hardware systems. The article categorizes distributed computing systems when taking into account the specifics of cloud, grid, cluster and other types of communal, parallel and distributed computations. It is recommended to use as a basis hybrid architectures that combine the positive properties of cloud, grid and cluster distributed computing systems. The concept of organization of distributed network computing as services, implemented at the client's request, is developed. A network model that describes the work of a hybrid cloud grid system with third-party auditors and cloud service providers, is proposed. A distinctive feature of the proposed model is that when implementing as a concept NCaaSoD (network architecture as a service on the user's request) before the execution of a request, such as Upload (uploading data to the cloud), meta-information about available resources in order to further create a virtual cluster from the available nodes of the network of the cloud service provider, and before requesting the download type request (request information from the cloud), information is requested about the nodes that store the results. Meta-information can be obtained not only from a third-party auditor, but also from a remote monitoring service. To further develop the proposed concepts to real technology, it is advisable to develop a methodology for transforming conceptual models of the system and functional architecture into specifications suitable for the formation of virtual topologies of a network computer such as NCaaSoD in a hybrid cloud environment.

Quality enhancement and GPU acceleration for a full-color holographic system using a relocated point cloud gridding method.

The calculation of realistic full-color holographic displays is hindered by the high computational cost. Previously, we suggested a point cloud gridding (PCG) method to calculate monochrome holograms of real objects. In this research, a relocated point cloud gridding (R-PCG) method is proposed to enhance the reconstruction quality and accelerate the calculation speed in GPU for a full-color holographic system. We use a depth camera to acquire depth and color information from the real scene then reconstruct the point cloud model virtually. The R-PCG method allows us to classify groups of object points with the same depth values into grids in the red, green, and blue (RGB) channels. Computer-generated holograms (CGHs) are obtained by applying a fast Fourier transform (FFT) diffraction calculation to the grids. The feasibility of the R-PCG method is confirmed by numerical and optical reconstruction.

Big Data Research: Database and Computing

Big data research has become popular and exciting studies in almost all scientific fields such as biology, chemistry, epidemiology, medicine and drug discovery. The various systems and platforms produce large amounts of data every day. It will be very helpful for the researchers and workers to deal with big data if the practical database and useful software are introduced in time. The Journal of Big Data Research (JBR) supplies an efficient and open access publishing platform for big data research. The first issue of JBR aims to foster the dissemination of high-quality big data studies in the biological, medical and chemical database as well as the new algorithm and software for big data processing. The database and computing framework are selected to introduce the development of big data in the biological, medicine and drug discovery. The mature and functional database can be serviced in big data research of scientific fields. It promotes the scientists to extract the useful and essential dataset from the massive data. The grid computing and cloud computing supplies a new paradigm that offers an effective framework of computing and services. The research papers are welcomed from the scopes of the practical database, new algorithm and software for big data studies. All these kinds of papers not only provide the effective application methods and platforms, but also give a good promising future for big data research.

Fast calculation method of computer-generated hologram using a depth camera with point cloud gridding

We propose a fast calculation method for a computer-generated hologram (CGH) of real objects that uses a point cloud gridding method. The depth information of the scene is acquired using a depth camera and the point cloud model is reconstructed virtually. Because each point of the point cloud is distributed precisely to the exact coordinates of each layer, each point of the point cloud can be classified into grids according to its depth. A diffraction calculation is performed on the grids using a fast Fourier transform (FFT) to obtain a CGH. The computational complexity is reduced dramatically in comparison with conventional methods. The feasibility of the proposed method was confirmed by numerical and optical experiments.

A Secure and Efficient Framework to Read Isolated Smart Grid Devices

With increasing deployments of smart grid systems, a large quantity of energy usage and grid status data have been collected by smart grid devices like smart meters. To secure these critical and sensitive data, it is crucial to prevent unauthorized readings from these devices. Many authentication protocols have been proposed to control access to smart grid devices that are a part of the smart grid data communication network; however, authentication protocols to control readings from the isolated smart grid devices are mostly ignored. In this paper, we propose a secure and efficient framework to enable secure data readings from the isolated smart grid devices based on a two-phase authentication protocol. The framework not only makes use of the smart reader as a bridge to connect the isolated smart grid device and the smart grid cloud, but also considers the physical constraints of all the devices in the system. Security analysis shows that our framework is efficient and secure under most typical attacks, meanwhile it satisfies the hardware constraints of smart grid devices. Comprehensive performance evaluation also validates the efficiency of the proposed framework.

Hybrid Trust Chain Security Model with Cloud Computing Based on Smart Grid

The paper proposes a hybrid trust chain model with cloud computing based on smart grid, for the purpose of enhancing the ability of smart grid cloud computing system to deal with most malicious attacks and reduce the security risk. The model merges the trust transfer model and the separation model, based on a set of trust chain establishment scheme defined by Trusted Computing Group (TCG), and improves the traditional chain and star trust model, in order to boot and establish a trusted secure computing environment. Proved by examples, the hybrid trust chain security model puts the security of cloud computing on the trusted computing platform, which has significant effects on improving the credibility of the computer system, increasing the security of the cloud foundation and ensuring the integrity of the information.

Agent-based Managing for Grid Cloud System — Design and Prototypal Implementation

Here, we present the design and architecture of an Agent-based Manager for Grid Cloud Systems (AMGCS) using software agents to ensure independency and scalability when the number of resources and jobs increase. AMGCS handles IaaS resources (Infrastructure-as-a-Service — compute, storage and physical resources), and schedules compute-intensive jobs for execution over available resources based on QoS criteria, with optimized task-execution and high resource-utilization, through the capabilities of grid clouds. This prototypal design and implementation has been tested and shown a proven ability to increase the reliability and performance of cloud application by distributing its tasks to more than one cloud system, hence increase the reliability of user jobs and complex tasks submitted from regular machines.