Semijoin Strategy Research Articles

A Hybrid Neural Network Electromagnetic Inversion Scheme (HNNEMIS) for Super-Resolution 3-D Microwave Human Brain Imaging

Super-resolution three-dimensional (3-D) electromagnetic (EM) inversion for microwave human brain imaging is a typical high contrast EM inverse problem and requires huge computational costs. This work proposes a hybrid neural network electromagnetic inversion scheme (HNNEMIS) which contains shallow and deep neural networks to alleviate the required huge computational costs and solve this high contrast inverse problem. In the proposed scheme, semi-join back propagation neural network (SJ-BPNN) is employed to nonlinearly map the measured scattered electric field to two output channels, namely the permittivity and conductivity of scatterers, respectively. Such a semi-join strategy decreases the computational burden in training and testing processes. Then, a deep learning technique, termed U-Net, is employed to further enhance the imaging quality of the output from SJ-BPNN. To decrease the training cost and make neural networks fast convergent for human brain inversion, a novel training dataset construction strategy which contains the characteristics of human brain is also proposed. Noise-free and noisy numerical examples demonstrate that HNNEMIS has superior super-resolution inversion capabilities for human brain imaging.

Super-Resolution 3-D Microwave Imaging of Objects With High Contrasts by a Semijoin Extreme Learning Machine

This article proposes a semijoin extreme learning machine (SJ-ELM) for super-resolution 3-D microwave imaging of objects with high contrasts. The proposed scheme develops a shallow neural network structure with the semijoin strategy to convert the scattered field data into two output channels, namely, the permittivity and the conductivity of objects, respectively. The semijoin strategy can decrease the inner matrix dimensions to reduce the computational burden for 3-D super-resolution imaging, so it is employed to connect between the nodes of the hidden layer and the output layer. The imaging performance of the proposed SJ-ELM and the conventional variational Born iterative method (VBIM) is first compared for imaging objects with different electrical sizes and contrasts, and then, different targets of imaging resolution are designed to evaluate both solvers. The proposed SJ-ELM is also assessed for imaging objects with high contrasts and experimental data and is demonstrated to have superior super-resolution imaging capabilities for high-contrast 3-D objects.

Distributed Spatial Data Fragmentation and Cross-Border Topological Join Optimization

PDF HTML阅读 XML下载 导出引用 引用提醒 分布式空间数据分片与跨边界拓扑连接优化方法 DOI: 10.3724/SP.J.1001.2011.03798 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: 国家自然科学基金(40971232, 41023001); 国家高技术研究发展计划(863)(2007AA12Z201); 测绘遥感信息工程国家重点实验室开放基金; 测绘遥感信息工程国家重点实验室专项科研经费; 武汉大学优秀博士论文培育基金(2008-25) Distributed Spatial Data Fragmentation and Cross-Border Topological Join Optimization Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:研究分布式空间数据库(distributed spatial database,简称DSDB)中数据按区域分片时的跨边界片段拓扑连接查询问题,并提出相应的优化方法.首先研究了分布式环境下的空间数据的分片与分布,提出了空间数据分片的扩展原则:空间聚集性、空间对象的不分割性、逻辑无缝保持性.然后,将区域分割分片环境下的片段连接分为跨边界和非跨边界两类;同时,将拓扑关系分为两类,重点研究跨边界的两类片段拓扑连接.提出了跨边界空间片段拓扑连接优化的两个定理,并给出了证明.以此为基础,给出了跨边界空间拓扑连接优化规则,包括连接去除规则和连接优化转化规则.最后设计了详细的实验,对自然连接策略、半连接策略以及所提出的连接策略进行效率比较,结果表明,所提出的方法对跨边界连接优化有明显优势.因此,所提出的理论和方法可以用于分布式跨边界拓扑关系查询的优化. Abstract:This paper aims at explaining the cross-border topological join queries of spatial fragments of the zone fragmentation data in distributed spatial database (DSDB), and the optimizing methods for such queries are proposed. First, the fragmentation and distribution of spatial data in a distributed environment are discussed, and the extra principles for spatial data fragmentation are put forward, including spatial clustering, non-partitioning on spatial objects, and maintaining logical seamless. Then, the fragment joins in zone fragmentation are classified into two categories: cross-border join and non-cross-border join; the topological relationships are also classified into two categories. Thus, the emphasis is put on the two types of cross-border topological joins. Two theorems for cross-border topological join optimization are proposed and proved. Based on the theorems, the optimizing rules for cross-border spatial topological join are given, including the removing rules and the transforming rules of fragment joins. Finally, tests are designed to compare three join strategies that include Na?ve join strategy, semi-join strategy and the proposed strategies. The results show that the proposed methods greatly improve the cross-border join optimizing efficiency. Therefore, the theorems and methods proposed in this work can be applied to the optimization of distributed cross-border spatial topological queries. 参考文献 相似文献 引证文献

Distributed stream join query processing with semijoins

This paper addresses the distributed stream processing of window-based multi-way join queries considering the semijoin as a key join operator. In distributed stream processing, data streams arriving at remote sites need to be shipped to the processing site for query execution. This typically introduces high communication overhead. Our observation is that semijoin, effective in reducing communication overhead in distributed database query processing, can be also effective in distributed stream query processing. The challenge, however, lies in the streaming nature of the tuples, as it requires continuous and incremental processing of an unbounded sequence of tuples instead of one-time processing of a set of stored tuples. This paper describes our comprehensive work done to address the challenge. Specifically, we first propose a distributed stream join processing model that handles the issue of network delays introduced from the shipment of data streams, and allows for efficient batch processing. Then, based on the model, we propose join algorithms in a multi-way join case: first, one-way join algorithms for different combinations of join placement and join method and, then, multi-way join algorithms assuming linear join ordering. Regarding the join method, two distributed join methods are introduced: (1) simple join, in which full tuples are forwarded to the query processing site and (2) semijoin-based join, in which partial tuples are forwarded. A semijoin-based join can be executed with different possible semijoin strategies which incur different communication overheads. We present a complete set of join algorithms considering all possible semijoin strategies, and propose an optimization algorithm. The join algorithms are executed continuously in an incremental manner as tuples arrive, and never ship tuples redundantly. The optimization algorithm constructs an efficient multi-way join plan by using a greedy heuristic which adds to the plan one stream with the minimum join execution cost in each step. Through extensive experiments, we conduct comparative studies of the performance among the proposed one-way join algorithms and the efficiency of the generated plan between the optimization algorithm based on the greedy heuristic and the exhaustive search, respectively.

Semijoin-Based Spatial Join Processing in Multiple Sensor Networks

This paper presents an energy-efficient spatial join algorithm for multiple sensor networks employing a spatial semijoin strategy. For optimization of the algorithm, we propose a GR-tree index and a grid-ID-based spatial approximation method, which are unique to sensor networks. The GR-tree is a distributed spatial index over the sensor nodes, which efficiently prunes away the nodes that will not participate in a spatial join result. The grid-ID-based approximation provides great reduction in communication cost by approximating many spatial objects in simpler forms. Our experiments demonstrate that the algorithm outperforms existing methods in reducing energy consumption at the nodes.

Extension of the one-shot semijoin strategy to minimize data transmission cost in distributed query processing

The optimization of general queries in a distributed database management system is an important and challenging research issue. The problem is to find an optimal evaluation strategy for a given general query. In this paper, we propose an approach based on a combination of join and parallel semijoin operations to minimize the amount of data transmission in distributed query processing. First, we describe an efficient distributed query processing strategy using only semijoins. This strategy selects an optimal set instead of a sequence of semijoins to be executed in parallel in three phases: a projection phase, a transmission phase and a reduction phase. Then, we apply a sequence of joins as reducers for query processing. Furthermore, we consider the problem of finding an optimal general sequence that fully reduces a general join query graph. We present a new method that combines parallel and sequential semijoins. We report on experiments that show that our approach based on parallel semijoins is not only efficient but also effective in reducing the total amount of data transmission required to process distributed queries.

A Global Heuristic For Distributed Join Operations

AbstractThis paper deals with the problem of query optimization in relational distributed database systeins. It concentrates on the development of heuristic procedures that attempt to minimize the communication costs incurred by the distributed processing of queries. In particular, the paper deals with a class of heuristics that use a semi-join strategy as the mechanism for communication cost reduction. These heuristics are classified into two types — local and global heuristics. The global heuristic proposed in this paper is based on an optimal solution to a mathematical model of a simplified version of the problem. It is shown that the global heuristic can generate beneficial semi-join operations not included in local heuristic solutions.

Composite semijoins in distributed query processing

Distributed query processing algorithms which use semijoins to reduce data transmission abound in the literature today. In most cases, the strategies of local processing activity and intersite semijoins generated by the algorithms can involve multiple semijoins having a common origination site and a common destination site. Whenever this situation arises in a semijoin strategy, there may be cost benefit in performing the two semijoins as one composite semijoin in which the two joining attributes are treated as one joining composite. We investigate this option and apply it to several well-known query processing algorithms. We show by example and through simulation that composite semijoins can be beneficial. The method of implementation in this paper is such that the response time cost of answering a query cannot be greater than the corresponding response time cost when no composities are used. The simulation shows that, statistically, benefit can be expected in all cases.

Optimising Two-Way Joins in Horizontally Partitioned Database Systems

This paper analyses the problem of joining two horizontally partitioned relations in a distributed database system using a semi-join strategy. A mathematical model is developed and the problem is proved to be NP-complete with respect to the number of fragments. Lower bounding and heuristic procedures are proposed, and the results of computational experiments are reported. These results reveal a good performance of the heuristic procedures, and demonstrate the benefit (in terms of communication cost savings) of using semi-join operations to reduce the size of fragments prior to their transmission to the join site.

Optimization of join operations in horizontally partitioned database systems

This paper analyzes the problem of joining two horizontally partitioned relations in a distributed database system. Two types of semijoin strategies are introduced, local and remote. Local semijoins are performed at the site of the restricted relation (or fragment), and remote semijoins can be performed at an arbitrary site. A mathematical model of a semijoin strategy for the case of remote semijoins is developed, and lower bounding and heuristic procedures are proposed. The results of computational experiments are reported. The experiments include an analysis of the heuristics' performance relative to the lower bounds, sensitivity analysis, and error analysis. These results reveal a good performance of the heuristic procedures, and demonstrate the benefit of using semijoin operations to reduce the size of fragments prior to their transmission. The algorithms for the case of remote semijoins were found to be superior to the algorithms for the case of local semijoins. In addition, we found that the estimation accuracy of the selectivity factors has a significant effect on the incurred communication cost.

An Optimal Algorithm for Processing Distributed Star Queries

The problem of optimal query processing in distributed database systems was shown to be NP-hard. However, for a special type of queries called star queries, we have developed a polynomial optimal algorithm. Semijoin tactics are applied for query processing. An execution graph is introduced to represent the semijoin programs associated with the distributed processing of the queries. We then identify optimality properties of semijoin programs for star queries, and use these properties to derive the optimal semijoin program. We have shown that the optimal semijoin program can be found from serial semijoin strategies, defined as serial semijoin programs which include each semijoin associated with the query exactly once. By making certain assumptions on the file sizes and the semijoin selectivities, we can obtain the optimal semijoin program from these strategies in polynomial time. Our assumption on selectivites is consistent in the sense that we consider the selectivity of a semijoin based on the current database state, i.e., we take into consideration the reduction effects of all prior semijoins.

Optimization of distributed tree queries

In this paper the problem of finding an optimum strategy of semi joins for solving tree queries is studied under the objective of total time minimization. Tree queries that are conjunctions of equi-join clauses such that any two relations in the query have at most one attribute in common are considered. This class of tree queries is a superset of classes of tree queries, such as chain queries and simple queries, that have been studied for semi-join optimization in the literature. An algorithm based on dynamic programming to find the optimum semi-join strategy for a given query is presented. The search space for finding the optimum is reduced by eliminating strategies that can never be the optimum. This is accomplished by utilizing a set of properties that a potentially optimum strategy should satisfy.