Static Data Distribution Research Articles

Prediction Based Semi-Supervised Online Personalized Federated Learning for Indoor Localization

Fingerprint-based indoor localization has drawn increasing attention with the development of deep learning. Nevertheless, it faces challenges from frequent data collection and the corresponding exposure of privacy. Federated Learning (FL) is introduced into indoor localization recently for overcoming these challenges. However, most of the current FL-based indoor localization studies only focus on the static data distributions and ignore the fact that users have their preferred localization requirements. In this paper, two typical indoor localization scenarios are considered. Clients with various localization demands and data distributions collect online unlabeled data in Scenario I. For Scenario II, only one requesting user (RU) with high mobility applies for the localization service. A Prediction based Semi-supervised Online Personalized Federated Learning (PSO-PFL) is proposed for addressing these problems in the two scenarios. Experiments are conducted based on two real-world datasets. The experiment results show that PSO-PFL achieves higher personalization accuracy than centralized training and Federated Averaging, and utilizes unlabeled data efficiently. The strategy of selecting clients via prediction achieves higher accuracy than the random selection strategy. In brief, the proposed methods protect users’ privacy via FL. And the challenges of dynamical and heterogeneous stream data in indoor localization are addressed by the proposed methods which provide better personalization localization service for users than baseline methods.

EER-ASSL: Combining Rollback Learning and Deep Learning for Rapid Adaptive Object Detection

We propose a rapid adaptive learning framework for streaming object detection, called EER-ASSL. The method combines the expected error reduction (EER) dependent rollback learning and the active semi-supervised learning (ASSL) for a rapid adaptive CNN detector. Most CNN object detectors are built on the assumption of static data distribution. However, images are often noisy and biased, and the data distribution is imbalanced in a real world environment. The proposed method consists of collaborative sampling and EER-ASSL. The EER-ASSL utilizes the active learning (AL) and rollback based semi-supervised learning (SSL). The AL allows us to select more informative and representative samples measuring uncertainty and diversity. The SSL divides the selected streaming image samples into the bins and each bin repeatedly transfers the discriminative knowledge of the EER and CNN models to the next bin until convergence and incorporation with the EER rollback learning algorithm is achieved. The EER models provide a rapid short-term myopic adaptation and the CNN models an incremental long-term performance improvement. EER-ASSL can overcome noisy and biased labels in varying data distribution. Extensive experiments shows that EER-ASSL obtained 70.9 mAP compared to state-of-the-art technology such as Faster RCNN, SSD300, and YOLOv2.

Exploring efficient data parallelism for genome read mapping on multicore and manycore architectures

Nowadays heterogeneous architectures formed by multicore and manycore systems have become attractive solutions to cope with the data booming in genomic-based studies. Our work explores the efficient usage of heterogeneous architectures in such area. In particular, we have studied the use of manycore components like the Xeon Phi accelerator, which has proved to be a convenient choice because it allows an easy migration of applications developed for multicore servers based on the × 86 architecture. Our study also focuses on the problem of sequence alignment, which is one of the fundamental and most costly computational stages in most genome variant studies. We concentrate our attention on BWA, one of the most popular sequence aligners, and we have focused our attention on three types of heterogeneous systems, one containing Intel multi-core CPUs and accelerators, one that are made up of several multi-core servers, and one large-scale system. Each with different characteristics in terms of number of CPUs, number of cores and system organization memory. Although the problem of alignment of sequences fits in the embarrassingly parallel pattern, achieving good performance and good scalability in heterogeneous environments can be complex. We have analyzed different strategies based on the distribution of data and the replication of certain data structures and we found that MDPR (Multi-level Data Parallelization and Replication) strategy has shown the best results in all the heterogeneous platforms tested. Its results have surpassed other strategies proposed in the literature and have shown its malleability to be used in different heterogeneous environments without the need to apply specific adjustments according to the underlying architecture. In the design of MDPR, different static and dynamic data distribution strategies have also been evaluated. The best results were obtained by the static strategy, which has a significant preprocessing cost. However, the dynamic strategy of data distribution using a round-robin mechanism obtained similar times without the need for the preprocessing stage. Although our proposal was applied to BWA using human genome data samples, this strategy can be easily applied to other sequence datasets and alignment tools that have similar operating principles with those of BWA aligner.

Deep Learning-Based Data Storage for Low Latency in Data Center Networks

Low-latency data access is becoming an upcoming and increasingly important challenge. The proper placement of data blocks can reduce data travel among distributed storage systems, which contributes significantly to the latency reduction. However, the dominant data placement optimization has primarily relied on prior known data requests or static initial data distribution, which ignores the dynamics of clients' data access requests and networks. The learning technology can help the data center networks (DCNs) learn from historical access information and make optimal data storage decision. Consider a more practical DCNs with fat-tree topology, we utilize a deep-learning technology k-means to help store data blocks and then improve the read and write latency of the DCN, where k is the number of cores in the fat-tree. The evaluation results demonstrate that the average write and read latency of the whole system can be lowered by 33% and 45%, respectively. And the best set of parameter k is analyzed and recommended to provide guidance to the real application, which is equal to the number of cores in the DCNs.

Emergent community agglomeration from data set geometry.

In the statistical learning language, samples are snapshots of random vectors drawn from some unknown distribution. Such vectors usually reside in a high-dimensional Euclidean space, and thus the "curse of dimensionality" often undermines the power of learning methods, including community detection and clustering algorithms, that rely on Euclidean geometry. This paper presents the idea of effective dissimilarity transformation (EDT) on empirical dissimilarity hyperspheres and studies its effects using synthetic and gene expression data sets. Iterating the EDT turns a static data distribution into a dynamical process purely driven by the empirical data set geometry and adaptively ameliorates the curse of dimensionality, partly through changing the topology of a Euclidean feature space R^{n} into a compact hypersphere S^{n}. The EDT often improves the performance of hierarchical clustering via the automatic grouping information emerging from global interactions of data points. The EDT is not restricted to hierarchical clustering, and other learning methods based on pairwise dissimilarity should also benefit from the many desirable properties of EDT.

Revised cloud storage structure for light-weight data archiving in LHD

The LHD data archiving system has newly selected GlusterFS distributed filesystem for the replacement of the present cloud storage software named “IznaStor/dSS”. Even though the prior software provided many favorable functionalities of hot plug and play node insertion, internal auto-replication of data files, and symmetric load balancing between all member nodes, it revealed a poor feature in recovering from an accidental malfunction of a storage node. Once a failure happened, the recovering process usually took at least several days or sometimes more than a week with a heavy cpu load. In some cases they fell into the so-called “split-brain” or “amnesia” condition, not to get recovered from it. Since the recovery time tightly depends on the capacity size of the fault node, individual HDD management is more desirable than large volumes of HDD arrays. In addition, the dynamic mutual awareness of data location information may be removed if some other static data distribution method can be applied. In this study, the candidate middleware of “OpenStack/Swift” and “GlusterFS” has been tested by using the real mass of LHD data for more than half a year, and finally GlusterFS has been selected to replace the present IznaStor. It has implemented very limited functionalities of cloud storage but a simplified RAID10-like structure, which may consequently provide lighter-weight read/write ability. Since the LABCOM data system is implemented to be independent of the storage structure, it is easy to plug off the IznaStor and on the new GlusterFS. The effective I/O speed is also confirmed to be on the same level as the estimated one from raw performance of disk hardware. This achievement may be informative to implement the ITER CODAC and the remote archiving system.

Dynamic self-organising map

We present in this paper a variation of the self-organising map algorithm where the original time-dependent (learning rate and neighbourhood) learning function is replaced by a time-invariant one. This allows for on-line and continuous learning on both static and dynamic data distributions. One of the property of the newly proposed algorithm is that it does not fit the magnification law and the achieved vector density is not directly proportional to the density of the distribution as found in most vector quantisation algorithms. From a biological point of view, this algorithm sheds light on cortical plasticity seen as a dynamic and tight coupling between the environment and the model.

Toward Malleable Model Coupling

Abstract Model coupling is a method to simulate complex multiphysics and multiscale phenomena. Most approaches involve static data distribution among processes without the consideration of top-level dynamic load balancing. Malleability, the ability to change the number of processes during execution, allows applications to configure themselves to better utilize available system resources. To date, however, malleability has been applied primarily to monolithic applications. We have extended the Model Coupling Toolkit (MCT) to support processing element malleability for coupled models, resulting in the Malleable Model Coupling Toolkit (MMCT). MMCT consists of a load balance manager (LBM) implementing a practical dynamic load-balancing algorithm and a malleable model registry that allows management of dynamically evolving MPI communicators. MMCT requires only standard MPI-2, sockets, and MCT. We benchmark MMCT using a synthetic, simplified coupled model application similar to the Community Climate System Model. Preliminary performance data demonstrate the efficacy of the LBM and a low (≈3%) monitoring overhead.

A Cluster Based Replication Architecture for Load Balancing in Peer-to-Peer Content Distribution

In P2P systems, large volumes of data are declustered naturally across a large number of peers. But it is very difficult to control the initial data distribution because every user has the freedom to share any data with other users. The system scalability can be improved by distributing the load across multiple servers which is proposed by replication. The large scale content distribution systems were improved broadly using the replication techniques. The demanded contents can be brought closer to the clients by multiplying the source of information geographically, which in turn reduce both the access latency and the network traffic. In addition to this, due to the intrinsic dynamism of the P2P environment, static data distribution cannot be expected to guarantee good load balancing. If the hot peers become bottleneck, it leads to increased user response time and significant performance degradation of the system. Hence an effective load balancing mechanism is necessary in such cases and it can be attained efficiently by intelligent data replication. In this paper, we propose a cluster based replication architecture for load-balancing in peer-to-peer content distribution systems. In addition to an intelligent replica placement technique, it also consists of an effective load balancing technique. In the intelligent replica placement technique, peers are grouped into strong and weak clusters based on their weight vector which comprises available capacity, CPU speed, access latency and memory size. In order to achieve complete load balancing across the system, an intracluster and inter-cluster load balancing algorithms are proposed. We are able to show that our proposed architecture attains less latency and better throughput with reduced bandwidth usage, through the simulation results.

QR factorization for shared memory and message passing

This paper describes the design, implementation, and performance of three new parallel QR factorization algorithms: shared memory, synchronous message passing, and asynchronous message passing. In contrast to existing parallel algorithms, the multiprocessor partitioning strategy is not governed by an underlying static data distribution scheme. Rather, a dynamic distribution strategy is employed to improve scalability on small problems. Experiments conducted on a 128-processor SGI Origin 2000 and a 64-processor HP SPP-2000 show that the new algorithms have a lower execution time than available tuned parallel routines installed on the machines including a version of ScaLAPACK’s distributed QR factorization algorithm PDGEQRF.

Static LU Decomposition on Heterogeneous Platforms

In this paper, the authors deal with algorithmic issues on heterogeneous platforms. They concentrate on dense linear algebra kernels, such as matrix multiplication or LU decomposition. Block-cyclic distribution techniques used in ScaLAPACK are no longer sufficient to balance the load among processors running at different speeds. The main result of this paper is to provide a static data distribution scheme that leads to an asymptotically perfect load balancing for LU decomposition, thereby providing solid foundations toward the design of a cluster-oriented version of ScaLAPACK.

SUPPLE: An efficient run-time support for non-uniform parallel loops

Abstract This paper presents SUPPLE (SUPort for Parallel Loop Execution), an innovative run-time support for the execution of parallel loops with regular stencil data references and non-uniform iteration costs. SUPPLE relies upon a static block data distribution to exploit locality, and combines static and dynamic policies for scheduling non-uniform iterations. It adopts, as far as possible, a static scheduling policy derived from the owner computes rule, and moves data and iterations among processors only if a load imbalance actually occurs. SUPPLE always tries to overlap communications with useful computations by reordering loop iterations and prefetching remote ones in the case of workload imbalance. The SUPPLE approach has been validated by many experimental results obtained by running a multi-dimensional flame simulation kernel on a 64-node Cray T3D. We have fed the benchmark code with several synthetic input data sets built on the basis of a load imbalance model. We have compared our results with those obtained with a CRAFT Fortran implementation of the benchmark.

ALGORITHMIC ISSUES ON HETEROGENEOUS COMPUTING PLATFORMS

This paper discusses algorithmic issues when computing with a heterogeneous network of work-stations (the typical poor man's parallel computer). Dealing with processors of different speeds requires to use more involved strategies than block-cyclic data distributions. Dynamic data distribution is a first possibility but may prove impractical and not scalable due to communication and control overhead. Static data distributions tuned to balance execution times constitute another possibility but may prove ineffcient due to variations in the processor speeds (e.g. because of different workloads during the computation). We introduce a static distribution strategy that can be refined on the fly, and we show that it is well-suited to parallelizing scientific computing applications such as finite-difference stencils or LU decomposition.

A Comparison of Implementation Strategies for Nonuniform Data-Parallel Computations

Data-parallel languages allow programmers to easily express parallel computations by means of high-level constructs. To reduce overheads, the compiler partitions the computations among the processors at compile-time, on the basis of the static data distribution suggested by the programmer. When execution costs are nonuniform and unpredictable, some processors may be assigned more work than others. Workload imbalance can be mitigated by cyclically distributing data and associated computations or by employing adaptive strategies which build a more balanced schedule at run-time, on the basis of the actual execution costs. This paper discusses static and hybrid (static+dynamic) scheduling strategies which can be used to balance the workloads derived from the execution of nonuniform parallel loops. A multidimensional flame simulation kernel has been used to evaluate different implementation strategies on a Cray T3E. We fed the benchmark code with synthetic input data sets built on the basis of a load imbalance model and we report and compare the results obtained.

Dynamic Data Partitioning for Distributed-Memory Multicomputers

For distributed-memory multicomputers such as the Intel Paragon, the IBM SP-1/SP-2, the NCUBE/2, and the Thinking Machines CM-5, the quality of the data partitioning for a given application is crucial to obtaining high performace. This task has traditionally been the user's responsibility, but in recent years much effort has been directed to automating the selection of data partitioning schemes. Several researchers have proposed systems that are able to produce data distributons that remain in effect for the entire execution of an application. For complex programs, however, such static data distributions may be insufficient to obtain acceptable performance. The selection of distributions that dynamically change over the course of a program's execution adds another dimension to the data partitioning problem. In this paper, we present a technique that can be used to automatically determine which partitionings are most beneficial over specific sections of a program while taking into account the added overhead of performing redistribution. This system has been implemented as part of the PARADIGM (PARAllelizing compiler for DIstributed-memory General-purpose Multicomputers) project at the University of Illinois. The complete system strives to provide a fully automated means to parallelize programs written in a serial programming model obtaining high performance on a wide range of distributed-memory multicomputers.

Parallel event-driven MOS timing simulator on distributed memory multiprocessors

PMOTA, a parallel event-driven relaxation-based timing simulator of MOS circuits on distributed memory multiprocessors, is proposed. Two new schemes, one a static data distribution and dynamic event assignment, the other a global synchronism local asynchronism scheme, are derived to reduce the communication overhead and achieve high speedup ratio, high processor utilisation, and balanced load among processor elements. Implementation results show that parallel simulations on a distributed memory multiprocessor are feasible for large-scale circuit simulation.