Scratch Space Research Articles

SPACE: Structured Compression and Sharing of Representational Space for Continual Learning

Humans learn incrementally from sequential experiences throughout their lives, which has proven hard to emulate in artificial neural networks. Incrementally learning tasks causes neural networks to overwrite relevant information learned about older tasks, resulting in ‘Catastrophic Forgetting’. Efforts to overcome this phenomenon often utilize resources poorly, for instance, by growing the network architecture or needing to save parametric importance scores, or violate data privacy between tasks. To tackle this, we propose SPACE, an algorithm that enables a network to learn continually and efficiently by partitioning the learnt space into a Core space, that serves as the condensed knowledge base over previously learned tasks, and a Residual space, which is akin to a scratch space for learning the current task. After learning each task, the Residual is analyzed for redundancy, both within itself and with the learnt Core space. A minimal number of extra dimensions required to explain the current task are added to the Core space and the remaining Residual is freed up for learning the next task. We evaluate our algorithm on P-MNIST, CIFAR and a sequence of 8 different datasets, and achieve comparable accuracy to the state-of-the-art methods while overcoming catastrophic forgetting. Additionally, our algorithm is well suited for practical use. The partitioning algorithm analyzes all layers in one shot, ensuring scalability to deeper networks. Moreover, the analysis of dimensions translates to filter-level sparsity, and the structured nature of the resulting architecture gives us up to 5x improvement in energy efficiency during task inference over the current state-of-the-art.

ATLAS utilisation of the Czech national HPC center

The Czech national HPC center IT4Innovations located in Ostrava provides two HPC systems, Anselm and Salomon. The Salomon HPC is amongst the hundred most powerful supercomputers on Earth since its commissioning in 2015. Both clusters were tested for usage by the ATLAS experiment for running simulation jobs. Several thousand core hours were allocatedto the project for tests, but the main aim is to use free resources waitigfor large parallel jobs of other users. Multiple strategies for ATLAS job execution were tested on the Salomon and Anselm HPCs. The solution described herein is based on the ATLAS experience with other HPC sites. ARC Compute Element (ARCCE) installed at the grid site in Prague is used for job submission to Salomon. The ATLAS production system submits jobs to the ARC-Evia ARC Control Tower (aCT). The ARC-CE processes job requirements from aTand creates a script for a batch system which is then executed via ssh. Sshfs is used to share scripts and input files between the site and the HPC cluster. The software used to run jobs is rsynced from the site's CVMFS installation to the HPC's scratch space every day to ensure availabiliy of recent software. Using this setting, opportunistic capacity of the Salomon HPC was exploited.

Effect of a novel EZH2 inhibitor GSK126 on prostate cancer cells

Objective: To investigate the effect of a novel EZH2 inhibitor GSK126 on cell growth, apoptosis and migration of prostate cancer cells. Methods: Prostate cancer PC-3 and DU145 cells were treated with GSK126 at different doses. Cell growth was detected by sulforhodamine assay. Cell apoptosis was assayed by Annexin V-/PI kit. Transwell chamber and wound healing assays were conducted to detect cell migration. The mRNA level was detected by quantitative PCR, and protein expression was detected by Western blot analysis. Results: GSK126 showed significant effect on cell growth and apoptosis when the dose was higher than 50 μmol/L. Wound healing assay revealed that scratch space in PC-3 cells was significantly increased in a dose-dependent manner in GSK126-treated groups[(247.2±24.4),(347.2±19.2) and (410.5±18.1) μm in low, medium and high dose (5.0, 20.0, 50.0 μmol/L), respectively] as compared with the control group[(171.3±17.8) μm](all P<0.05). Transwell assay showed that migrated PC-3 cells in control group was 322.0±17.9,while those in GSK126-treated groups were 198.3±15.4 (low),82.7±6.2 (medium) and 30.2±4.1 (high), and the differences between the control group and GSK126-treated groups were significant(all P<0.05). In addition, GSK126 up-regulated E-cadherin mRNA expression and down-regulated N-cadherin and Vimentin mRNA expression, whereas had no significant effect on Snail, Fibronectin and VEGF-A mRNA expression. The protein expression of E-cadherin was elevated but VEGF-A protein did not change in GSK126-treated groups. Similar results were exhibited in DU145 cell. Conclusion: GSK126 can significantly inhibit cell migration and invasion in prostate cancer PC-3 and DU145 cells, which may be resulted from its effect on epithelial-mesenchymal transition. GSK126 may be used as a potential anti-prostate cancer dug in clinic.

Effect of colon cancer metastasis associated gene-1 on biological function of gastric cancer cell line MGC-803 by regulating hepatocyte growth factor/c-Met signaling pathway

Objective To study the effect of colon cancer metastasis associated gene-1 (MACC-1) expression on the proliferation and apoptosis of gastric cancer cell line MGC-803 in vitro. Methods Cultured gastric cancer MGC-803 cells and passaged cells were divided into three groups: group A [MACC-1-short hairpin RNA (shRNA) and PSV2neo3 co transfection group]; group B (shNC and PSV2neo co transfection group); group C (non transfection group). By Western blotting and real-time quantitative polymerase chain reaction (Real-time PCR), the expression of MACC-1 protein and mRNA was detected in the three groups, and that of of c-Met, E-cadherin and N-cadherin in 3 groups of MGC-803 cells were detected. Flow cytometry and scratch test were used to analyze migration, proliferation and apotosis of MGC-803 cells transfected with MACC-1-shRNA. Results (1) Western blotting and Real-time PCR test results showed that as compared with groups B and C, MACC-1 and c-Met protein and mRNA expression in group A were significantly down-regulated (P<0.01). (2) Western blotting and Real-time PCR detection results showed as compared with groups B and C, E-cadherin protein and mRNA expression levels were significantly increased (P<0.01), c-Met and N-cadherin mRNA and protein levels were significantly decreased (P<0.01) in group A. There was no significant difference between groups B and C. (3) Scratch assay showed the scratch space was wide, and healing was inhibited in group A. the migration ability of MGC-803 cells in group A was reduced as compared with groups B and C (P<0.01), but there was no statistically significant difference in the scratch space between groups B and C. (4) Flow cytometry revealed that the apoptosis rate in groups A, B and C was (21.06±1.12)%, (2.18±0.14)% and (2.04±0.11%) respectively. The apoptosis rate in group A was significantly higher than in group C (P<0.01), and there was no significant difference between groups B and C. Conclusion The transfection of MACC-1 gene can promote the proliferation and inhibit the apoptosis of gastric cancer cell line MGC-803 cells, which may play a role in the regulation of hepatocyte growth factor (HGF)/c-Met signal transduction pathway. Key words: Gastric cancer; Colon cancer metastasis associated gene-1; Apoptosis; Biological function

The Effect of Flashcache and Bcache on I/O Performance

Solid state drives (SSDs) provide significant improvements in random I/O performance over traditional rotating SATA and SAS drives. While the cost of SSDs has been steadily declining over the past few years, high density SSDs continue to remain prohibitively expensive when compared to traditional drives. Currently, 1 TB SSDs generally cost more than USD $1,000, while 1 TB SATA drives typically retail for under USD $100. With ever-increasing x86_64 server CPU core counts, and therefore job slot counts, local scratch space density and random I/O performance have become even more important for HEP/NP applications.Flashcache and Bcache are Linux kernel modules which implement caching of SATA/SAS hard drive data on SSDs, effectively allowing one to create hybrid SSD drives using software. In this paper, we discuss our experience with Flashcache and Bcache, and the effects of this software on local scratch storage performance.

On Timely Staging of HPC Job Input Data

Innovative scientific applications and emerging dense data sources are creating a data deluge for high-end supercomputing systems. Modern applications are often collaborative in nature, with a distributed user base for input and output data sets. Processing such large input data typically involves copying (or staging) the data onto the supercomputer's specialized high-speed storage, scratch space, for sustained high I/O throughput. This copying is crucial as remotely accessing the data while an application executes results in unnecessary delays and consequently performance degradation. However, the current practice of conservatively staging data as early as possible makes the data vulnerable to storage failures, which may entail restaging and reduced job throughput. To address this, we present a timely staging framework that uses a combination of job start-up time predictions, user-specified volunteer or cloud-based intermediate storage nodes, and decentralized data delivery to coincide input data staging with job start-up. Evaluation of our approach using both PlanetLab and Azure cloud services, as well as simulations based on three years of Jaguar supercomputer (No. 3 in Top500) job logs show as much as 91.0 percent reduction in staging times compared to direct transfers, 75.2 percent reduction in wait time on scratch, and 2.4 percent reduction in usage/hour. (An earlier version of this paper appears in [30].).

Differentiation of muscle-derived stem cells in the rat penile corpus cavernosum into endothelial cells in vitro

Objective To observe the potential of the rat penile corpus cavernosum muscle-derived stem cells (MDSCs) differentiation into endothelial cells (ECs) in the presence of vascular endothelial growth factor (VEGF) and basic fibroblast growth factor (bFGF) in vitro.Methods MDSCs were purified through density gradient centrifugation and differential attachment.PP1-PP6 cells were obtained,and the expression of stem cell marker Sea-1 and myoblast marker Desmim in PP6 cells was detected by using immunofluorescence.The PP6 cells at third passage were induced to differentiate into endothelial cells within 21 days after VEGF and bFGF stimulation.The differentiating MDSCs were assessed by using immunofluorescence of ECs markers [CD31,fetal liver kinase-1 (Flk-1)],reverse transcriptase-polymerase chain reaction (RT-PCR) of ECs markers [CD31,endothelial nitric oxide synthase (eNOS),Flk-1] and stem cell markers (Oct-4) at 0,5,10,15 and 21 days.Dil-Ac-LDL uptake test was used to test the ability of cells taking up acetylated LDL,and scratch tests was used to observe the scratch space distance after 24 h.Results We observed that PP6 cells expressed MDSCs markers (Desmin,Sca-1) and the differentiating MDSCs expressed ECs markers (CD31,Flk-1) within 21 days after VEGF and bFGF stimulation.As compared with 0 day,the expression of Oct-4 was declined by 0.445 ± 0.025 times,and that of ECs-specific marker mRNA was significantly increased in the differentiated MDSCs at 21 days by 2.541 ± 0.146,2.364 ± 0.175 and 2.538 ± 0.100 times for C D31,eNOS and Flk-1,respectively (P ＜ 0.05 for all).An increased uptake of Dil-Ac-LDL in MDSCs-derived ECs was observed at 21 days compared to undifferentiated MDSCs,and the migration ability of MDSCs-derived ECs was significantly increased.The scratch space distance of differentiated MDSCs was 282.01 ± 3.84 and that of undifferentiated MDSCs was 450.35 ± 1.86 with the difference between statistically significant (P ＜ 0.05).Conclusion The rat penile corpus cavernosum MDSCs can differentiate into ECs in the presence of VEGF and bFGF in vitro and differentiated MDSCs have the typical functions for mature ECs. Key words: Penile corpus cavernosum; Muscle-derived stem cells; Vascular endothelial growth factor; Basic fibroblast growth factor; Differentiation; Endothelial cells

Timely Result-Data Offloading for Improved HPC Center Scratch Provisioning and Serviceability

Modern High-Performance Computing (HPC) centers are facing a data deluge from emerging scientific applications. Supporting large data entails a significant commitment of the high-throughput center storage system, scratch space. However, the scratch space is typically managed using simple “purge policies,” without sophisticated end-user data services to balance resource consumption and user serviceability. End-user data services such as offloading are performed using point-to-point transfers that are unable to reconcile center's purge and users' delivery deadlines, unable to adapt to changing dynamics in the end-to-end data path and are not fault-tolerant. Such inefficiencies can be prohibitive to sustaining high performance. In this paper, we address the above issues by designing a framework for the timely, decentralized offload of application result data. Our framework uses an overlay of user-specified intermediate and landmark sites to orchestrate a decentralized fault-tolerant delivery. We have implemented our techniques within a production job scheduler (PBS) and data transfer tool (BitTorrent). Our evaluation using both a real implementation and supercomputer job log-driven simulations show that: the offloading times can be significantly reduced (90.4 percent for a 5 GB data transfer); the exposure window can be minimized while also meeting center-user service level agreements.

Special issue on interaction with coupled and public displays

Recent developments have seen the widespread proliferation of both large public displays and small personal display technologies. In parallel, novel forms of human– computer interaction such as multi-touch technology have transformed these devices from passive displays into interactive systems. Examples of such small displays include PDAs, Tablets and iPhones while examples of large interactive surfaces (multi-touch driven displays) include the Diamondtouch and Microsoft Surface. With these, users can now interact directly with the displayed objects by simply touching the display, creating a sense of immediacy and natural interaction. This sense of immediacy gives rise to a range of human–computer interactions along with support for multi-device or public display collaborative activities. Furthering Weiser’s vision, the ultimate aim is to weave such new forms of seamless interaction and computation into the fabric of our lives until they are indistinguishable from it. Large interactive surfaces offer great potential for faceto-face work and social interaction and provide natural ways to directly manipulate virtual objects. Small devices afford the individual a personal workspace or ‘‘scratch space’’ to formulate ideas before bringing them to a wider audience. Recent research demonstrates that advanced visual interfaces can be built around large public touchdriven displays or as a combination of both private and public displays coupled together. Such computer-mediated multi-device interaction between local touch-driven displays and shared public ones presents a number of novel and challenging research problems. The special issue is based on an open call and extended versions of papers presented during PPD08. PPD08 was an international workshop on designing multi-touch interaction techniques for coupled public and private displays at AVI 2008, the International Working Conference on Advanced Visual Interfaces held in May 2008 in Napoli, Italy. A number of papers included in this special issue focus on interaction with public displays, while others concentrate on specific areas and applications of coupled multi-device interaction. Boerdonk, Tieben, Klooster and van den Hoven report on the design and evaluation of a multi-user interactive canvas called TouchMeDare to support social interaction. Wallace, Scott, Stutz, Enns and Inkpen report on a comprehensive user study on the impact of display configuration and software interface design on taskwork and teamwork in multi-display groupware systems. They highlight the significant impact of various display configurations on performance. Terrenghi, Quigley and Dix provide a comprehensive taxonomy for the analysis of Multi-Person Display Ecosystems and a detailed analysis of many reference examples. This taxonomy and A. Quigley Complex and Adaptive Systems Laboratory, School of Computer Science and Informatics, University College Dublin, Belfield Office Park, Dublin 4, Ireland

A fast sorting algorithm for large volume datasets

We introduce a new and efficient sorting technique, referred to as BT-Sort after the Bayer-balanced Tree (B-Tree) method. It is fast and parsimonious on usage of memory and scratch space, and is thus deemed very attractive for sorting very large datasets. One property of BT-Sort is that it minimizes the use of random I/O needed for accessing the scratch disk, regardless of the size of datasets being sorted. Furthermore, it uses sequential I/O to/from scratch disks in a very efficient way, and does not require large memory. For example, with such features, sorting 1.2 TB of input shot gathers into ordered common-midpoint gathers with BT-Sort would require only three days using four computing nodes (each of say 32 GB, 375 MHz RISC architecture), whereas the same exercise would typically require six to eight weeks to complete using conventional sorting techniques. From a seismic operational point of view, a rapid increase in seismic traces might result in altering some of the algorithms needed in the processing. Taking Saudi Aramco as an example, the current level of seismic-data acquisition there stands at 10 seismic crews, eight of which are 3D-seismic. The average count on these crews is about 2500 channels per crew. With an average daily production rate of some 2000 vibration points, the corresponding volume of seismic data flowing into processing is some 40 million traces per day. Consequently, some of the processing algorithms are being constantly updated in order to handle these massively increasing seismic volumes. Frequent sorting of seismic data into various domains (e.g. common-midpoint, common-receiver, etc.) has traditionally been avoided mainly because of the lack of efficient sorting algorithms. In the extreme case, sorting very large datasets with only limited computer resources (a few GB of memory) is a hugely inefficient task as the operation could take months to complete. As a compromise, geophysicists at times have opted to reformulate some existing processing algorithms such that seismic data would be kept in a certain preferred order throughout the processing sequence. One example was the reformulation of the dip-moveout operator by Biondi and Ronen (1987), so that it can be applied directly to recorded shot profiles instead of common-offset gathers if data cannot be sorted to common offsets (due to cost or other practical reasons), thereby avoiding the cost-prohibitive sorting process. Another, even worse shortcoming lies in the inability to apply some domain-specific processing algorithms, such as those for noise suppression and Radon multiple removal that require implementation in the common-receiver and common-midpoint (CMP) domains, respectively. In the case of (PRT) to normal-moveout-corrected CMP gathers, for example, geophysicists have attempted to speed up the process by optimizing the PRT operator (Kelamis and Chiburis 1992; Beylkin and Vassiliou 1998); this multiple elimination process in its entirety can benefit even further if data were sorted from shot to CMP gathers in a much faster way such as that proposed here. Unfortunately, lack of a fast sort may inevitably force the seismic data to suffer from unnecessary noisy contaminations if the expensive (slow) sort is to be avoided (Al Dossary et al. 1998). When using conventional (quick or heap) sorting algorithms, one may in theory estimate the cost (in terms of the number of operations to be performed) as N log2 N (Press et al. 1988), where N is the number of traces. On this basis, 500 million traces, for example, would require about 15 billion operations to sort them. In practice, however, such an implementation is more complicated than simply coping with a formidable number of operations. Specifically, the entire dataset should physically reside in memory (internal sorting) in a stand-alone system, a task most computer-hardware configurations cannot support.

In-place reconstruction of version differences

In-place reconstruction of differenced data allows information on devices with limited storage capacity to be updated efficiently over low-bandwidth channels. Differencing encodes a version of data compactly as a set of changes from a previous version. Transmitting updates to data as a version difference saves both time and bandwidth. In-place reconstruction rebuilds the new version of the data in the storage or memory the current version occupies-no scratch space is needed for a second version. By combining these technologies, we support highly mobile applications on space-constrained hardware. We present an algorithm that modifies a differentially encoded version to be in-place reconstructible. The algorithm trades a small amount of compression to achieve this property. Our treatment includes experimental results that show our implementation to be efficient in space and time and verify that compression losses are small. Also, we give results on the computational complexity of performing this modification while minimizing lost compression.

Putting More Genetics into Genetic Algorithms

The majority of current genetic algorithms (GAs), while inspired by natural evolutionary systems, are seldom viewed as biologically plausible models. This is not a criticism of GAs, but rather a reflection of choices made regarding the level of abstraction at which biological mechanisms are modeled, and a reflection of the more engineering-oriented goals of the evolutionary computation community. Understanding better and reducing this gap between GAs and genetics has been a central issue in an interdisciplinary project whose goal is to build GA-based computational models of viral evolution. The result is a system called Virtual Virus (VIV). VIV incorporates a number of more biologically plausible mechanisms, including a more flexible genotype-to-phenotype mapping. In VIV the genes are independent of position, and genomes can vary in length and may contain noncoding regions, as well as duplicative or competing genes. Initial computational studies with VIV have already revealed several emergent phenomena of both biological and computational interest. In the absence of any penalty based on genome length, VIV develops individuals with long genomes and also performs more poorly (from a problem-solving viewpoint) than when a length penalty is used. With a fixed linear length penalty, genome length tends to increase dramatically in the early phases of evolution and then decrease to a level based on the mutation rate. The plateau genome length (i.e., the average length of individuals in the final population) generally increases in response to an increase in the base mutation rate. When VIV converges, there tend to be many copies of good alternative genes within the individuals. We observed many instances of switching between active and inactive genes during the entire evolutionary process. These observations support the conclusion that noncoding regions serve as scratch space in which VIV can explore alternative gene values. These results represent a positive step in understanding how GAs might exploit more of the power and flexibility of biological evolution while simultaneously providing better tools for understanding evolving biological systems.

Computing the MDM T decomposition

The MDM T factorization of an n×n symmetric indefinite matrix A can be used to solve a linear system with A as the coefficient matrix. This factorization can be computed efficiently using an algorithm given in 1977 by Bunch and Kaufman. The LAPACK project has been implementing block versions of well-known algorithms for solving dense linear systems and eigenvalue problems. The block version of the MDM T decomposition algorithm in LAPACK requires the user to specify a block size b by supplying an n×b scratch array. It then makes ( n/b −2) 2 /2 invocations of a matrix-matrix product subroutine with one matrix no larger than b×b, n 2 /(2 b )− b invocations of a matrix-vector product routine with a matrix no larger than b×b , and between n − n/b and 2( n − n/b ) invocations of a matrix-vector product routine with matrices with less than b columns. Because the user can query LAPACK about an optimal block size, our concern is focused on users who cannot change the amount of available scratch space or who neglect to use this facility and are unaware of a performance degradation with small block sizes. This article suggests two alternative algorithms. The first is a block algorithm requiring b×b scratch space and is about 5% slower than LAPACK's current block algorithm with large b . The user does not have to specify the block size. The second algorithm is a rejuvenation of an old implementation of the MDM T decomposition algorithm that requires n matrix-vector products. The performance of the various algorithms on a specific machine is dependent on the manufacturer's implementation of the different basic linear algebra subroutines that they each invoke. Our data indicate that on the Cray Y-MP, Alliant, and Convex the time for the rejuvenated algorithm is either less than or within 10% of that of LAPACK's block algorithm with large b .

An Implementation of Multiple and Multivariate Fourier Transforms on Vector Processors

Several algorithms for multiple and multivariate complex Fourier transforms are suggested. They use little or no scratch space, i.e., they are essentially in-place. Furthermore, they display high levels of regular parallelism needed for vector-parallel computers. Data access is mainly by stride 1. These algorithms correspond to matrix factorizations using transposition permutations. An essential tool used in defining the permutations and the factorizations are Kronecker products. The suggested algorithms are combined with a method for one-dimensional fast Fourier transforms which is also in-place and accesses data with stride 1. Although the number of floating point operations and the total degree of parallelism is left unchanged by the methods used here, data access is much more regular. This is reflected in substantially lower values of a new indicator called loop overhead indicator and often by execution times approximately 2 to 5 times lower, even for moderate problem sizes on the Fujitsu VP 2200, compared to the implementation of the fundamental tensor product factorization.

Why we need ESS signalling theory

Evolutionarily stable strategy (ESS) models of biological signalling are important because the intimate coevolution of signalling and receiving strategies is complicated. Tentative results from a numerical study of error-prone signalling show the value of formal modelling. Error in perception can create discreteness in the distribution of signals produced, and so observed discreteness in nature may call for no more complicated explanation. Further developments in the theory of signalling may include a link with theories of aggression such as the sequential assessment game. The technical device of a 'scratch space' may allow a natural development of 'two-way' information games in which each contestant plays the roles of signaller and receiver simultaneously. This device may also incidentally derive mental states from purely strategic considerations.

Using Strassen's algorithm to accelerate the solution of linear systems

Strassen's algorithm for fast matrix-matrix multiplication has been implemented for matrices of arbitrary shapes on the CRAY-2 and CRAY Y-MP supercomputers. Several techniques have been used to reduce the scratch space requirement for this algorithm while simultaneously preserving a high level of performance. When the resulting Strassen-based matrix multiply routine is combined with some routines from the new LAPACK library, LU decomposition can be performed with rates significantly higher than those achieved by conventional means. We succeeded in factoring a 2048 x 2048 matrix on the CRAY Y-MP at a rate equivalent to 325 MFLOPS.

FFTs in external or hierarchical memory

Conventional algorithms for computing large one-dimensional fast Fourier transforms (FFTs), even those algorithms recently developed for vector and parallel computers, are largely unsuitable for systems with external or hierarchical memory. The principal reason for this is the fact that most FFT algorithms require at least m complete passes through the data set to compute a 2 m -point FFT. This paper describes some advanced techniques for computing an ordered FFT on a computer with external or hierarchical memory. These algorithms (1) require as few as two passes through the external data set, (2) employ strictly unit stride, long vector transfers between main memory and external storage, (3) require only a modest amount of scratch space in main memory, and (4) are well suited for vector and parallel computation. Performance figures are included for implementations of some of these algorithms on Cray supercomputers. Of interest is the fact that a main memory version outperforms the current Cray library FFT routines on the CRAY-2, the CRAY X-MP, and the CRAY Y-MP systems. Using all eight processors on the CRAY Y-MP, this main memory routine runs at nearly two gigaflops.