On-line Data Processing System Research Articles

Development of the Online Data Processing System for the BM@N Experiment at NICA

Development of the Online Data Processing System for the BM@N Experiment at NICA

The essence of online data processing

Data processing systems are a fundamental component of the modern computing stack. These systems are routinely deployed online: they continuously receive the requests of data processing operations, and continuously return the results to end users or client applications. Online data processing systems have unique features beyond conventional data processing, and the optimizations designed for them are complex, especially when data themselves are structured and dynamic. This paper describes DON Calculus, the first rigorous foundation for online data processing. It captures the essential behavior of both the backend data processing engine and the frontend application, with the focus on two design dimensions essential yet unique to online data processing systems: incremental operation processing (IOP) and temporal locality optimization (TLO). A novel design insight is that the operations continuously applied to the data can be defined as an operation stream flowing through the data structure, and this abstraction unifies diverse designs of IOP and TLO in one calculus. DON Calculus is endowed with a mechanized metatheory centering around a key observable equivalence property: despite the significant non-deterministic executions introduced by IOP and TLO, the observable result of DON Calculus data processing is identical to that of conventional data processing without IOP and TLO. Broadly, DON Calculus is a novel instance in the active pursuit of providing rigorous guarantees to the software system stack. The specification and mechanization of DON Calculus provide a sound base for the designers of future data processing systems to build upon, helping them embrace rigorous semantic engineering without the need of developing from scratch.

A Low-Cost Online Data Acquisition and Processing System for Industrial Pollutants’ Monitoring

Recently, the prevention and control of industrial pollution has attracted extensive attention. Many studies based on emission data have achieved great success, but there are relatively few studies on the emission data acquisition. To this end, we propose a data acquisition and processing system (DAPS) for online monitoring of industrial pollutants. The system receives real-time emission data through the RS485, automatically generates periodic data, and saves it in the SQL Server database. Real-time Ethernet (RTE) is used to transmit control signals between the system and programmable logic controllers (PLCs). Emission data is transmitted to local industrial control systems and remote monitoring centers (RMCs) via a wireless network. We tested the acquisition function and communication function of the system through experiments and evaluated the overall performance of the system by observing the memory and CPU usage. The system is characterized by simple implementation and versatility and can be applied to continuous monitoring of various industrial pollutants.

A horizontally scalable online processing system for trigger-less data acquisition

The vast majority of high energy physics experiments rely on data acquisition and hardware-based trigger systems performing a number of stringent selections before storing data for offline analysis. The online reconstruction and selection performed at the trigger level are bound to the synchronous nature of the data acquisition system, resulting in a trade-off between the amount of data collected and the complexity of the online reconstruction performed. Exotic physics processes, such as long-lived and slow-moving particles, are rarely targeted by online triggers as they require complex and nonstandard online reconstruction, usually incompatible with the time constraints of most data acquisition systems. The online trigger selection can thus impact as one of the main limiting factors to the experimental reach for exotic signatures. Alternative data acquisition solutions based on the continuous and asynchronous processing of the stream of data from the detectors are therefore foreseeable as a way to extend the experimental physics reach. Trigger-less data readout systems, paired with efficient streaming data processing solutions, can provide a viable alternative. In this document, an end-to-end implementation of a fully trigger-less data acquisition and online data processing system is discussed. An easily scalable and deployable implementation of such an architecture is proposed, based on open-source distributed computing frameworks capable of performing asynchronous online processing of streaming data. The proposed schema can be suitable for deployment as a fully integrated data acquisition system for small-scale experimental apparatus, or to complement the trigger-based data acquisition systems of larger experiments. A muon telescope setup consisting of a set of gaseous detectors is used as the experimental development testbed in this work, and a fully integrated online processing pipeline deployed on cloud computing resources is implemented and described.

GIPSY-OASIS 기반 LX GNSS 온라인 자료처리 시스템 개발

위성항법시스템의 활용이 증가함에 따라 다양한 자료처리 서비스가 개발되고 있다. 특히 인터넷을 이용한 자료처리 서비스는 사용자가 손쉽게 정확도 높은 자료처리 결과를 얻을 수 있도록 도와준다. 온라인 자료처리 시스템은 전 세계의 여러 연구기관과 국가에서 개발되어 운영되고 있지만 국내 활용이 제한적이다. 이 연구에서는 일반적인 GNSS 후처리 뿐만 아니라, 지적 측량에도 활용 가능한 온라인 자료처리 시스템을 개발하였다. 이 연구에서 개발한 시스템은 국토지리정보원 SUWN을 기준으로 좌표를 산출하며, GIPSY-OASIS의 정밀자료처리(PPP) 기법을 사용한다. 현재시스템은 사용자가 프로그램을 통해 좌표를 산출하고자 하는 관측자료를 FTP를 통해 공간정보연구원의 데이터처리 서버로 업로드하면, 서버에서 자동으로 좌표를 계산하여 유저에게 이메일로 자료처리 결과를 전송한다. 결과검증을 위해 국토지리정보원 상시관측소 고시좌표와 비교한 결과 동서방향으로 약 1.4 cm, 남북방향 -1.0 cm, 수직방향 0.5 cm의 차이를 보였다. Data processing service via internet help user to get the GNSS data processing result more precise and easily. Thus, online data process system is operated and developed by various research groups and national. But this service is difficult to use in domestic cadastral survey. In this study, we developed the online data processing system for a domestic cadastral survey. This is calculated coordinate using NGII CORS(SUWN) fiducially. And use PPP technique by GIPSY-OASIS. If user choose the observation data which want to calculate the coordinate, then is uploaded to GIPSY-OASIS server through FTP. After upload is complete, server automatically calculate coordinate, and send the report about result using e-mail. And it takes 2 minutes runtime on the basis of the 3 sessions. To verify the result, we used the data on SOUL, JUNJ as compared with notified-coordinate from NGII. As a result, got the difference for east-west 1.4 cm, north-south -1.0 cm, vertical 0.5 cm.

多脉冲强流电子束在线测试数据处理

多脉冲强流电子束在线测试数据处理

Quantitative estimation and clinical significance of accumulation and washout of thallium-201 chloride in follicular thyroid neoplasm.

Uptake and washout ratios of thallium-201 chloride (201TlCl) were studied to confirm their clinical applicability in the differential diagnosis of benign and malignant follicular lesions of the thyroid. Sixty-six patients with follicular tumor of the thyroid underwent preoperative thallium scintigram after an intravenous injection of 2 mCi (74 MBq) of 201TlC1. The early accumulation and washout ratios of 201TlCl were obtained by an online data-processing system. All tumors were surgically resected and histopathologically diagnosed as either follicular adenoma (49 patients) or follicular carcinoma (17 patients). Scintigraphic values in terms of the early accumulation and washout ratios were compared between follicular adenoma and follicular carcinoma. Both the early accumulation and washout ratios were significantly higher in follicular carcinoma than in follicular adenoma. It was concluded that dynamic studies on accumulation and washout rates of 201TlCl might be clinically reliable to differentiate between follicular adenoma and follicular carcinoma.

Instrumentation for and first results on nuclear responses for supernova explosions

Our collaboration has set up a focal plane detection system and a focal plane polarimeter at the large acceptance Big-Bite Spectrometer at AGOR. The detector systems are equipped with a high performance readout and online data processing system, which allows polarization transfer and charge transfer measurements at extreme forward angles with high precision. Preliminary results on GT+ strength distributions obtained in (d,2He) measurements revealing the fine structure of the distributions are presented. Their relation to recent calculations of stellar weak interaction rates is discussed.

A device for on-line electromigration measurements of radioelements in free electrolytes

An electromigration device has been developed for the on-line measurement of absolute individual ion mobilities of radioactive species in free electrolytes. The device is composed of a migration cell, a γ-radiation scanning unit and a data processing system. The performance of two BGO-scintillators as scanning unit and the reliability of an on-line data processing system were studied with calibrated radioactive sources of given velocities. The system was found to work with a high reproducibility rate. The migration velocities were determined with an accuracy better than 1%.

The Development of a Substructure On-Line Testing System for Seismic Response Analysis of a Geotechnical System

A substructure on-line testing (SOT) system was developed for the purpose of conducting a quantitative simulation of the behavior of horizontal subsurface layers subjected to earthquake motions (SH wave) under undrained conditions. In this method, a computer analysis of the seismic response and a pseudo-dynamic loading test which estimates the restoration force of the materials were combined with a computer on-line data processing system. With the SOT system, which was devised to analyze the overall seismic response behavior of subsoils, an on-line test was adopted only for layers demonstrating complicated hysteretic behavior with respect to a restoration force, while a numerical model was applied to the remaining layers. The following tests were conducted in order to verify this system: 1) The reliability of a series of 6 hollow torsional shear apparatus installed in the SOT system was verified by comparing the test results with the results of cyclic undrained simple shear tests conducted by Tatsuoka et al. (1989); 2) A test on a quasi-elastic specimen was conducted based on the SOT system using a test algorithm developed for this study, and an elastic analysis was carried out using the modulus of elasticity obtained from this test. Based on a comparison between the respective results of the test and the analysis, the reliability of the test algorithm was verified; 3) Using the system thus verified, a seismic response liquefaction test on a saturated sandy subsoils was conducted, which showed that the SOT system developed in this study was capable of accurately simulating the seismic response of a level subsoils.

定容積型一面せん断試験機による粉体崩壊曲線の測定について

A new constant volume type direct shear test method is described. In this method, the volume of a powder bed is kept constant by adjusting a normal load, and the P. Y. L., containing Y. L. and C. Y. L., can be directly obtained from the observed shearing curve. Then, a shear tester, which has a normal load adjusting device and the on-line data processing system, has been developed. By operating this tester and observing the shear behavior in the powder bed, the suitable sizes of the shear cell and the suitable experimental conditions are determined. When using this cell, it can be proved that the Y. L. and the C. Y. L. connect smoothly on a critical state line. These results show that the P. Y. L. can be exactly obtained from only a Y. L. test and a C. Y. L. test.It was shown that this method is available for measuring the P. Y. L.

High speed graphic program on a personal computer and its utilization in JIPP T-IIU online data-processing system.

A high speed graphic program was developed on a personal computer PC 9801. Using this program, one can draw a waveform of successive 16 bit-integer data, such as obtained by analog-to-digital convertor. The program is written by the machine language and has a form of a subroutine program which can be called from main programs under N88 BASIC. The time for drawing one waveform is 4 ms, which is two orders faster than the time with standard graphic routines of BASIC interpreter. This program is very convenient for the real-time display of plasma-monitoring raw data, such as plasma current, loop voltage, rf power etc. in tokamak experiments. This program has been utilized in JIPP T-IIU experiments and enables to display data of 8 channel ADC within a few seconds before the system transmits the data from CAMAC to the computer center of the institute. The program and its utilization are presented.

A compact on-line data processing system for the Tokyo Surface Ship Gravity Meter.

As the first step of developing a compact surface ship gravity meter, a compact gravity data processing system on-Iine with a Tokyo Surface Ship Gravity Meter (T.S.S.G.) was newly designed and tested at sea. This system was made compact by using a single board computer for gravity computations, a handhold personal computer for data recording, and a Loran C receiver for navigation. Free-air anomalies are obtained by this on-line processing. Over-all accuracy of this data processing is estimated to be 2-3 mgal; if the error from the Eotvos correction is not considered, the accuracy is estimated to be 0.1 mgal. The data processing system was successfully tested on board a research vessel and used for the gravity measurements around Japan in the case of the Japan-France Cooperative Project KAIKO.

長期間にわたるラットの行動量,EEG,EOG,EMGのオンライン,オフライン並列情報処理システムと応用実験

長期間にわたるラットの行動量,EEG,EOG,EMGのオンライン,オフライン並列情報処理システムと応用実験

An Inexpensive, On-Line Data Processing System for Gel Permeation Chromatography

Abstract An inexpensive, on-line data processing system is described for the Waters 200 gel permeation chromatograph. The system consists of (a) interface chassis; (b) microprocessor with ∼3K memory and (c) a console device with cassette tapes for the storage of data and programs.

The On-Line Data Processing System at Nuclear and High Energy Physics Experiments

The On-Line Data Processing System at Nuclear and High Energy Physics Experiments

ディジタル・ボイド速度計の開発

A digital void velocimeter with a single-type probe was developed that enables the measurement of void surface transit-time in a gas-liquid two-phase flow irrespective of void surface deformation. The accuracy of void velocity measurement by the conventional single-type probe technique is governed by the deformation of the void surface, and the error is within ±10% for the group of voids with almost identical curvatures.The present scheme was made possible by building a signal control circuit linked to a microcomputer. An on-line data processing system was also developed, and the error of the measurement was within ±6%.

Real time online data processing system for the e.e.g. and body movement during the lifetime of a freely moving mouse

Real time online data processing system for the e.e.g. and body movement during the lifetime of a freely moving mouse

On-Line Computerized Data Handling System for Treating Patients With Renal Disease

Some patients with renal disease accrue 6,000 to 8,000 individual data items relating to symptoms, signs, laboratory information, and treatment in one year. To deal with the problem of handling so many data items, the following steps were taken: (1) a dictionary of terms peculiar to nephrology was created; (2) manual time-oriented records for nephrology were constructed; and (3) an on-line data processing system, using a PDP-11/70 computer and remote teleprocessing terminal, was developed. On the terminal screen, up to 11 consecutive patient visits can be displayed horizontally, with 18 data items displayed vertically. Up to 30 of the most recent patient visits are easily accessible. For patient treatment, a special feature allows a rearrangement and instantaneous display of any combination of data, thus permitting review of essential feedback relationships between clinical events and treatment.

A Mini-Based Banking System

In Japan, on-line banking systems were the first systems to offer computerized services on a full scale. Indeed, together with seat reservation systems, banking systems led the technological development of on-line data processing systems in this country. The reasons for this were much the same as those that prompted the same phenomenon in the US: