Log Storage Research Articles

System log isolation for containers

Container-based virtualization is increasingly popular in cloud computing due to its efficiency and flexibility. Isolation is a fundamental property of containers and weak isolation could cause significant performance degradation and security vulnerability. However, existing works have almost not discussed the isolation problems of system log which is critical for monitoring and maintenance of containerized applications. In this paper, we present a detailed isolation analysis of system log in current container environment. First, we find several system log isolation problems which can cause significant impacts on system usability, security, and efficiency. For example, system log accidentally exposes information of host and co-resident containers to one container, causing information leakage. Second, we reveal that the root cause of these isolation problems is that containers share the global log configuration, the same log storage, and the global log view. To address these problems, we design and implement a system named private logs (POGs). POGs provides each container with its own log configuration and stores logs individually for each container, avoiding log configuration and storage sharing, respectively. In addition, POGs enables private log view to help distinguish which container the logs belong to. The experimental results show that POGs can effectively enhance system log isolation for containers with negligible performance overhead.

A secure and efficient log storage and query framework based on blockchain

Log data is crucial for security threat detection and audit analysis. However, traditional log systems are susceptible to tampering, posing a significant security risk to information systems. Although blockchain technology has been introduced to enhance tamper resistance, existing blockchain-based log systems still suffer from storage and query efficiency issues. In this paper, we propose a novel secure and efficient log storage and query framework that combines on-chain and off-chain collaboration. An inverted index table is constructed by extracting keywords from logs, which are stored on the blockchain as on-chain data, while the logs themselves are maintained as off-chain data. This approach facilitates the rapid retrieval of specific keywords and ensures the immutability of the logs. Furthermore, we propose a secure and efficient log query method featuring a smart contract designed to automatically handle requests from legitimate log queriers. We also design a data structure based on merkle adaptive radix tree (MART) and merkle B+ tree (MBT) to store index entries, thereby achieving efficient log retrieval. We provide formal security proofs and comprehensively evaluate the proposed framework’s performance experimentally. Results demonstrate that MBT and MART reduce average query times by 20.09% and 51% respectively, compared to the state-of-the-art schemes.

Exploiting Data-pattern-aware Vertical Partitioning to Achieve Fast and Low-cost Cloud Log Storage

Cloud logs can be categorized into on-line, off-line, and near-line logs based on the access frequency. Among them, near-line logs are mainly used for debugging, which means they prefer a low query latency for better user experience. Besides, the storage system for near-line logs prefers a low overall cost including the storage cost to store compressed logs, and the computation cost to compress logs and execute queries. These requirements pose challenges to achieving fast and cheap cloud log storage. This article proposes LogGrep, the first log compression and query tool that exploits both static and runtime patterns to properly structurize and organize log data in fine-grained units. The key idea of LogGrep is “vertical partitioning”: it stores each log entry into multiple partitions by first parsing logs into variable vectors according to static patterns and then extracting runtime pattern(s) automatically within each variable vector. Based on such runtime patterns, LogGrep further decomposes the variable vectors into fine-grained units called “Capsules” and stamps each Capsule with a summary of its values. During the query process, LogGrep can avoid decompressing and scanning Capsules that cannot match the keywords, with the help of the extracted runtime patterns and the Capsule stamps. We further show that the interactive debugging can well utilize the advantages of the vertical-partitioning-based method and mitigate its weaknesses as well. To this end, LogGrep integrates incremental locating and partial reconstruction to mitigate the read amplification incurred by vertical-partitioning-based method. We evaluate LogGrep on 37 cloud logs from the production environment of Alibaba Cloud and the public datasets. The results show that LogGrep can reduce the query latency and the overall cost by an order of magnitude compared with state-of-the-art works. Such results have confirmed that it is worthwhile applying a more sophisticated vertical-partitioning-based method to accelerate queries on compressed cloud logs.

Research and optimisation of low-cost cloud log storage system based on data pattern awareness

Research and optimisation of low-cost cloud log storage system based on data pattern awareness

A blockchain-based log storage model with efficient query

A blockchain-based log storage model with efficient query

LCHAIN: A Secure Log Storage Mechanism using IPFS and Blockchain Technology

Data security is a very important and crucial part of Cloud storage. Day by day, thousands of operations function over the Cloud, verify logs generated from all transactions is very difficult. The attacker may temper or remove targeted logs or attack traces on the system with stealthy techniques. It is required to maintain the security of the log to trace back all the transactions to identify such tempering and loss of logs. Temper-proof log storage is a challenging issue on the Cloud. To overcome the issue, we propose strong and secure log storage using Blockchain technology and IPFS. Detecting log tempering and tracing is challenging due to large log volumes. We recommend the usage of Blockchain technology due to its inherent feature of immutability to address the issue of log tempering. We present LChain which provides immutable storage of logs with tracing. Blockchain technology helps yp create immutable logs but also offers non-repudiation and scalability. Smart contracts are used for efficient searching and enhancing computational power.

Cloud Storage untuk Embedded Intrusion Detection System

The Corona Virus (COVID-19) pandemic has had a major social and economic impact on the world. Along with the potential challenges of sharing domains, brings with it many cybersecurity challenges that need to be addressed in a timely manner for critical infrastructure. The increase in the use of internet technology during this pandemic is directly proportional to the increase in the development of Information and Communication Technology (ICT) and cybercrime. Therefore, it is necessary to elaborate the existing ICTs to reduce the impact caused by attacks on the network according to the needs and capabilities of the users. This study applies a Network Intrusion Detection System (NIDS) based on the Raspberry Pi 4 Model B using Snort IDS with log storage media on cloud storage by visualizing the alerts generated to facilitate the analysis of anomalies that occur on the network. The result of this research is that there are attack signatures that are not available in the default rules so that further configuration is needed on Snort. The performance of the IDS sensor does not reduce the capability of the IDS sensor which acts as a hotspot when an attack occurs.

Immutable Log Storage as a Service on Private and Public Blockchains

Service Level Agreements (SLA) are employed to ensure the performance of Cloud solutions. When a component fails, the importance of logs increases significantly. All departments may turn to logs to determine the cause of the issue and find the party at fault. The party at fault may be motivated to tamper with the logs to hide their role. We argue that the critical nature of Cloud logs calls for immutability and verification mechanism without the presence of a single trusted party. This article proposes such a mechanism by describing a blockchain-based log storage system, called Logchain, which can be integrated with existing private and public blockchain solutions. Logchain uses the immutability feature of blockchain to provide a tamper-resistance platform for log storage. Additionally, we propose a hierarchical structure to address blockchains’ scalability issues. To validate the mechanism, we integrate Logchain into Ethereum and IBM Blockchain. We show that the solution is scalable and perform the analysis of the cost of ownership to help a reader select an implementation that would address their needs. The Logchain's scalability improvement on a blockchain is achieved without any alteration of blockchains’ fundamental architecture. As shown in this work, it can function on private and public blockchains and, therefore, can be a suitable alternative for organizations that need a secure, immutable log storage platform.

Vehicle to Everything (V2X) and Edge Computing: A Secure Lifecycle for UAV-Assisted Vehicle Network and Offloading with Blockchain

Due to globalization and advances in network technology, the Internet of Vehicles (IoV) with edge computing has gained increasingly more attention over the last few years. The technology provides a new paradigm to design interconnected distributed nodes in Unmanned Aerial Vehicle (UAV)-assisted vehicle networks for communications between vehicles in smart cities. The process hierarchy of the current UAV-assisted networks is also becoming more multifaceted as more vehicles are connected, requiring accessing and exchanging information, performing tasks, and updating information securely. This poses serious issues and limitations to centralized UAV-assisted vehicle networks, directly affecting computing-intensive tasks and data offloading. This paper bridges these gaps by providing a novel, transparent, and secure lifecycle for UAV-assisted distributed vehicle communication using blockchain hyperledger technology. A modular infrastructure for Vehicle-to-Everything (V2X) is designed and ‘B-UV2X’, a blockchain hyperledger fabric-enabled distributed permissioned network-based consortium structure, is proposed. The participating nodes of the vehicle are interconnected with others in the chain of smart cities and exchange different information such as movement, etc., preserving operational logs on the blockchain-enabled immutable ledger. This automates IoV transactions over the proposed UAV-assisted vehicle-enabled consortium network with doppler spread. Thus, for this purpose, there are four different chain codes that are designed and deployed for IoV registration, adding new transactions, updating the ledger, monitoring resource management, and customized multi-consensus of proof-of-work. For lightweight IoV authentication, B-UV2X uses a two-way verification method with the defined hyperledger fabric consensus mechanism. Transaction protection from acquisition to deliverance and storage uses the NuCypher threshold proxy re-encryption mechanism. Simulation results for the proposed B-UV2X show a reduction in network consumption by 12.17% compared to a centralized network system, an increase in security features of up to 9.76%, and a reduction of 7.93% in the computational load for computed log storage.

LogInjector: Detecting Web Application Log Injection Vulnerabilities

Web applications widely use the logging functionality, but improper handling can bring serious security threats. An attacker can trigger the execution of malicious data by writing malicious data to the web application logs and then accessing the view–logs interface, resulting in a vulnerability of the web application log injection. However, detecting this type of vulnerability requires automatic discovery of log-injectable interfaces and view–logs interfaces, which is difficult. In addition, bypasssing the application-specific input-filtering checks to write an effective payload to the log is also challenging. This paper proposes LogInjector, an efficient web application log injection vulnerability detection method. First, it obtains the log storage form and location and then finds the log-injectable interfaces through the extended dynamic crawler. Second, it automatically identifies the web application view–logs interfaces. Finally, LogInjector utilizes a dynamic testing approach based on the feedback-guided mutation to detect web application log injection vulnerabilities. To verify the effectiveness of LogInjector, we test it in 14 popular web applications in real-world cases and compare it with Black Widow, the state-of-the-art web vulnerability scanner. LogInjector detects 16 web application log injection vulnerabilities, including 6 zero-day vulnerabilities, while Black Widow can only detect three log injection vulnerabilities, demonstrating the effectiveness of LogInjector in practice.

Processability and physico‐mechanical properties of ultrahigh‐molecular‐weight polyethylene using low‐molecular‐weight olefin wax

AbstractThe influence of molecular weight of olefin wax as a plasticizer on the processability and properties of ultrahigh‐molecular‐weight polyethylene (UHMWPE) was thoroughly investigated in this study. Two different grades of olefin wax (H5 and 210P) were used separately to plasticize UHMWPE. Comparative results were analyzed based on mixing torque, mechanical, thermal, thermo‐mechanical, and morphological aspects. The Gaussian mathematical model was applied to the internal thermo‐shear batch mixing torque data to predict processability and plasticization of the resulting compounds. A decrease in the peak torque value with increase in wax content revealed the plasticization effect. Across both types of wax (H5 and 210P) at 4, 8, and 10 phr content, the 10 phr H5 wax plasticized UHMWPE batch showed a minimum peak torque of 4.5 Nm with a broad mixing temperature window (∆T) of 32.4°C. The relatively low‐molecular‐weight wax (H5) seemed to be more efficient at uncoiling the entanglements among the amorphous phase of UHMWPE, perhaps because of its smooth interpenetration between ties. The lowest degree of crystallinity range (34%–40%) and the highest nucleation rate (0.9 min−1) were found with 10 phr H5 wax plasticized UHMWPE. The tensile strength of 30.5 MPa and elastic modulus of 127 MPa, achieved by the addition of H5 wax at 10 phr loading, confirmed the optimum effect of plasticization. The plasticization effect was also established from the loss moduli and Cole–Cole plots (log loss modulus vs. log storage modulus). A smaller number of multiple stress‐induced overshoots and the inclining approach toward the viscous dominant regime with temperature sweep were observed in the case of 10 phr H5 wax composition. Scanning electron microscopy (SEM) analysis revealed disappearance of interfacial crazing and interparticle boundaries progressively with the increase in H5 wax content. Overall, plasticized UHMWPE with 10 phr content of H5 wax showed the optimum processability and properties.

Hermes: Fault-tolerant middleware for blockchain interoperability

Blockchain interoperability reduces the risk of investing in blockchain systems by avoiding vendor lock-in, enabling a new digital economy, and providing migration capabilities. However, seamless interoperability among enterprises requires service providers to comply with different regulations, e.g., data privacy regulations and others that apply to financial services. For supporting interoperability, organizations can connect to each blockchain via a gateway. However, these gateways should be resilient to crashes to maintain a consistent state across ledgers. To realize this vision, we propose Hermes, a fault-tolerant middleware that connects blockchain networks and is based on the Open Digital Asset Protocol (ODAP). Hermes is crash fault-tolerant by allying a new protocol, ODAP-2PC, with a log storage API that can leverage blockchain to secure logs, providing transparency, auditability, availability, and non-repudiation. We briefly explore a use case for cross-jurisdiction asset transfers, illustrating how one can leverage Hermes to support cross-chain transactions compliant with legal and regulatory frameworks.

An Autonomous Log Storage Management Protocol with Blockchain Mechanism and Access Control for the Internet of Things.

As the Internet of Things (IoT) has become prevalent, a massive number of logs produced by IoT devices are transmitted and processed every day. The logs should contain important contents and private information. Moreover, these logs may be used as evidences for forensic investigations when cyber security incidents occur. However, evidence legality and internal security issues in existing works were not properly addressed. This paper proposes an autonomous log storage management protocol with blockchain mechanism and access control for the IoT. Autonomous model allows sensors to encrypt their logs before sending it to gateway and server, so that the logs are not revealed to the public during communication process. Along with blockchain, we introduce the concept “signature chain”. The integration of blockchain and signature chain provides efficient management functions with valuable security properties for the logs, including robust identity verification, data integrity, non-repudiation, data tamper resistance, and the legality. Our work also employs attribute-based encryption to achieve fine-grained access control and data confidentiality. The results of security analysis using AVSIPA toolset, GNY logic and semantic proof indicate that the proposed protocol meets various security requirements. Providing good performance with elliptic curve small key size, short BLS signature, efficient signcryption method, and single sign-on solution, our work is suitable for the IoT.

Shadow-Chain: A Decentralized Storage System for Log Data

Better use of personal health and clinical data from different stakeholders is essential and highly significant. However, the sharing of such data could easily raise concerns over preserving personal privacy, particularly when the risk of disclosure is uncertain. To tackle this issue, shadow- chain, a decentralized storage system for log data based on blockchain, is proposed for realizing data sharing between parties that do not trust each other. In the system, a behavior monitor is designed to intercept the data reading activities. Through an asynchronous read-write mechanism, the monitor can minimize operational impact on the source's storage system. Meanwhile, the homomorphic encryption scheme and the distributed log collection system are adopted to guarantee the validity and reliability of the log data. Furthermore, the hybrid storage strategy is developed to reduce the storage overhead of the log data. In the strategy, only the hash of log data is stored in the blockchain of the shadow-chain, while the data itself and the associated index information are placed in common storage media. Experiments have been conducted to estimate the performance of our shadow-chain. The results show that the impact of shadow-chain on the source storage system can be controlled within 5 percent, and 75 percent log storage overhead can be reduced through hybrid storage strategy.

Optimized forest planning: allocation of log storage yards in the Amazonian sustainable forest management area

In Amazonian native forest management, forest road infrastructures, such as log storage yards and skid trails, are the most expensive attributes and are responsible for the significant environmental impacts with selective tree felling. Road optimization is crucial for reducing environmental impacts and production costs and is strongly linked to the optimal location of storage yards, which are essential to forest road planning. Considering the present problem and the current solutions available, this case study aimed to evaluate the efficiency and eventual gains of optimized forest planning (OFP), as compared to traditional forest planning (TFP). The TFP method is currently used most frequently by forest companies in the Amazon region. The study area of 126.41 ha belongs to the National Forest (NAFO) Saracá-Taquera, Forest Management Unit II (FMU-II), Annual Production Unit (APU) 04/2018, and Work Unit (WU) 2. For the analysis, two areas were defined for exploration: the first was explored using the OFP mathematical model (57.75 ha) and the second followed the TFP plan of the company (68.66 ha). Plans and executions for both areas were compared. The OFP model significantly reduced the Euclidean distances between tree and yards, with only 0.23 km difference in the amount of planned forest roads, when compared to TFP. Additionally, OFP demonstrated a higher productivity (trees.h-1 and m3.h-1), a reduction of skid distance (by an average of 17.16%), and reduced the cost of log skidding (m3 by 25.76%). Thus, this study proved that OFP is a viable solution that can be adopted by companies to increase productivity.

Reducing Honeypot Log Storage Capacity Consumption – Cron Job with Perl-Script Approach

Honeypot is a decoy computer system that is used to attract and monitor hackers’ activities in the network. The honeypot aims to collect information from the hackers in order to create a more secure system. However, the log file generated by honeypot can grow very large when heavy traffic occurred in the system, such as Distributed Denial of Services’ (DDoS) attack. The DDoS possesses difficulty when it is being processed and analyzed by the network administrator as it required a lot of time and resources. Therefore, in this paper, we propose an approach to decrease the log size that is by using a Cron job that will run with a Perl-script. This approach parses the collected data into the database periodically to decrease the log size. Three DDoS attack cases were conducted in this study to show the increasing of the log size by sending a different amount of packet per second for 8 hours in each case. The results have shown that by utilizing the Cron job with Perl-script, the log size has been significantly reduced, the disk space used in the system has also decreased. Consequently, this approach capable of speeding up the process of parsing the log file into the database and thus, improving the overall system performance. This study contributes to providing a pathway in reducing honeypot log storage using the Cron job with Perl-Script.

Определение геометрических характеристик двухъярусной пакетной сплоточной единицы, расположенной на твердом основании

Two-storied package rafting units (TPRU) are essential for implementation of the concept of environmentally friendly forest transport operation of medium and small rivers’ system, which provides commercial accessibility of wood raw materials from the remote forest areas of Russia, and sustainable use of resources. The use of TPRU implies the need of engineering calculations, for instance, to substantiate their dimensions in case of the units formation on a solid basement (at log storage area). Selection of dimensions should be sufficient for the correct assessment of their structural behavior. An additional study, the purpose of which is to obtain dependences for determination of dimensions of TPRU, was proposed to conduct as a result of the literature data analysis. The research method is theoretical. The packages were considered as weightless flexible shells filled up with one type of loose medium. We have validated the capability of use the second order parametric equations of elastics with no inflexion point for describing cross-section of packages. A system of equations that connect the desired dimensions is derived. The analytical solution of the obtained system turned out to be impracticable. A computer algorithm for solving the numerical method was developed and implemented. The calculations were performed by operating with specific characteristics corresponding to the unit area of a separate package. The dependences ofspecific dimensions of TPRU placed on a solid basement on the shape factor of a separately lying package were obtained by approximating the calculation results. Transition to the absolute values of height, width, and length of the rafting units and the working section length of the external strapping is provided by multiplying the specific characteristics by the square root of the package cross-sectional area. An approximating equitation is also obtained for calculating the absolute value of the working part length of the external strapping by the shape factor and the height of a separately lying package. The approximating dependencies,which allow to determine by the shape factor of a detached package the position of the zero pressure surfaces of a loose medium for lower and upper packages, as well as correlation between the heights of the mentioned packages, were proposed to resolve the issues related to the strength characteristics of a rafting unit under the considered conditions. The determination coefficients of the suggested approximating dependencies have values not less than 0.99. The theoretic data reliability is confirmed by the experimental technique applying the physical models.For citation: Posypanov S.V. Dimensional Determination of a Two-Storied Package Rafting Unit Placed on a Solid Basement. Lesnoy Zhurnal [Forestry Journal], 2019, no. 5, pp.135–147. DOI: 10.17238/issn0536-1036.2019.5.135

Evidence-Based Context-Aware Log Data Management for Integrated Monitoring System

Integrated monitoring system, enabled with semi-structured datastore, is a promising solution for monitoring SaaS systems. However, according to increasing scale of SaaS systems and their long-term of service operations, the monitoring system has faced the problem in response times of log analysis and storage consumption. Our empirical observation is that the problem is primarily derived from the unselective log processing of semi-structure datastore, whereas there should be heterogeneities in log data that we can take advantage of for efficient log management. Based on this observation, we first attest this insight by investigating the usage patterns of log data in a quantitative manner with an actual dataset of log access histories obtained from a SaaS system serving to enterprise users, and we show that there are heterogeneities in required retention period of logs, response time, and amount of data, depending on log data category and its analysis scenario. Armed with the evidence found from the investigation, we design a methodology of context-aware log management, key features of which are to speculatively pre-cache the log analysis and to proactively archive ones depending on log data category and analysis scenario. Evaluation with a prototype implementation shows that the proposed method reduces the response time and the storage consumption.

Unsupervised intrusion detection through skip-gram models of network behavior

Detecting intrusions is one of the main objectives of computer security. Attacks have become overly sophisticated over the years in order to remain effective and stealthy. Major breaches are typically perpetrated using techniques that are polymorphic, multi-vector, multi-stage and targeted, that is, adopting forms that were never seen before. Anomaly detection, which does not make any assumption about the shape of a potential attack but instead on legitimate behavior, seems to be a suitable approach in order to defeat sophisticated intrusions. Skip-gram modeling, a word2vec algorithm variant, was leveraged to model systems’ legitimate network behavior. The resulting model was then used to spot intrusions in a test dataset. The optimal configuration led to 99.20% precision, 82.07% recall, and 91.02% accuracy, with a false positive rate of 0.61%, which is significantly lower than most state-of-the-art methods. These metrics were achieved under a fully unsupervised setting, that is, without any prior knowledge of what constitutes an attack. Furthermore, the approach provides benefits in terms of interpretability and log storage requirements, as it requires a small amount of input features. It also produces information about systems behavior and their relationships, that can be reused by other analysis techniques to obtain further insights.

Soil moisture variation on the outlet area of devices dewatering forest roads in Beskid Sądecki

Abstract Water run-off - a natural part of water circulation in the natural environment - is a harmful and undesired phenomenon for humans. The most spectacular and most serious one is its erosive activity. In adverse conditions, such a runoff has high energy, it causes serious reshaping of a terrain and it destroys the natural environment and infrastructure. In the forest environment, a run-off process occurs basically on roads, skidding trails and in log storage sites, which are areas with soil cover properties changed by humans. The devices used for dehydrating a transport system direct water collected in the form of concentrated streams to a slope, which frequently causes serious linear damage or landslides. The objective of the tests carried out was to determine the spatial distribution of soil moisture on the area behind the outlet of devices dehydrating forest roads within the context of changes in selected soil properties affecting a risk of landslide formation. The tests included four water drains dehydrating a forest slope road with a soil surface. In the area located behind the outlet, according to the model accepted, soil samples were collected for testing a current moisture value, determining grain composition and a hydraulic conductivity coefficient, calculated based on the simplified Hazen formula. There were significant differences in the soil moisture means in the three-measurement series conducted; nevertheless, the highest moisture means were obtained after the lowest precipitation, and the lowest moisture means were obtained after the highest precipitation. It applies to all water drains observed, separately and collectively. Average moisture measures in all dehydrating facilities differed among one another, although in different sessions, this dependence was observed in various facility pairs. The statistical analysis did not demonstrate any significant differences in moisture in terms of the distance of moisture testing location from the road edge, or in locations with different distances from the water run-off line behind the outlet of dehydrating devices. Significant differences could be observed in the analysis of average moisture values in the soil layers. Such an analysis result could be explained by a relatively low soil permeability and low slope gradient on the area examined. Usually, a few percent fluctuation in soil moisture, despite precipitation diversity, in combination with a high amount of granular soil structure, does not create conditions facilitating the formation of landslides.