Linux Environments Research Articles

A tool for integrating agrometeorological observation data for digital agriculture: A Minnesota case study

AbstractAgrometeorological data are essential for understanding production using digital agriculture techniques. However, integrating agrometerological observations from multiple sources remains a challenge. Often, digital agriculture scientists download and clean the same datasets many times. We present a prototype system that simplifies the process of collecting, cleaning, integrating, and aggregating data from meteorological data sources by providing a simplified user interface, database, and application programming interface. The prototype provides a standard interface for querying multiple geospatial formats (raster and vector) and integrates observation networks including the National Oceanic and Atmospheric Administration Global Historical Climatology Network (NOAA GHCN), NOAA NClim‐Grid (NOAA's Gridded Climate Normals), and Ameriflux BASE. The system automatically checks and updates data, saving storage space and processing time, and allows users to summarize data spatially and temporally. Provided as open source code and browser‐based user interface, the application and integration system can be run across Windows, Linux, and Mac environments to support broader use of multi‐source agrometeorology data.

ROS Gateway: Enhancing ROS Availability across Multiple Network Environments.

As the adoption of large-scale model-based AI grows, the field of robotics is undergoing significant changes. The emergence of cloud robotics, where advanced tasks are offloaded to fog or cloud servers, is gaining attention. However, the widely used Robot Operating System (ROS) does not support communication between robot software across different networks. This paper introduces ROS Gateway, a middleware designed to improve the usability and extend the communication range of ROS in multi-network environments, which is important for processing sensor data in cloud robotics. We detail its structure, protocols, and algorithms, highlighting improvements over traditional ROS configurations. The ROS Gateway efficiently handles high-volume data from advanced sensors such as depth cameras and LiDAR, ensuring reliable transmission. Based on the rosbridge protocol and implemented in Python 3, ROS Gateway is compatible with rosbridge-based tools and runs on both x86 and ARM-based Linux environments. Our experiments show that the ROS Gateway significantly improves performance metrics such as topic rate and delay compared to standard ROS setups. We also provide predictive formulas for topic receive rates to guide the design and deployment of robotic applications using ROS Gateway, supporting performance estimation and system optimization. These enhancements are essential for developing responsive and intelligent robotic systems in dynamic environments.

Managing Linux servers with LLM-based AI agents: An empirical evaluation with GPT4

This paper presents an empirical study on the application of Large Language Model (LLM)-based AI agents for automating server management tasks in Linux environments. We aim to evaluate the effectiveness, efficiency, and adaptability of LLM-based AI agents in handling a wide range of server management tasks, and to identify the potential benefits and challenges of employing such agents in real-world scenarios. We present an empirical study where a GPT-based AI agent autonomously executes 150 unique tasks across 9 categories, ranging from file management to editing to program compilations. The agent operates in a Dockerized Linux sandbox, interpreting task descriptions and generating appropriate commands or scripts. Our findings reveal the agent’s proficiency in executing tasks autonomously and adapting to feedback, demonstrating the potential of LLMs in simplifying complex server management for users with varying technical expertise. This study contributes to the understanding of LLM applications in server management scenarios, and paves the foundation for future research in this domain.

A practical approach for finding anti-debugging routines in the Arm-Linux using hardware tracing

As IoT devices are being widely used, malicious code is increasingly appearing in Linux environments. Sophisticated Linux malware employs various evasive techniques to deter analysis. The embedded trace microcell (ETM) supported by modern Arm CPUs is a suitable hardware tracer for analyzing evasive malware because it is almost artifact-free and has negligible overhead. In this paper, we present an efficient method to automatically find debugger-detection routines using the ETM hardware tracer. The proposed scheme reconstructs the execution flow of the compiled binary code from ETM trace data. In addition, it automatically identifies and patches the debugger-detection routine by comparing two traces (with and without the debugger). The proposed method was implemented using the Ghidra plug-in program, which is one of the most widely used disassemblers. To verify its effectiveness, 15 debugger-detection techniques were investigated in the Arm-Linux environment to determine whether they could be detected. We also confirmed that our implementation works successfully for the popular malicious Mirai malware in Linux. Experiments were further conducted on 423 malware samples collected from the Internet, demonstrating that our implementation works well for real malware samples.

SEDA 2024 update: enhancing the SEquence DAtaset builder for seamless integration into automated data analysis pipelines.

The initial version of SEDA assists life science researchers without programming skills with the preparation of DNA and protein sequence FASTA files for multiple bioinformatics applications. However, the initial version of SEDA lacks a command-line interface for more advanced users and does not allow the creation of automated analysis pipelines. The present paper discusses the updates of the new SEDA release, including the addition of a complete command-line interface, new functionalities like gene annotation, a framework for automated pipelines, and improved integration in Linux environments. SEDA is an open-source Java application and can be installed using the different distributions available ( https://www.sing-group.org/seda/download.html ) as well as through a Docker image ( https://hub.docker.com/r/pegi3s/seda ). It is released under a GPL-3.0 license, and its source code is publicly accessible on GitHub ( https://github.com/sing-group/seda ). The software version at the time of submission is archived at Zenodo (version v1.6.0, http://doi.org/10.5281/zenodo.10201605 ).

SCExecute: custom cell barcode-stratified analyses of scRNA-seq data.

In single-cell RNA-sequencing (scRNA-seq) data, stratification of sequencing reads by cellular barcode is necessary to study cell-specific features. However, apart from gene expression, the analyses of cell-specific features are not sufficiently supported by available tools designed for high-throughput sequencing data. We introduce SCExecute, which executes a user-provided command on barcode-stratified, extracted on-the-fly, single-cell binary alignment map (scBAM) files. SCExecute extracts the alignments with each cell barcode from aligned, pooled single-cell sequencing data. Simple commands, monolithic programs, multi-command shell scripts or complex shell-based pipelines are then executed on each scBAM file. scBAM files can be restricted to specific barcodes and/or genomic regions of interest. We demonstrate SCExecute with two popular variant callers-GATK and Strelka2-executed in shell-scripts together with commands for BAM file manipulation and variant filtering, to detect single-cell-specific expressed single nucleotide variants from droplet scRNA-seq data (10X Genomics Chromium System).In conclusion, SCExecute facilitates custom cell-level analyses on barcoded scRNA-seq data using currently available tools and provides an effective solution for studying low (cellular) frequency transcriptome features. SCExecute is implemented in Python3 using the Pysam package and distributed for Linux, MacOS and Python environments from https://horvathlab.github.io/NGS/SCExecute. Supplementary data are available at Bioinformatics online.

Digital forensic analysis methodology for private browsing: Firefox and Chrome on Linux as a case study

The web browser has become one of the basic tools of everyday life. A tool that is increasingly used to manage personal information. This has led to the introduction of new privacy options by the browsers, including private mode. In this paper, a methodology to explore the effectiveness of the private mode included in most browsers is proposed. A browsing session was designed and conducted in Mozilla Firefox and Google Chrome running on four different Linux environments. After analyzing the information written to disk and the information available in memory, it can be observed that Firefox and Chrome did not store any browsing-related information on the hard disk. However, memory analysis reveals that a large amount of information could be retrieved in some of the environments tested. For example, for the case where the browsers were executed in a VMware virtual machine, it was possible to retrieve most of the actions performed, from the keywords entered in a search field to the username and password entered to log in to a website, even after restarting the computer. In contrast, when Firefox was run on a slightly hardened non-virtualized Linux, it was not possible to retrieve any browsing-related artifacts after the browser was closed.

Kernel-level tracing for detecting stegomalware and covert channels in Linux environments

Modern malware is becoming hard to spot since attackers are increasingly adopting new techniques to elude signature- and rule-based detection mechanisms. Among the others, steganography and information hiding can be used to bypass security frameworks searching for suspicious communications between processes or exfiltration attempts through covert channels. Since the array of potential carriers is very large (e.g., information can be hidden in hardware resources, various multimedia files or network flows), detecting this class of threats is a scarcely generalizable process and gathering multiple behavioral information is time-consuming, lacks scalability, and could lead to performance degradation.In this paper, we leverage the extended Berkeley Packet Filter (eBPF), which is a recent code augmentation feature provided by the Linux kernel, for programmatically tracing and monitoring the behavior of software processes in a very efficient way. To prove the flexibility of the approach, we investigate two realistic use cases implementing different attack mechanisms, i.e., two processes colluding via the alteration of the file system and hidden network communication attempts nested within IPv6 traffic flows. Our results show that even simple eBPF programs can provide useful data for the detection of anomalies, with a minimal overhead. Furthermore, the flexibility to develop and run such programs allows to extract relevant features that could be used for the creation of datasets for feeding security frameworks exploiting AI.

MMALE—A Methodology for Malware Analysis in Linux Environments

MMALE—A Methodology for Malware Analysis in Linux Environments

Impact analysis of SYN flood DDOS attack on HAPROXY and NLB cluster-base web servers

<p>In recent, the high available internet service is main demand of the most people. However, online services occasionally become inaccessible due to various threats and attacks. Synchronization (SYN) flood Distributed Denial of Service (DDoS) is the most used and has a serious effect on the public network services. Hence, the outcome of this attack on the commonly utilized cluster-based web servers is systematically illustrated in this paper. Moreover, performance of Internet Information Service 10.0 (IIS 10.0) on Windows server 2016 and Apache 2 on Linux Ubuntu 16.04 server is evaluated efficiently. The performance measuring process is done on both Network Load Balancing (NLB) and High Available Proxy (HAProxy) in Windows and Linux environments respectively as methods for web server load balancing. Furthermore, stability, efficiency and responsiveness of the web servers are depended as the study evaluation metrics. Additionally, average CPU usage and throughput of the both mechanisms are measured in the proposed system. The results show that the IIS 10.0 cluster-based web servers are more responsiveness, efficiency and stable with and without SYN flood DDoS attack. Also, the performance of IIS 10.0 web server is better than of the Apache 2 in term of the average CPU usage and throughput.</p>

Diving in uncharted waters: An updated genetics toolkit highlights the challenges of polyploidy in landscape genomics analyses.

As molecular ecologists, we have by necessity become adept at working across computational platforms. A diverse community of scientists has developed a broad array of analytical resources spanning command line to graphical user interface across Linux, Mac, and Windows environments and a dizzying array of program-specific input formats. In light of this, we often explore our data like free divers - filling our lungs with air and descending for a short period of time into one part of our data set before resurfacing, reformatting, and preparing for our next analysis. In this issue of Molecular Ecology Resources, Meirmans (2020) presents an updated version of GenoDive, a program with a toolkit that provides users with the opportunity to stay a while and delve deeper into the diverse portfolio of information provided by a genomic data set. The comprehensive nature of GenoDive coupled with its unique capability to handle both diploid and polyploid data also provides an opportunity to reflect on the unevenness of resources available for the analysis of polyploid versus diploid data. Since new updates include the addition of plug-ins for genotype-environment association analyses, we limit the observations presented here to the common tools used for landscape genomics analyses.

Platform-Independent Malware Analysis Applicable to Windows and Linux Environments

Most cyberattacks use malicious codes, and according to AV-TEST, more than 1 billion malicious codes are expected to emerge in 2020. Although such malicious codes have been widely seen around the PC environment, they have been on the rise recently, focusing on IoT devices such as smartphones, refrigerators, irons, and various sensors. As is known, Linux/embedded environments support various architectures, so it is difficult to identify the architecture in which malware operates when analyzing malware. This paper proposes an AI-based malware analysis technology that is not affected by the operating system or architecture platform. The proposed technology works intuitively. It uses platform-independent binary data rather than features based on the structured format of the executable files. We analyzed the strings from binary data to classify malware. The experimental results achieved 94% accuracy on Windows and Linux datasets. Based on this, we expect the proposed technology to work effectively on other platforms and improve through continuous operation/verification.

Software-Based Video–Audio Production Mixer via an IP Network

Modern television production has promoted the simultaneous use of several cameras and sound sources, which increases the complexity and costs of broadcasting studios. This paper describes the design and implementation of a video-audio production mixer via an IP network. It is presented as a potential replacement for traditional professional production systems on the basis of cost reduction, as it is a software-based system that uses the existing technologies and can be built in community television stations and economic private productions. A prototype combining five cameras, a title generator, a multimedia player, microphone sound, music, and other resources for video recording or Internet live streaming has been implemented. The system also features a voice intercommunication capability to support teamwork. This technology has been mounted using different high-definition cameras with High-Definition Multimedia Interface (HDMI) outputs, desktop computers, mobile phones and other non-dedicated equipment available for free or at a low cost. Although it works on non-dedicated hardware, this system provides video routing, sound managing, and audiovisual mixing with an approximate total delay of only 1.4 s. It has been mounted mostly on Linux environments to guarantee reliability and the extensive use of free software, which demonstrates the feasibility of building a cost-effective video-audio production mixer, using the available devices and techniques.

G2P: a Genome-Wide-Association-Study simulation tool for genotype simulation, phenotype simulation and power evaluation.

Plenty of Genome-Wide-Association-Study (GWAS) methods have been developed for mapping genetic markers that associated with human diseases and agricultural economic traits. Computer simulation is a nice tool to test the performances of various GWAS methods under certain scenarios. Existing tools are either inefficient in terms of computation and memory efficiency or inconvenient to use to simulate big, realistic genotype data and phenotype data to evaluate available GWAS methods. Here, we present a GWAS simulation tool named G2P that can be used to simulate genotype data, phenotype data and perform power evaluation of GWAS methods. G2P is a user-friendly tool with all functions is provided in both graphical user interface and pipeline manners and it is available for Windows, Mac and Linux environments. Furthermore, G2P achieves maximum efficiency in terms of both memory usage and simulation speed; with G2P, the simulation of genotype data that includes 1 000 000 samples and 2 000 000 markers can be accomplished in 5 h. The G2P software, user manual, and example datasets are freely available at GitHub: https://github.com/XiaoleiLiuBio/G2P. Supplementary data are available at Bioinformatics online.

Experiences on Evaluating Network Simulators: A Methodological Approach

There exists a variety of network simulators, used to imitate the protocols, nodes, and connections in data networks. They differ in their design, goals, and characteristics. Thus, comparing simulators requires a clear and standardized methodology. In this paper, based on a set of measurable and comparable criteria, we propose an approach to evaluate them. We validate the suggested approach with two network simulators, namely Packet Tracer and GNS3. In that regard, a test scenario is put forward on the two simulators, both in Linux and Windows environments, and their performance is monitored based on the suggested approach. This paper does not propose a method for selecting the best simulator, but it rather supplies the researchers with an evaluation tool, that can be used to describe, compare, and select the most suitable network simulators for a given scenario.

Integrating Agile Methodologies with DevOps Practices in Linux Environments: A Comparative Study

Integrating Agile Methodologies with DevOps Practices in Linux Environments: A Comparative Study

OpenDrift v1.0: a generic framework for trajectory modelling

Abstract. OpenDrift is an open-source Python-based framework for Lagrangian particle modelling under development at the Norwegian Meteorological Institute with contributions from the wider scientific community. The framework is highly generic and modular, and is designed to be used for any type of drift calculations in the ocean or atmosphere. A specific module within the OpenDrift framework corresponds to a Lagrangian particle model in the traditional sense. A number of modules have already been developed, including an oil drift module, a stochastic search-and-rescue module, a pelagic egg module, and a basic module for atmospheric drift. The framework allows for the ingestion of an unspecified number of forcing fields (scalar and vectorial) from various sources, including Eulerian ocean, atmosphere and wave models, but also measurements or a priori values for the same variables. A basic backtracking mechanism is inherent, using sign reversal of the total displacement vector and negative time stepping. OpenDrift is fast and simple to set up and use on Linux, Mac and Windows environments, and can be used with minimal or no Python experience. It is designed for flexibility, and researchers may easily adapt or write modules for their specific purpose. OpenDrift is also designed for performance, and simulations with millions of particles may be performed on a laptop. Further, OpenDrift is designed for robustness and is in daily operational use for emergency preparedness modelling (oil drift, search and rescue, and drifting ships) at the Norwegian Meteorological Institute.

Design of the IBM z14 microprocessor

The latest-generation IBM Z processor provides enhanced performance and compute capacity compared to its IBM z13 predecessor. This paper describes some of the major improvements that include an additional perceptron branch predictor, a completely redesigned translation engine that is tightly integrated into the core pipeline, and an integrated level-1 cache directory and translation lookaside buffer design. Outside of the central processing unit (CPU), the cache sizes have increased on each cache level, and each processor chip now contains 10 CPUs. The system topology has been optimized to improve cache transfer latencies for workloads spanning multiple processor chips. The bus interfaces between the chips have been redesigned to improve peak bus traffic handling. In combination, these enhancements provide significant performance improvements in traditional data-serving workloads, as well as in virtualized Linux environments running database, analytic, and cognitive workloads.

Investigation of the spatial structure and interactions of the genome at sub-kilobase-pair resolution using T2C.

Chromosome conformation capture (3C) and its derivatives (e.g., 4C, 5C and Hi-C) are used to analyze the 3D organization of genomes. We recently developed targeted chromatin capture (T2C), an inexpensive method for studying the 3D organization of genomes, interactomes and structural changes associated with gene regulation, the cell cycle, and cell survival and development. Here, we present the protocol for T2C based on capture, describing all experimental steps and bio-informatic tools in full detail. T2C offers high resolution, a large dynamic interaction frequency range and a high signal-to-noise ratio. Its resolution is determined by the resulting fragment size of the chosen restriction enzyme, which can lead to sub-kilobase-pair resolution. T2C's high coverage allows the identification of the interactome of each individual DNA fragment, which makes binning of reads (often used in other methods) basically unnecessary. Notably, T2C requires low sequencing efforts. T2C also allows multiplexing of samples for the direct comparison of multiple samples. It can be used to study topologically associating domains (TADs), determining their position, shape, boundaries, and intra- and inter-domain interactions, as well as the composition of aggregated loops, interactions between nucleosomes, individual transcription factor binding sites, and promoters and enhancers. T2C can be performed by any investigator with basic skills in molecular biology techniques in ∼7-8 d. Data analysis requires basic expertise in bioinformatics and in Linux and Python environments.

Typing and Characterization of Bacteria Using Bottom-up Tandem Mass Spectrometry Proteomics

Methods for rapid and reliable microbial identification are essential in modern healthcare. The ability to detect and correctly identify pathogenic species and their resistance phenotype is necessary for accurate diagnosis and efficient treatment of infectious diseases. Bottom-up tandem mass spectrometry (MS) proteomics enables rapid characterization of large parts of the expressed genes of microorganisms. However, the generated data are highly fragmented, making downstream analyses complex. Here we present TCUP, a new computational method for typing and characterizing bacteria using proteomics data from bottom-up tandem MS. TCUP compares the generated protein sequence data to reference databases and automatically finds peptides suitable for characterization of taxonomic composition and identification of expressed antimicrobial resistance genes. TCUP was evaluated using several clinically relevant bacterial species (Escherichia coli, Pseudomonas aeruginosa, Staphylococcus aureus, Streptococcus pneumoniae, Moraxella catarrhalis, and Haemophilus influenzae), using both simulated data generated by in silico peptide digestion and experimental proteomics data generated by liquid chromatography-tandem mass spectrometry (MS/MS). The results showed that TCUP performs correct peptide classifications at rates between 90.3 and 98.5% at the species level. The method was also able to estimate the relative abundances of individual species in mixed cultures. Furthermore, TCUP could identify expressed β-lactamases in an extended spectrum β-lactamase-producing (ESBL) E. coli strain, even when the strain was cultivated in the absence of antibiotics. Finally, TCUP is computationally efficient, easy to integrate in existing bioinformatics workflows, and freely available under an open source license for both Windows and Linux environments.