Code Reorganization Research Articles

Less is more: Minimizing code reorganization using XTREE

Context: Developers use bad code smells to guide code reorganization. Yet developers, textbooks, tools, and researchers disagree on which bad smells are important. How can we offer reliable advice to developers about which bad smells to fix?Objective: To evaluate the likelihood that a code reorganization to address bad code smells will yield improvement in the defect-proneness of the code.Method: We introduce XTREE, a framework that analyzes a historical log of defects seen previously in the code and generates a set of useful code changes. Any bad smell that requires changes outside of that set can be deprioritized (since there is no historical evidence that the bad smell causes any problems).Evaluation: We evaluate XTREE’s recommendations for bad smell improvement against recommendations from previous work (Shatnawi, Alves, and Borges) using multiple data sets of code metrics and defect counts.Results: Code modules that are changed in response to XTREE’s recommendations contain significantly fewer defects than recommendations from previous studies. Further, XTREE endorses changes to very few code metrics, so XTREE requires programmers to do less work. Further, XTREE’s recommendations are more responsive to the particulars of different data sets. Finally XTREE’s recommendations may be generalized to identify the most crucial factors affecting multiple datasets (see the last figure in paper).Conclusion: Before undertaking a code reorganization based on a bad smell report, use a framework like XTREE to check and ignore any such operations that are useless; i.e. ones which lack evidence in the historical record that it is useful to make that change. Note that this use case applies to both manual code reorganizations proposed by developers as well as those conducted by automatic methods.

Understanding modern device drivers

Device drivers are the single largest contributor to operating-system kernel code with over 5 million lines of code in the Linux kernel, and cause significant complexity, bugs and development costs. Recent years have seen a flurry of research aimed at improving the reliability and simplifying the development of drivers. However, little is known about what constitutes this huge body of code beyond the small set of drivers used for research. In this paper, we study the source code of Linux drivers to understand what drivers actually do, how current research applies to them and what opportunities exist for future research. We determine whether assumptions made by most driver research, such as that all drivers belong to a class, are indeed true. We also analyze driver code and abstractions to determine whether drivers can benefit from code re-organization or hardware trends. We develop a set of static-analysis tools to analyze driver code across various axes. Broadly, our study looks at three aspects of driver code (i) what are the characteristics of driver code functionality and how applicable is driver research to all drivers, (ii) how do drivers interact with the kernel, devices, and buses, and (iii) are there similarities that can be abstracted into libraries to reduce driver size and complexity? We find that many assumptions made by driver research do not apply to all drivers. At least 44% of drivers have code that is not captured by a class definition, 28% of drivers support more than one device per driver, and 15% of drivers do significant computation over data. From the driver interactions study, we find USB bus offers an efficient bus interface with significant standardized code and coarse-grained access, ideal for executing drivers in isolation. We also find that drivers for different buses and classes have widely varying levels of device interaction, which indicates that the cost of isolation will vary by class. Finally, from our driver similarity study, we find 8% of all driver code is substantially similar to code elsewhere and may be removed with new abstractions or libraries.

Understanding modern device drivers

Device drivers are the single largest contributor to operating-system kernel code with over 5 million lines of code in the Linux kernel, and cause significant complexity, bugs and development costs. Recent years have seen a flurry of research aimed at improving the reliability and simplifying the development of drivers. However, little is known about what constitutes this huge body of code beyond the small set of drivers used for research. In this paper, we study the source code of Linux drivers to understand what drivers actually do, how current research applies to them and what opportunities exist for future research. We determine whether assumptions made by most driver research, such as that all drivers belong to a class, are indeed true. We also analyze driver code and abstractions to determine whether drivers can benefit from code re-organization or hardware trends. We develop a set of static-analysis tools to analyze driver code across various axes. Broadly, our study looks at three aspects of driver code (i) what are the characteristics of driver code functionality and how applicable is driver research to all drivers, (ii) how do drivers interact with the kernel, devices, and buses, and (iii) are there similarities that can be abstracted into libraries to reduce driver size and complexity? We find that many assumptions made by driver research do not apply to all drivers. At least 44% of drivers have code that is not captured by a class definition, 28% of drivers support more than one device per driver, and 15% of drivers do significant computation over data. From the driver interactions study, we find USB bus offers an efficient bus interface with significant standardized code and coarse-grained access, ideal for executing drivers in isolation. We also find that drivers for different buses and classes have widely varying levels of device interaction, which indicates that the cost of isolation will vary by class. Finally, from our driver similarity study, we find 8% of all driver code is substantially similar to code elsewhere and may be removed with new abstractions or libraries.

Vectorizing the Community Land Model

In this paper we describe our extensive efforts to rewrite the Community Land Model (CLM) so that it provides good vector performance on the Earth Simulator in Japan and the Cray X1 at Oak Ridge National Laboratory. We present the technical details of the old and new internal data structures, the required code reorganization, and the resulting performance improvements. We describe and compare the performance and scaling of the final CLM Version 3.0 (CLM3.0) on the IBM Power4, the Earth Simulator, and the Cray X1. At 64 processors, the performance of the model is similar on the IBM Power4, the Earth Simulator, and the Cray X1. However, the Cray X1 offers the best performance of all three platforms tested from 4 to 64 processors when OpenMP is used. Moreover, at low processor counts (16 or fewer), the model performs significantly better on the Cray X1 than on the other platforms. The vectorized version of CLM was publicly released by the National Center for Atmospheric Research as the standalone CLM3.0, as a part of the new Community Atmosphere Model Version 3.0 (CAM3.0), and as a component of the Community Climate System Model Version 3.0 (CCSM3.0) on June 23, 2004.

Efficient instruction scheduling for a pipelined architecture

As part of an effort to develop an optimizing compiler for a pipelined architecture, a code reorganization algorithm has been developed that significantly reduces the number of runtime pipeline interlocks. In a pass after code generation, the algorithm uses a dag representation to heuristically schedule the instructions in each basic block.Previous algorithms for reducing pipeline interlocks have had worst-case runtimes of at least O ( n 4 ). By using a dag representation which prevents scheduling deadlocks and a selection method that requires no lookahead, the resulting algorithm reorganizes instructions almost as effectively in practice, while having an O ( n 2 ) worst-case runtime.

Instruction cache performance of a commercial workload on the Motorola 88110 microprocessor

Commercial workloads, such as transaction processing, stress the memory hierarchies of current computers. TPC-B is a transaction processing benchmark that generates significant operating system and database activity, and a TPC-B instruction trace is analyzed for the Motorola 88110 microprocessor. Instruction set usage reveals one branch in every four to five instructions, and instruction cache performance indicates that this branching behavior causes a large number of words to be brought in by cache refills but never used. Code reorganization optimizations to reduce unused words and avoid cache misses are simulated and found to reduce misses in the on-chip instruction cache by almost one fifth.

Optimization of particle-in-cell codes on reduced instruction set computer processors

General strategies are developed to optimize particle‐cell codes written in Fortran for reduced instruction set computer processors which are commonly used on massively parallel computers. These strategies include data reorganization to improve cache utilization and code reorganization to improve efficiency of arithmetic pipelines. Results show performance improvements of 1.4–3.4 times can be achieved. © 1996 American Institute of Physics.

Available instruction-level parallelism for superscalar and superpipelined machines

Superscalar machines can issue several instructions per cycle. Superpipelined machines can issue only one instruction per cycle, but they have cycle times shorter than the latency of any functional unit. In this paper these two techniques are shown to be roughly equivalent ways of exploiting instruction-level parallelism. A parameterizable code reorganization and simulation system was developed and used to measure instruction-level parallelism for a series of benchmarks. Results of these simulations in the presence of various compiler optimizations are presented. The average degree of superpipelining metric is introduced. Our simulations suggest that this metric is already high for many machines. These machines already exploit all of the instruction-level parallelism available in many non-numeric applications, even without parallel instruction issue or higher degrees of pipelining.

Efficient instruction scheduling for a pipelined architecture

As part of an effort to develop an optimizing compiler for a pipelined architecture, a code reorganization algorithm has been developed that significantly reduces the number of runtime pipeline interlocks. In a pass after code generation, the algorithm uses a dag representation to heuristically schedule the instructions in each basic block. Previous algorithms for reducing pipeline interlocks have had worst-case runtimes of at least O ( n 4 ). By using a dag representation which prevents scheduling deadlocks and a selection method that requires no lookahead, the resulting algorithm reorganizes instructions almost as effectively in practice, while having an O ( n 2 ) worst-case runtime.

Postpass Code Optimization of Pipeline Constraints

Pipeline interlocks are used in a pipelined architecture to prevent the execution of a machine instruction before its operands are available. An alternative to this complex piece of hardware is to rearrange the instructions at compile time to avoid pipeline interlocks. This problem is called code reorganization and is studied. The basic problem of reorganization of machine-level instructions at compile time is shown to be np-complete. A heuristic algorithm is proposed, and its properties and effectiveness are explored. Empirical data from MIPs, a VLSI processor design, are given. The impact of code reorganization techniques on the rest of a compiler system is discussed. 30 references.