Inter-module Dependencies Research Articles

Performance Analysis and Optimization of Distributed Workflows in Heterogeneous Network Environments

Large-scale e-science features complex DAG-structured workflows comprised of computing modules with intricate inter-module dependencies. Mapping such workflows in heterogeneous network environments and optimizing their end-to-end performance are crucial to the success of scientific collaborations that require fast system response and smooth data flow. We construct analytical cost models and formulate workflow mapping as optimization problems for minimum end-to-end delay and maximum frame rate. The difficulty of these problems essentially arises from the topological matching nature in the spatial domain, which is further compounded by the resource sharing complicacy in the temporal dimension. For unitary processing applications, we develop a workflow mapping algorithm based on a recursive critical path optimization procedure to minimize the latency; while for streaming applications, we conduct a rigorous workflow stability analysis and develop a layer-oriented dynamic programming solution based on topological sorting to identify and minimize the global bottleneck time. The accuracy of the proposed exact delay calculation algorithm is verified in comparison with an approximate solution, a dynamic distributed system simulation program, and a real network deployment, and the performance superiority of the proposed mapping approaches are illustrated by extensive simulation-based comparisons with existing algorithms and verified by large-scale experiments on real-life scientific workflows through effective system implementation and deployment in real networks.

An Empirical Study of RefactoringChallenges and Benefits at Microsoft

It is widely believed that refactoring improves software quality and developer productivity. However, few empirical studies quantitatively assess refactoring benefits or investigate developers’ perception towards these benefits. This paper presents a field study of refactoring benefits and challenges at Microsoft through three complementary study methods: a survey, semi-structured interviews with professional software engineers, and quantitative analysis of version history data. Our survey finds that the refactoring definition in practice is not confined to a rigorous definition of semantics-preserving code transformations and that developers perceive that refactoring involves substantial cost and risks. We also report on interviews with a designated refactoring team that has led a multi-year, centralized effort on refactoring Windows. The quantitative analysis of Windows 7 version history finds the top 5 percent of preferentially refactored modules experience higher reduction in the number of inter-module dependencies and several complexity measures but increase size more than the bottom 95 percent. This indicates that measuring the impact of refactoring requires multi-dimensional assessment.

SURVEY ON SOFTWARE REMODULARIZATION TECHNIQUES

Re-modularization of software systems is a key tech nique used in testing and maintenance phase of the software development lifecycle. This process helps the developer in revi ewing the modules and the interaction between those modules of the developed software systems. Re-modularization process also al lows the developer to make the software system more efficient by adding additional features according to their future requi rement and enhancement. Based on the review results , the developed software system can be re-modularized, thus finally making t he software system to be efficient and less fault-p rone. This paper deals with the survey on various methods that are used for re- modularization of software systems. The methods inc lude hierarchical clustering method, automated clustering methods and clustering using Genetic Algorithm (GA).The usage of hierarchical clustering helps in understanding the software doma in, software system and the measures used for clust ering the software system. Automatic clustering approaches are used in order t o group together highly cohesive components, in a m odule of the software system where the cohesiveness is measured in terms of intra-module links and reduce the coupling betwe en modules which is measured in terms of the inter-module dependencies of the software system. Genetic Algorithm belongs t o large class of Evolutionary Algorithm, which generates the solutio ns to optimization problems using techniques inspir ed by natural evolution such as inheritance mutation, crossover and selecti on. In general, Genetic Algorithm requires the gene tic representation of the solution domain and the fitness function to evaluat e the solution domain. In this survey the working p rinciple of various remodularization methods are discussed and the effici ent re-modularization approach is selected based on its features and advantages.

A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids

Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.

Modularity and incremental innovation: the roles of design rules and organizational communication

As little attention has been paid to the relationship between modularity and near decomposability, extant studies have not unveiled the impact of modularity on incremental innovation completely. We argue that the modular structure is a special case of nearly decomposable structure, in which the interdependencies between modules are specified by design rules, and the degree of modularity is defined by the level of near decomposability and the extent to which intermodule dependencies are specified. The results of our simulation experiments show that in the term of near decomposability, the increase of modularity leads to higher innovation advantage in the short term, but effective communication between modules can help systems with moderate and low modularity gain more innovation benefits in the long term; in the aspect of design rules, modularization may restrict the search space of the incremental innovation within each module, but under some conditions the option value of modularity may offset or even exceed the restriction effect of design rules.

Metrics for Measuring the Quality of Modularization of Large-Scale Object-Oriented Software

The metrics formulated to date for characterizing the modularization quality of object-oriented software have considered module and class to be synonymous concepts. But a typical class in object oriented programming exists at too low a level of granularity in large object-oriented software consisting of millions of lines of code. A typical module (sometimes referred to as a superpackage) in a large object-oriented software system will typically consist of a large number of classes. Even when the access discipline encoded in each class makes for clean class-level partitioning of the code, the intermodule dependencies created by associational, inheritance-based, and method invocations may still make it difficult to maintain and extend the software. The goal of this paper is to provide a set of metrics that characterize large object-oriented software systems with regard to such dependencies. Our metrics characterize the quality of modularization with respect to the APIs of the modules, on the one hand, and, on the other, with respect to such object-oriented inter-module dependencies as caused by inheritance, associational relationships, state access violations, fragile base-class design, etc. Using a two-pronged approach, we validate the metrics by applying them to popular open-source software systems.

The complexity measure of program based on the inter-module dependency

The complexity of the program is one of the useful measures in the objective and quantitative evaluation of the software. However, in the past measures, the complexity is evaluated based only on a single factor, such as program size, flow of control or dataflow information. This paper aims at solution of the problems in the past measures and proposes a new measure for the complexity of the program, called MCM (module complexity metrics), which is obtained as a combination of the intermodule dependencies. The feature of the proposed measure MCM is that various factors affecting the complexity of the module can be evaluated in an integrated way. Then the correlation between the proposed measure MCM and the conventional typical measures are analyzed, and it is shown that the complexity of the program can be evaluated more accurately by the proposed measure MCM, while maintaining the properties owned by the past measures.

Efficient recompilation of module interfaces in a software development environment

This paper presents global interface analysis algorithms that analyze and limit the effects of an editing change to a basic interface of a software system. The algorithms improve on the deficiencies of the traditional compilation rule found in strongly-typed, separately-compiled programming languages, which often forces the recompilation of modules that are not at all affected by a change to a basic interface. The algorithms assume a software development environment that provides efficient access to the compilation dependencies and the module interfaces of the components being implemented. The algorithms are designed to operate on recursive compilation dependencies since separate compilation of recursive inter-module dependencies can easily be implemented in such an environment.

The Multics kernel design project

We describe a plan to create an auditable version of Multics. The engineering experiments of that plan are now complete. Type extension as a design discipline has been demonstrated feasible, even for the internal workings of an operating system, where many subtle intermodule dependencies were discovered and controlled. Insight was gained into several tradeoffs between kernel complexity and user semantics. The performance and size effects of this work are encouraging. We conclude that verifiable operating system kernels may someday be feasible.