A framework for evaluating inherent biases within ship design tools

Abstract

In early-stage design, critical decisions must be made within a limited information environment. Designers conduct and interpret analyses, taking into account any associated decision risks, with the hopes of making meaningful design trade-offs. To aid in this, synthesis tools are often utilized to generate potential solutions to enable one to characterize the desired design space. While previous research has demonstrated that solutions are prescribed by the tools that generate them, there seems to be little research toward determining the impact of these predispositions on the quality of solutions relative to the intended outcome. To make truly informed decisions, designers must be able to assess a tool’s inherent biases, its intended applications, and its contextual appropriateness to the design questions that it is being used to answer. Currently, this inability can promote inaccurate perspectives of the desired design space, can negatively impact the ultimate success of the design, and poses a currently unquantified risk within the design process.This paper presents a framework for evaluating the inherent biases within a ship design synthesis tool. A genetic algorithm was developed to generate ensembles of solutions associated with a given design tool. The Parent–Child Network, derived from the dynamics within the genetic algorithm, was developed to capture causal relationships between solutions over time. The algorithm was modified to impose designer-determined biases on the solution generation process for the purpose of creating multiple data sets that enable comparative assessments of their resultant quality. This newly developed approach is used to establish the comparative context necessary for a model’s biases to be analyzed and understood by creating a reference and biasing cases. Two aspects of quality, solution-centric and generative, are defined, and a variety of solution-centric and generative analyses are developed to evaluate a tool’s inherent tendencies. A comprehensive case study was conducted to demonstrate implementing the framework and interpreting the analyses. The case study’s results demonstrate that the framework can successfully identify a model’s biases, providing designers with the contextual information necessary to make truly informed decisions.