A Survey of Systems Engineering Effectiveness

Abstract

Abstract : The Systems Engineering Division (SED) of the National Defense Industrial Association (NDIA) established the Systems Engineering Effectiveness Committee (SEEC) to obtain quantitative evidence of the effect of Systems Engineering (SE) best practices on Project Performance. The SEEC developed and executed a survey of defense industrial contractors (i.e., suppliers to the government) to identify the SE best practices utilized on defense projects, collect performance data on these projects, and search for relationships between the application of these SE best practices and Project Performance. The SEEC surveyed a sample of the population of major government contractors and subcontractors consisting of contractors and subcontractors represented in the NDIA SED. The survey questionnaire was developed using the SE expertise and the broad diversity of experience of the SEEC members. The questionnaire consisted of three sections; one to identify the characteristics of the responding project, a second to assess the project's utilization of SE best practices, and a third to collect measures of Project Performance. The survey data was collected by the Carnegie Mellon Software Engineering Institute (SEI) via the Web. Policies ensuring the anonymity of the respondents and the confidentiality of their responses were enforced to protect the competition-sensitive information supplied. Responses sufficient for most analyses were received from a total of 46 projects; another 18 projects provided partial responses useful for basic descriptive purposes. These responses were analyzed by the SEI to identify relationships between the deployment of SE best practices and overall project/program performance. The results of this analysis are published in this report. Only aggregated results are contained in this report; no information traceable to any individual respondent, project, or organization is included.