Student Attitudes toward STEM: The Development of Upper Elementary School and Middle/High School Student Surveys

Abstract

Abstract Survey Instruments for Measuring Student Attitudes toward STEMThe national economy is in need of engineers and allied skilled workers in science, technology,and mathematics (STEM) fields, and this need is only expected to grow over time. As part of on-going efforts to understand impacts of K-12 STEM education and workforce developmentprograms, surveys focusing on student attitudes towards STEM were developed by researchers atNorth Carolina State University. The Upper Elementary School and Middle/High School StudentAttitudes toward STEM (S-STEM) Surveys use a 5-point Likert-style response scale to measurestudents’ confidence and interest in mathematics, science, engineering and technology, and 21stcentury skills. The surveys use a 4-point interest-level scale to measure students’ interest invarious STEM career areas, including engineering. This paper describes the process fordeveloping these two valid, reliable survey instruments.The surveys’ STEM constructs were adapted from a survey created by evaluators of a program atthe engineering schools of Northeastern University, Tufts University, Worcester PolytechnicInstitute, and Boston University – the program was designed to increase female middle schoolstudents’ interest in engineering. The North Carolina Student Learning Conditions Surveyprovided the basis for the items measuring students’ confidence in their 21st century skills.Finally, the list of STEM subject career areas was derived from multiple national sources,including the National Academy for Engineering.A pilot version of the middle/high school survey was administered to students, and results wereused to conduct the first round of factor analysis and subject matter expert review. Results led tothe dropping of some items and the re-wording of others, especially for the purposes of reducinggender bias and refining measurement of technical and engineering skills. An upper elementaryversion of the survey was created from this initial, revised version. Each item was rewritten to areading level appropriate for fourth and fifth grade students, and other improvements were madebased upon results from cognitive interviews with fifth grade students and feedback fromsubject-matter experts.The revised middle/high school version of the S-STEM Survey was administered toapproximately 9,000 middle and high school students, with the upper elementary versionadministered to approximately 900 fourth and fifth graders. Using this response data, furthervalidity and reliability tests were conducted. Factor analysis results showed strong, clearconstructs with high reliability after dropping just a few items. Results from another round ofsubject-matter expert reviews demonstrated that both surveys were of appropriate length and atappropriate reading-levels. Differential item functioning analyses showed that 6th-12th graderscomprehended the survey in similar ways and that female and male students differed slightly intheir comprehension of the relationships between mathematics, science, and engineering andtechnology.The process described in this paper demonstrates that the Upper Elementary and Middle/HighSchool S-STEM Surveys are valid and reliable instruments. These surveys can serve as usefultools for measuring impacts of K-12 engineering and STEM education programs on studentattitudes, supporting critical efforts to improve the quality and quantity of engineering educationand workforce development.