Underspecified Problem Research Articles

Examining the potential and pitfalls of ChatGPT in science and engineering problem-solving

The study explores the capabilities of OpenAI's ChatGPT in solving different types of physics problems. ChatGPT (with GPT-4) was queried to solve a total of 40 problems from a college-level engineering physics course. These problems ranged from well-specified problems, where all data required for solving the problem was provided, to under-specified, real-world problems where not all necessary data were given. Our findings show that ChatGPT could successfully solve 62.5% of the well-specified problems, but its accuracy drops to 8.3% for under-specified problems. Analysis of the model's incorrect solutions revealed three distinct failure modes: (1) failure to construct accurate models of the physical world, (2) failure to make reasonable assumptions about missing data, and (3) calculation errors. The study offers implications for how to leverage LLM-augmented instructional materials to enhance STEM education. The insights also contribute to the broader discourse on AI's strengths and limitations, serving both educators aiming to leverage the technology and researchers investigating human-AI collaboration frameworks for problem-solving and decision-making.

When is Early Classification of Time Series Meaningful

Since its introduction two decades ago, there has been increasing interest in the problem of early classification of time series. This problem generalizes classic time series classification to ask if we can classify a time series subsequence with sufficient accuracy and confidence after seeing only some prefix of a target pattern. The idea is that the earlier classification would allow us to take immediate action, in a domain in which some practical interventions are possible. For example, that intervention might be sounding an alarm or applying the brakes in an automobile. In this work, we make a surprising claim. In spite of the fact that there are dozens of papers on early classification of time series, it is not clear that any of them could ever work in a real-world setting. The problem is not with the algorithms per se but with the vague and underspecified problem description. Essentially all algorithms make implicit and unwarranted assumptions about the problem that will ensure that they will be plagued by false positives and false negatives even if their results suggested that they could obtain near-perfect results. We will explain our findings with novel insights and experiments and offer recommendations to the community.

Application of artificial neural networks to identify equilibration in computer simulations

Determining which microstates generated by a thermodynamic simulation are representative of the ensemble for which sampling is desired is a ubiquitous, underspecified problem. Artificial neural networks are one type of machine learning algorithm that can provide a reproducible way to apply pattern recognition heuristics to underspecified problems. Here we use the open-source TensorFlow machine learning library and apply it to the problem of identifying which hypothetical observation sequences from a computer simulation are “equilibrated” and which are not. We generate training populations and test populations of observation sequences with embedded linear and exponential correlations. We train a two-neuron artificial network to distinguish the correlated and uncorrelated sequences. We find that this simple network is good enough for > 98% accuracy in identifying exponentially-decaying energy trajectories from molecular simulations.

Teaching Database Design with Constraint-Based Tutors

Design tasks are difficult to teach, due to large, unstructured solution spaces, underspecified problems, non-existent problem solving algorithms and stopping criteria. In this paper, we comment on our approach to develop KERMIT, a constraint-based tutor that taught database design. In later work, we re-implemented KERMIT as EER-Tutor, and extended its instructional domain. Several evaluation studies performed with KERMIT and EER-Tutor show that they are effective Intelligent Tutoring Systems (ITSs). We also comment on various extensions made to EER-Tutor over the years. There are several contributions of our research, such as developing effective problem-solving support for conceptual database design in terms of interface design. Our database design tutors deal with large solution spaces efficiently by specifying constraints that capture equivalent solution states, and using ideal solutions to capture the semantics of the problem. Instead of requiring a problem solver, the ITS checks whether the student’s database schema is correct by matching it to constraints and the ideal solution. Another contribution of our work is in guidelines for developing effective feedback to the student.

Differences between Spatial and Visual Mental Representations

This article investigates the relationship between visual mental representations and spatial mental representations in human visuo-spatial processing. By comparing two common theories of visuo-spatial processing – mental model theory and the theory of mental imagery – we identified two open questions: (1) which representations are modality-specific, and (2) what is the role of the two representations in reasoning. Two experiments examining eye movements and preferences for under-specified problems were conducted to investigate these questions. We found that significant spontaneous eye movements along the processed spatial relations occurred only when a visual mental representation is employed, but not with a spatial mental representation. Furthermore, the preferences for the answers of the under-specified problems differed between the two mental representations. The results challenge assumptions made by mental model theory and the theory of mental imagery.

Fidelity and validity of simulator training

Through a case study, this article explores a number of theoretical issues related to the often taken for granted relationship between simulator fidelity and the quality and transferability of training in complex, dynamic, safety-critical settings. A counterexample based on mid-fidelity simulation is presented and the assumed coincidence of fidelity and validity is tested, that is the study tests the equation of constructed photorealism (built to mimic reality) and effective development of the competence that operators require to manage situations that involve underspecified problems, time pressure constraints and complex group interaction. The article concludes that such competence development cannot rely only on highly context-specific (photorealistic) environments. Further, it will be argued that lower-fidelity simulation, when appropriately designed, can provide competence development with pedagogical and economic advantages.

Sub-pixel confusion–uncertainty matrix for assessing soft classifications

The prevailing concerns on ecological and environmental issues, occurring especially at regional to global scales, have prompted significant advances on the use of remote sensing data for the estimation of land cover information at sub-pixel level. However, the quality of such classification products, as well as the performance of the classification protocol employed, are difficult to quantify. This paper had the objectives of 1) reviewing the existing alternatives, while identifying major drawbacks and desirable properties, for sub-pixel accuracy assessment based on cross-comparison matrices, and 2) developing theoretical grounds, for a more general accuracy assessment of soft classifications, that account for the sub-pixel class distribution uncertainty. It was found that, for a sub-pixel confusion matrix to exhibit a diagonalization characteristic that allows identifying a perfect matching case, the agreement measure must be constrained at pixel level, whereas a disagreement measure can take into account the sub-pixel distribution uncertainty, leading to an underspecified problem termed the sub-pixel area allocation problem. It was demonstrated that the sub-pixel area allocation problem admits a unique solution if, and only if, no more than one class is either over- or underestimated at each pixel. In this case, the sub-pixel confusion can be uniquely determined. When no unique solution exists, the space of feasible solutions can be represented by confusion intervals. A new cross-comparison matrix that reports the confusion intervals in the form of a center value plus–minus maximum error was proposed to account for the sub-pixel distribution uncertainty. The new matrix is referred to as sub-pixel confusion–uncertainty matrix (SCM). Sub-pixel accuracy measures were also derived from this matrix. The practical use of the SCM and derived indices was demonstrated in assessing an invasive species detection method and a fuzzy classification of urban land use/land cover through remote sensing procedures.

Stability and traction control of an actively actuated micro‐rover

AbstractThis article addresses the issue of enhancing mobility of actively actuated vehicles by the use of optimal force distribution, and by actively controlling the location of the center of mass of the vehicle body. Force distribution in actively actuated vehicles is an underspecified problem, and optimal force distribution leads to improved contact conditions. Actively actuated vehicles also possess the ability to vary their configuration. This capability can be used to locate the center of mass of the system to further enhance mobility by providing sufficient traction, and ensuring vehicle stability, during difficult vehicle maneuvers. In this work, the general idea of control of the location of the center of mass, followed by optimal force distribution, is addressed first. This is followed by an application of these ideas to Gofor, a simple actively actuated wheeled micro‐rover system, that has been designed and fabricated at the Jet Propulsion Laboratory. An algorithm to achieve desired traction and stability has been developed. Simulation studies were performed to investigate negotiation of a vertical surface. It was found that one set of wheels readily climbed the vertical surface with the use of active mass and force distribution. This is a significant improvement over traditional wheeled vehicle systems. © 1994 John Wiley & Sons, Inc.

An Exact Optimization of Interaction Forces in Three-Fingered Manipulation

Manipulation with three fingers in frictional contact is known to be an underspecified problem. The minimum norm solution, also referred to as the equilibrating solution field, can be modified by superimposing a linear combination of the resulting null solutions, called the interaction force field. The modification proposed here, applicable to the three-dimensional problem, minimizes the maximum value of the friction angle at the points of contract. A grasp of maximum possible stability under a set of contact and loading conditions is hence achieved.