Protocol Analysis Study Research Articles

A Data-Driven LiFePO4 Battery Capacity Estimation Method Based on Cloud Charging Data from Electric Vehicles

The accuracy of capacity estimation is of great importance to the safe, efficient, and reliable operation of battery systems. In recent years, data-driven methods have emerged as promising alternatives to capacity estimation due to higher estimation accuracy. Despite significant progress, data-driven methods are mainly developed by experimental data under well-controlled charge–discharge processes, which are seldom available for practical battery health monitoring under realistic conditions due to uncertainties in environmental and operational conditions. In this paper, a novel method to estimate the capacity of large-format LiFePO4 batteries based on real data from electric vehicles is proposed. A comprehensive dataset consisting of 85 vehicles that has been running for around one year under diverse nominal conditions derived from a cloud platform is generated. A classification and aggregation capacity prediction method is developed, combining a battery aging experiment with big data analysis on cloud data. Based on degradation mechanisms, IC curve features are extracted, and a linear regression model is established to realize high-precision estimation for slow-charging data with constant-current charging. The selected features are highly correlated with capacity (Pearson correlation coefficient < 0.85 for all vehicles), and the MSE of the capacity estimation results is less than 1 Ah. On the basis of protocol analysis and mechanism studies, a feature set including internal resistance, temperature, and statistical characteristics of the voltage curve is constructed, and a neural network (NN) model is established for multi-stage variable-current fast-charging data. Finally, the above two models are integrated to achieve capacity prediction under complex and changeable realistic working conditions, and the relative error of the capacity estimation method is less than 0.8%. An aging experiment using the battery, which is the same as those equipped in the vehicles in the dataset, is carried out to verify the methods. To the best of the authors’ knowledge, our study is the first to verify a capacity estimation model derived from field data using an aging experiment of the same type of battery.

An investigation into the cognitive, metacognitive, and spatial markers of creativity and efficiency in architectural design

AbstractThis paper presents a preliminary study into the spatial features that can be used to distinguish creativity andefficiency in design layouts, and the distinct pattern of cognitive and metacognitive activity that is associated with creative design. In a design experiment, a group of 12 architects were handed a design brief. Their drawing activity was recorded and they were required to externalize their thoughts during the design process. Both design solutions and verbal comments were analysed and modelled. A separate group of experienced architects used their expert knowledge to assign creativity and efficiency scores to the 12 design solutions. The design solutions were evaluated spatially. Protocol analysis studies including linkography and macroscopic analysis were used to discern distinctive patterns in the cognitive and metacognition activity of designs marked with the highest and least creativity scores. Entropy models of the linkographs and knowledge graphs were further introduced Finally, we assessed how creativity and efficiency correlates to experiment variables, cognitive activity, metacognitive activity, spatial and functional distribution of spaces in the design solutions, and the number and type of design constraints applied through the course of design. Through this investigation, we suggest that expert knowledge can be used to assess creativity and efficiency in designs. Our findings indicate that efficient layouts have distinct spatial features, and that cognitive and metacognitive activity in design that yields a highly creative outcome corresponds to higher frequencies of design moves and higher linkages between design moves.

MODELLING PROPORTIONS AND SEQUENCES OF OPERATIONS IN TEAM DESIGN ACTIVITIES

AbstractThe presented research aims at modelling and formalising the process of team design activity as an interplay between the evolution of design problems and solutions. The motivation founds primarily on a presumption that there exist regularities in designing which can be captured and formalised using the appropriate models. The study thus investigates whether the identified design operation proportions and sequence probabilities are consistent throughout the different parts of team conceptual design activities. It does so by exploring the utility of mathematical models built based on the correlations and statistically significant sequences underlying the previously identified designing patterns. The developed mathematical model was tested by replicating moving-average analyses of design operation proportions and sequences, which were originally observed in the protocol analysis study. A close fit was found between the simulated and the observed data, particularly in providing insights regarding operation patterns and proportion trends. The presented models and modelling methodology are potentially an appropriate means for the next steps in describing, and consequently predicting and supporting team design activity dynamics.

Revisiting Problem-Solution Co-Evolution in the Context of Team Conceptual Design Activity

The conventional prescriptive and descriptive models of design typically decompose the overall design process into elementary processes, such as analysis, synthesis, and evaluation. This study revisits some of the assumptions established by these models and investigates whether they can also be applied for modelling of problem-solution co-evolution patterns that appear during team conceptual design activities. The first set of assumptions concerns the relationship between performing analysis, synthesis, and evaluation and exploring the problem and solution space. The second set concerns the dominant sequences of analysis, synthesis, and evaluation, whereas the third set concerns the nature of transitions between the problem and solution space. The assumptions were empirically tested as part of a protocol analysis study of team ideation and concept review activities. Besides revealing inconsistencies in how analysis, synthesis, and evaluation are defined and interpreted across the literature, the study demonstrates co-evolution patterns, which cannot be described by the conventional models. It highlights the important role of analysis-synthesis cycles during both divergent and convergent activities, which is co-evolution and refinement, respectively. The findings are summarised in the form of a model of the increase in the number of new problem and solution entities as the conceptual design phase progresses, with implications for both design research and design education.

A Silent Revolution: From Sketching to Coding – A Case Study on Code-based Design Tool Learning

Along with the information technology rising, Computer Aided Design activities are becoming more modern and more complex. But learning how to operation these new design tools has become the main problem lying in front of each designer. This study was purpose on finding problems encountered during code-based design tools learning period of designers, especially those influential factors. The research was based on two design protocol analysis studies. A pilot study was performed for research details adjustment. Three selected participants have been told to complete one sketching work with two different digital design tools. A 30 minutes teaching was performed on each participant. Processing was introduced for data analysis. We found digital methods can be used to performing design process analyzing. We construct a multi-scale analyzing structure for design process and we demonstrate the possibility of pure quantitative study of the design process.

The Impact of Data Quality Tags on Decision-Making Outcomes and Process

It has been proposed that metadata describing data quality (DQ), termed DQ tags, be made available in situations where decision makers are unfamiliar with the data context, for example, in data warehouses. However, there have been conflicting reports as to the impact of such DQ tags on decision-making outcomes. Early studies did not explicitly consider the usability and semantics of the DQ tag designs used experimentally or the impact of such tags on decision process, except in suggestions for future research. This study addresses these issues, focusing on the design of usable DQ tags whose semantics are explicitly specified and exploring the impact of such DQ tags on decision outcomes and process. We use the information quality framework InfoQual, the interaction design technique of contextual inquiry, and cognitive process tracing to address DQ tag semantics, usability, and impact on decision process, respectively. In distinct contrast to earlier laboratory experiments, there was no evidence that the preferred decision choice changed with DQ tags, but decision time was significantly increased and there were indications of reduced consensus. These results can be explained by understanding the impact of DQ tags on decision process using concurrent protocol analysis, which involves participants verbalizing thoughts while making a decision. The protocol analysis study shows that DQ tags are associated with increased cognitive processing in the earlier phases of decision making, which delays generation of decision alternatives.

Think-Aloud Technique and Protocol Analysis in Clinical Decision-Making Research

In this article, we aim to describe the practical aspects of research actions that should be taken into account when two methods--the think-aloud technique for data collection and protocol analysis as an analysis technique--are combined with research of decision making in acute clinical settings. These methods are rarely used together or written about in health care research. In addition, careful consideration of the pros and cons of study sampling, data collection, data management, and analysis techniques or approaches is still lacking when these methods are used in health care research. We discuss the background of the think-aloud technique and protocol analysis and review previous studies that have used these methods. The examples in the article are based on our study, in which we investigated the decision making of critical care experts. We find that the combination of the think-aloud technique and protocol analysis is applicable when investigating complex and overlapping decision-making processes and rapid, ad hoc decisions made by critical care experts.

Decision making in the conceptual design phases: a comparative study

This paper presents a comparative study of the design processes of final-year industrial design students of two countries, the Netherlands and the Portugal, while conducting an individual design task. This task was identical for both groups making this comparison possible although the studies are 15 years apart. This new study gave us opportunity to observe new aspects initially not focused upon. The operational aims of this study are the identification and comparison of the way senior design students in both groups take decisions, the relation with design moves along the process, and the factors influencing the decisions and moves. For that purpose both verbal protocol analysis studies (VPA) were analysed on the basis of activities and decision-making moments described in terms of reasons behind it and goals intended to be achieved through it. The results indicate the relevance of two aspects: (1) the abductive reasoning that supports designing gains visibility through analysis based on decision making, where idea generation plays a key role and (2) the notion of design as a decision-making process could bridge, in a meaningful way, design education and design practice in organisations.

Exploring problem decomposition in conceptual design among novice designers

Conceptual product design is commonly described as problem solving. In the present study we attempt to expand this view. Focusing on the solution search phase, we analyse explicit and implicit problem decomposition techniques and integrate them into a descriptive cognitive model. To evaluate the prevalence of decomposition modes empirically, we provide results from a verbal protocol analysis study involving 16 senior students of mechanical engineering. Data indicated that the subjects apply top–down control strategies coupled to implicit decomposition. Explicit decomposition was used seldom and without obvious benefits. We relate these results to the model that considers implicit decomposition as an integral part of the problem interpretation process and discuss the role of decomposition in a structured idea generation process.

Divergent thinking: Strategies and executive involvement in generating novel uses for familiar objects

Although the Alternative Uses divergent thinking task has been widely used in psychometric and experimental studies of creativity, the cognitive processes underlying this task have not been examined in detail before the two studies are reported here. In Experiment 1, a verbal protocol analysis study of the Alternative Uses task was carried out with a Think aloud group (N=40) and a Silent control group (N=64). The groups did not differ in fluency or novelty of idea production indicating no verbal overshadowing. Analysis of protocols from the Think aloud group suggested that initial responses were based on a strategy of Retrieval from long-term memory of pre-known uses. Later responses tended to be based on a small number of other strategies: property-use generation, imagined Disassembly of the target object into components and scanning of Broad Use categories for possible uses of the target item. Novelty of uses was particularly associated with the Disassembly strategy. Experiment 2 (N=103) addressed the role of executive processes in generating new and previously known uses by examining individual differences in category fluency, letter fluency and divergent task performance. After completing the task, participants were asked to indicate which of their responses were new for them. It was predicted and found in regression analyses that letter fluency (an executively loading task) was related to production of 'new' uses and category fluency was related to production of 'old' uses but not vice versa.

Library Catalogue Users Are Influenced by Trends in Web Searching

A review of:
 
 Novotny, Eric. “I Don’t Think I Click: A Protocol Analysis Study of Use of a Library Online Catalog in the Internet Age.” College & Research Libraries, 65.6 (Nov. 2004): 525-37.
 
 Objective – To explore how Web-savvy users think about and search an online catalogue. 
 
 Design – Protocol analysis study.
 
 Setting – Academic library (Pennsylvania State University Libraries).
 
 Subjects – Eighteen users (17 students, 1 faculty member) of an online public access catalog, divided into two groups of nine first-time and nine experienced users. 
 
 Method – The study team developed five tasks that represented a range of activities commonly performed by library users, such as searching for a specific item, identifying a library location, and requesting a copy. Seventeen students and one faculty member, divided evenly between novice and experienced searchers, were recruited to “think aloud” through the performance of the tasks. Data were gathered through audio recordings, screen capture software, and investigator notes. The time taken for each task was recorded, and investigators rated task completion as “successful,” “partially successful,” “fail,” or “search aborted.” After the searching session, participants were interviewed to clarify their actions and provide further commentary on the catalogue search. 
 
 Main results – Participants in both test groups were relatively unsophisticated subject searchers. They made minimal use of Boolean operators, and tended not to repair failed searches by rethinking the search vocabulary and using synonyms. Participants did not have a strong understanding of library catalogue contents or structure and showed little curiosity in developing an understanding of how to utilize the catalogue. 
 
 Novice users were impatient both in choosing search options and in evaluating their search results. They assumed search results were sorted by relevance, and thus would not typically browse past the initial screen. They quickly followed links, fearlessly tried different searches and options, and rapidly abandoned false trails. 
 
 Experienced users were more effective and efficient searchers than novice users. They used more specific keyword terms and were more persistent to review their search options and results. Through their prior experience, they knew how to interpret call numbers, branch library location codes, and library terminology such as ‘periodicals’. 
 
 Participants expected the catalogue to rank results based on relevancy like an Internet search engine. While most were observed to understand intuitively the concept of broadening or narrowing a search, a ‘significant minority’ added a term to an already too-narrow search to improve their search results. 
 
 When interviewed, participants suggested several ways to improve the catalog search query, such as adding summaries and contents, ranking results by relevance and degree of exact match to search terms, including an Amazon-like “find more like this” feature, and providing context-sensitive and interactive online help, especially at the point when a search has produced too many or too few hits. 
 
 Conclusions – The study concluded that library catalogue users are heavily influenced by trends in Web searching. No matter what type of search a task called for, the participants tended to expect a simple keyword search to lead to optimal results presented in relevancy-ranked order. 
 
 Because users do not generally know or care about the structure of a bibliographic record, and many have little concept of what a library catalogue is for or what it contains, Novotny suggests that user instruction needs to address these basics. He also suggests that library professionals and library system vendors must work together to address the clear evidence that library catalogues are failing their users. 

I Don’t Think I Click: A Protocol Analysis Study of Use of a Library Online Catalog in the Internet Age

A protocol analysis study was undertaken to explore how a Web-savvy generation of library users searches the online catalog. Eighteen users, including experienced and novice searchers, were recruited. Participants agreed to be recorded and to express their thoughts aloud while searching. Analysis of these data has revealed several distinct trends. Most notable among these trends has been the impact of Internet search engines on user expectations. Given the influence of the Web, these assumptions are likely common throughout higher education. This research reveals where bottlenecks occur and provides insights into how libraries can design systems that help users around trouble spots.

Entrepreneurship as a science of the artificial

This essay connects four key ideas from Herbert Simon’s “Sciences of the Artificial” to recent research on entrepreneurial expertise: (1) natural laws constrain but do not dictate our designs; (2) we should seize every opportunity to avoid the use of prediction in design; (3) locality and contingency govern the sciences of the artificial; and, (4) near-decomposability is an essential feature of enduring designs. The essay is based on a series of conversations and emails with Simon about the empirical findings of my doctoral dissertation that involved a protocol analysis study of expert founder-entrepreneurs.

NoDes: kNOwledge-based modeling for detailed DESign process — from analysis to implementation

This paper reviews the nature of detailed design process and discusses the implementation of a knowledge-based system (KBS) founded on a protocol analysis study. The main objective is to discuss the features of a knowledge-based modeling approach, kNOwledge-based modeling for detailed DESign process (NoDes). This approach was first used for the development of the wind, which is an interactive KBS for the synthesis of window parts. We take the wind as an exemplary prototype implementation to demonstrate the abilities and performance of our knowledge-based modeling approach. Besides knowledge analysis and simulation of task execution, this paper also argues the efficiency of the object-oriented programming approach in the system implementation.

Composition and construction in experts' and novices' weaving design

The purpose of this study was to analyze qualitative differences between novices' and experts' design processes and investigate the relationships between visual and technical designing in depth. Four participants, two professional weaving designers and two graduate students of textiles, participated in the protocol-analysis study. The results of the study indicated that the basic difference between experts and novices in their approach to weaving design is the novices' tendency to structure the process in a serial fashion around the visual elements of the task whereas experts were able to continuously consider the visual and technical elements of weaving in a parallel way during the design process.

EXPERTISE AND LEARNING: AN ALTERNATIVE PERSPECTIVE ON THE THEORY–PRACTICE DEBATE

The categories of ‘theory’ and ‘practice’ and the debate about the ‘gap between theory and practice’ have already produced a lively literature with calls for management accounting research to pursue a variety of ideas. The authors wish to contribute to this literature by reporting on a novel protocol analysis study of six qualified accountants as they talked about a short case study. The case study was used as the basis for undergraduate student assessment, as well as for the protocol analysis study, and so the authors were able to collect a substantial body of data on the differences and similarities of response between students, or novices, and qualified accountants, or experts. The paper uses a learning model to explain this data. The objective of the paper is to establish the nature of theory and practice as a learning model which includes ‘novice’ and ‘expert’ performance, and within which theory and practice are more interdependent than the concept of a ‘gap’ might suggest.

Research Report—The Relevance of Application Domain Knowledge: The Case of Computer Program Comprehension

The field of software, has, to date, focused almost exclusively on application-independent approaches. In this research, we demonstrate the role of application domain knowledge in the processes used to comprehend computer programs. Our research sought to reconcile two apparently conflicting theories of computer program comprehension by proposing a key role for knowledge of the application domain under examination. We argue that programmers use more top-down comprehension processes when they are familiar with the application domain. When the application domain is unfamiliar, programmers use processes that are more bottom-up in nature. We conducted a protocol analysis study of 24 professional programmers comprehending programs in familiar and unfamiliar application domains. Our findings confirm our thesis.

Discovering the design problem

The design disciplines of architecture, engineering, and computer science have provided three distinct models of the design process. We hypothesize that these models merely indicate three approaches to design, and that fragments of all three models will be found in any one discipline. To discuss this issue we present a discipline-independent cognitive framework which we then apply to a protocol analysis study of five engineering designers. Our results indicate that the designer's experience plays a key role in determining the design process.

Learning to Specify Information Requirements: The Relationship between Application and Methodology

The systems development process involves establishing the information requirements of an application and successively transforming those requirements into a computer-based model of the application. Attention iS usually focused almost exclusively on the method of transformation, however, with little recognition of the role of the application. As a first step in examining the relevance of knowledge of the application to the systems development process, this study addresses whether there are synergistic effects of application and methodology knowledge in specifying information requirements. This was achieved via a repeated-measures protocol analysis study that manipulated both experience with the application and knowledge of the methodology. The results show that in learning to specify information requirements, novice analysts: • performed more effectively over time when trained to use a methodology applied the methodology more effectively when familiar with the application; • performed more effectively when they used procedural methodology knowledge rather than declarative methodology knowledge alone; • improved the effectiveness of their problem solving over time only when they used procedural methodology knowledge; • produced idiosyncratic results based on the application. Based on the findings of this research, it appears that research into the nature of applications, as well as methodologies, is warranted. From the viewpoint of the practitioner, since application knowledge is idiosyncratic, it may be necessary to include more than one application-knowledgable person on a systems development team.

The form of design processes: a protocol analysis study

Protocol analysis (PA) methodology is discussed in light of its potential for empirical analysis of design problem-solving behaviour. A literature review highlights previous protocol analysis studies in design, with particular attention to Akin's 1 PA case study. Presented is a new model-based PA method whereby the verbal behaviour of designers is analysed, quantified, and statistically manipulated to reveal a unique and scientifically rigorous facsimile of problem-solving processes. A recent study based on the above method is outlined, and results of the study are reported.