Unstructured Qualitative Data Research Articles

Discovering Hidden Patterns: Applying Topic Modeling in Qualitative Research

In qualitative studies, researchers must devote a significant amount of time and effort to extracting meaningful themes from huge sets of texts and examining the links between themes, which are frequently done manually. The availability of natural language models has enabled the application of a wide range of techniques for automatically detecting hierarchy, linkages, and latent themes in texts. This paper aims to investigate the coherence of the topics acquired from the analysis with the predefined themes, the hierarchy between the topics, the similarity between the topics and the proximity-distance between the topics by means of the topic model based on BERTopic using unstructured qualitative data. The qualitative data for this study was gathered from 106 students engaged in a university-run pedagogical formation certificate program. In BERTopic procedure, paraphrase-multilingual-MiniLM-L12-v2 model was used as sentence transformer model, UMAP was used as dimension reduction method and HDBSCAN algorithm was used as clustering method. It is found that BERTopic successfully identified six topics corresponding to the six predicted themes in unstructured texts. Moreover 74% of the texts containing some themes could be classified accurately. The algorithm was also able to successfully identify which topics were similar and which topics differed significantly from the others. It was concluded that BERTopic is a procedure that can identify themes that researchers do not notice depending on the density of the data in qualitative data analysis and has the potential to enable qualitative research to reach more detailed findings.

Using Natural Language Processing and Machine Learning to Identify Internal Medicine-Pediatrics Residency Values in Applications.

Although holistic review has been used successfully in some residency programs to decrease bias, such review is time-consuming and unsustainable for many programs without initial prescreening. The unstructured qualitative data in residency applications, including notable experiences, letters of recommendation, personal statement, and medical student performance evaluations, require extensive time, resources, and metrics to evaluate; therefore, previous applicant screening relied heavily on quantitative metrics, which can be socioeconomically and racially biased. Using residency applications to the University of Utah internal medicine-pediatrics program from 2015 to 2019, the authors extracted relevant snippets of text from the narrative sections of applications. Expert reviewers annotated these snippets into specific values (academic strength; intellectual curiosity; compassion; communication; work ethic; teamwork; leadership; self-awareness; diversity, equity, and inclusion; professionalism; and adaptability) previously identified as associated with resident success. The authors prospectively applied a machine learning model (MLM) to snippets from applications from 2023, and output was compared with a manual holistic review performed without knowledge of MLM results. Overall, the MLM had a sensitivity of 0.64, specificity of 0.97, positive predictive value of 0.62, negative predictive value of 0.97, and F1 score of 0.63. The mean (SD) total number of annotations per application was significantly correlated with invited for interview status (invited: 208.6 [59.1]; not invited: 145.2 [57.2]; P < .001). In addition, 8 of the 10 individual values were significantly predictive of an applicant's invited for interview status. The authors created an MLM that can identify several values important for resident success in internal medicine-pediatrics programs with moderate sensitivity and high specificity. The authors will continue to refine the MLM by increasing the number of annotations, exploring parameter tuning and feature engineering options, and identifying which application sections have the highest correlation with invited for interview status.

Defining Virtual Consumerism Through Content and Sentiment Analyses.

This study set out to better understand virtual consumerism (VC) by applying natural language processing (NLP) methods for sentiment and content analyses. A total of 318 articles related to VC were identified on theguardian.com Web site and analyzed by text mining methodology. A thematic, content analysis using the Leximancer program was performed to explore VC as a concept, and its related concepts and concept associations. For the purposes of "deep-dive insights," further content and sentiment analyses were performed with MonkeyLearn and valence aware dictionary for sentiment reasoning. This triangulation in methodology enabled a comprehensive unstructured qualitative data analysis. The study identified key themes that characterize and define VC. It uncovered that, although there is predominantly positive sentiment toward VC reported in The Guardian online articles, negative sentiment also exists, presenting challenges for the industry to maneuver. The findings reveal that in the context of VC, a virtual experience is also a social experience in a virtual space, which is becoming and evolving. There are certain industries and sectors that are embracing VC, such as marketing, advertising and public relations, software development/IT, art/design, and entertainment, as well as science/technology. Some sectors and industries are experiencing challenges, such as security/law enforcement and medical, and hence display negative sentiment toward VC. Overall, this study presents a working definition of VC, a synopsis of the state of VC, and highlights areas for potential research to further our understanding of this phenomenon. It contributes to an improved understanding of VC for the industry and academia, and provides impetus for future studies focused on the emergent VC-relevant conceptual relationships.

An NLP-based Method to Calibrate the Degree of a Concept Emerging From Unstructured Qualitative Data

An NLP-based Method to Calibrate the Degree of a Concept Emerging From Unstructured Qualitative Data

Analysis of cervical cancer policy documents in East-African countries

Abstract Background Cervical cancer is largely a preventable disease according to data by Global Burden of Disease. Among the 20 countries with the highest incidence of cervical cancer globally, 16 are African countries. Relevant cancer control interventions in most low-income countries are partially developed attributing to unavailability of statistics limiting understanding and approximation of the magnitude of the burden of cancer. In order to evaluate health policies for prevention and management for cervical cancer among East-African countries a legal mapping was carried out. Methods Legal mapping as a qualitative research method is used to determinate existing policies, collect and analyze the information, and ultimately measure the effect of these policies on health outcomes. Legislation was searched from government websites, national cancer institute sites, international and national legal databases in a uniform way. Nvivo 12 software for content analysis of unstructured qualitative data related information was used for analysis. Coding of the data was developed by two researchers. Reliability of policies assessed according to Cohen's Kappa inter-rater reliability. Results The sample included 24 policies, plans guidelines, acts and strategic documents from 10 East-African countries. No relevant legislation identified from Somali and Djibouti. Majority of the countries had policy documents related to cervical cancer and touched on the components of cancer control. Majority of the East-African countries have not established screening registries leading to poor or no reliable data for assessing the cancer burden. Prophylactic vaccination against HPV is yet to be included in the national immunization schedules. Conclusions Findings show limited coverage of the identified policies, discrepancy to international guides. To reduce the harmful impact of the discordance between scientific evidence and policies, surveillance systems have to be set up/strengthened. Key messages Legislation which is brought into force to prevent cervical cancer is a pillar of effective public health efforts. Evidence based policy mapping plays a key role in giving insight on the burden of disease thereby making informed decisions for health.

Using artificial intelligence to examine online patient reviews.

Healthcare consumers are increasingly turning to online sources such as educational websites, forums and social media platforms to share their experiences with medical services and to demystify the uncertainties associated with undergoing various procedures. This study demonstrates a non-invasive way of understanding the feelings and emotions that consumers share via electronic word of mouth. By using IBM Watson, a content analysis tool that harnesses artificial intelligence, we show how a large amount of unstructured qualitative data can be transformed into quantitative data that can be subsequently analysed to generate novel insights into what patients are sharing about their healthcare experiences online.

QCA and the harnessing of unstructured qualitative data

AbstractThis paper proposes qualitative comparative analysis (QCA) as a novel method to harness unstructured data sets such as publicly available reports and news articles. It shows how QCA and conventional qualitative IS research can complement each other. In particular, it demonstrates how qualitative IS research can combine typical qualitative coding techniques with a specific type of QCA, namely crisp‐set QCA (csQCA). The paper illustrates how QCA offers qualitative IS research an innovative approach to explicate the combination of conditions associated with particular outcomes. Drawing on an empirical study of green IS, it showcases the potential of QCA to harness large unstructured qualitative material and generate deeper insights about emerging IS phenomena. The paper also highlights how QCA can contribute to the data collection and analysis stages of qualitative IS research.

New approaches to psychographic consumer segmentation

PurposeWhile the motivation for collecting art has received considerable attention in the literature, less is known about the characteristics of the typical art collector. This paper aims to explore these characteristics to develop a typology of art consumers using a mixed method approach over several studies.Design/methodology/approachThis is achieved by analyzing qualitative data, gathered via semi-structured interviews of art collectors, and quantitatively by means of natural language processing analysis and automated text analysis and using correspondence analysis to analyze and present the results.FindingsThe study’s findings reveal four distinct clusters of art collectors based on their “Big Five” personality traits, as well as uncovering insights into how these types talk about their possessions.Research limitations/implicationsIn addition to contributing to the arts marketing literature, the findings provide a more nuanced understanding of consumers that managers can use for market segmentation and target marketing decisions in other markets. The paper also offers a methodological contribution to the literature on correspondence analysis by demonstrating the “doubling” procedure to deal with percentile data.Practical implicationsIn addition to contributing to the arts marketing literature, the findings provide a more nuanced understanding of art collectors that managers can use for market segmentation and target marketing decisions. The paper also offers a methodological contribution to the literature on correspondence analysis by demonstrating a non-traditional application of correspondence analysis using the “doubling” procedure. Buyer behavior in the fine art market is not exhaustively studied. By understanding the personality traits of consumers in the art market, sales forces can better provide assistance and product to consumers. Further, understanding the personalities of consumers is better for art retail spaces to better serve consumers.Originality/valueThis paper demonstrates a unique mixed methods approach to analyzing unstructured qualitative data. It shows how text data can be used to identify measurable market segments for which targeted strategies can be developed.

Data-focused managerial challenges within the hotel sector

Purpose The purpose of this paper is to explore the managerial decision-making challenges of the use of data from online and offline customer data flooding into firms and managed through interactive dashboards. The paper provides insights into how managers within the hotel sector both manage and use the data as a core tool in discussions with finance managers/directors during negotiation on critical marketing investments. Design/methodology/approach The research design followed a multiple-case study design of five market-orientated hotels, which were selected as high performing data and dashboard users. Across each case company multiple informants were interviewed and shadowed, including marketing managers, general managers and finance managers/directors. There was also an in-depth investigation of the collection, dissemination and particularly use of traditional market research data and online data from social media sites, web sites and web and mobile analytics. This was augmented by a study of the customized, off site managed interactive databases, in use in all case companies. Findings This paper identifies managerial challenges experienced by general managers, marketing and financial managers using digital customer data. It investigated the power of the customer voice internally and found that unstructured qualitative data had greater visibility and usage once supported by a financial imperative. It also found that the use of interactive dashboards were a powerful manifestation of data technology use and increased marketing’s visibility, power and accountability within the firm. Research limitations/implications The paper provided a limited hotel perspective which impacts on generalizability. The findings should be tested quantitatively to ascertain their validity across a wider sector of businesses and operations. Though multiple site and cross-functional research adds value, one site might have provided more depth. Practical implications This paper reveals that managers need support in terms of time, resources and personnel to manage the flood of information into their organizations. It also suggested that general and marketing managers need to develop the skills to interpret the data for decision-making, so that the financial implications are understand. Originality/value Most social media studies within the hotel sector focus on the impact of social media reviews on consumer purchasing processes and/or on how managers communicate online with consumers. This paper takes an internal managerial perspective on data use for decision-making. This paper expands our understanding of marketing strategic decision-making through an in-depth exploration into how cross-functional decision-making uses contemporary customer data.

Evaluating Representativeness of Qualitative Text Data in Identifying UX Issues

ABSTRACTWith increasing recognition of user experience (UX), it became important to identify UX issues from customers’ opinions expressed in a form of text that is unstructured qualitative data. Since reliability of qualitative data has little been investigated, it requires much time and effort for researchers to define meanings and values from the qualitative data. Therefore, this article proposes a novel approach to measuring the stability of textual data by adopting semantic network analysis, as shown by our analysis on in-depth interview transcripts and online customer review data. Among the semantic networks generated from text, subnetworks were sampled from the original networks until the representativeness of each sample size was determined. Then, similarities between the subnetworks and the original network were calculated by applying a correlation analysis to determine whether stability occurs. The results indicated that variables of word frequency and network-level statistics affect stability of the network. The suggested method will be able to assist in identifying UX issues.

Evaluating hotels rating prediction based on sentiment analysis services

Purpose– The purpose of this paper is to assess the reliability of numerical ratings of hotels calculated by three sentiment analysis algorithms.Design/methodology/approach– More than one million reviews and numerical ratings of hotels in seven cities in four countries were extracted from TripAdvisor web site. Reviews were classified as positive or negative using three sentiment analysis tools. The percentage of positive reviews was used to predict numerical ratings that were then compared with actual ratings.Findings– All tools classified reviews as positive or negative in a way that correlated positively with numerical ratings. More complex algorithms worked better, yet predicted ratings showed reasonable agreement with actual ratings for most cities. Predictions for hotels were less reliable if based on less than 50-60 percent of available reviews.Practical implications– These results validate that sentiment analysis can be used to transform unstructured qualitative data on user opinion into quantitative ratings. Current tools may be useful for summarizing opinions of user reviews of products and services on web sites that do not require users to post numerical ratings such as traveler forums. This summarizing may be valuable not just to potential users, but also to the service and product providers and offers validation and benchmarking for future improvement of opinion mining and prediction techniques.Originality/value– This work assesses the correlation between sentiment analysis of hotels’ reviews and their actual ratings. The authors also evaluated the reliability of results of sentiment analysis calculated by three different algorithms.

Identity and athletic retirement: A prospective study

Objective The purpose of this study was to reexamine the relationship between identity and athletic retirement using a longitudinal, prospective design. Method One-on-one, in-depth interviews were conducted with three male and three female university student athletes at three times: at the outset of their last season of competition, approximately 1 month after their retirement, and approximately 1 year later. The unstructured raw data was analyzed inductively following procedures established in the literature [Côté, Salmela, Baria, & Russell (1993). Organizing and interpreting unstructured qualitative data. The Sport Psychologist, 7, 127–137; Côté, Salmela, & Russell (1995). The knowledge of high-performance gymnastic coaches: Methodological framework. The Sport Psychologist, 9, 65–75; Miller & Kerr (2002a). The athletic, academic, and social experiences of intercollegiate student-athletes. Journal of Sport Behavior, 25, 346–367]. Results The findings revealed the participants committed themselves strongly to their athletic goals and anticipated disrupted identities upon retirement. As a result, they employed several coping strategies including the proactive diminishment of their athletic identities prior to retirement. Decreasing the prominence of their athletic identities precluded a major identity crisis or confusion upon and following athletic retirement. Conclusions The study concludes that the redefinition of self long before sport career termination may protect one's identity during this transition process.

Concepts of trust among patients with serious illness

This paper examines conceptions of trust among three groups of respondents diagnosed with either breast cancer, Lyme disease or mental illness. Interviews were carried out using an open-ended interview guide to explore how patients made assessments of trust in their doctors and health care plans. The guide followed a conceptual approach that asked questions about competence, agency/fiduciary responsibility, control, disclosure and confidentiality. Respondents were given ample opportunity to raise other areas of concern. The data were organized using the NUDIST© software package for the analysis of non-numerical and unstructured qualitative data. Patients viewed trust as an iterative process and commonly tested their physicians against their knowledge and expectations. Interpersonal competence, involving caring, concern and compassion, was the most common aspect of trust reported, with listening as a central focus. Most patient comments referred to learnable skills and not simply to personality characteristics. Technical competence also received high priority but was often assessed by reputation or interpersonal cues. Patients were much concerned that doctors be their agents and fight for their interests with health care plans. Disclosure and confidentiality were less common concerns; most patients anticipated that doctors would be honest with them and respect their confidences. Patients’ responses also appeared to vary by their disease, their socio-demographic characteristics, their involvement with self-help groups, and how their illness conditions unfolded.

The Coaching Model: A Grounded Assessment of Expert Gymnastic Coaches’ Knowledge

An expert system approach (Buchanan et al., 1983) was used to identify and conceptualize the knowledge of 17 Canadian expert high-performance gymnastic coaches. The knowledge elicitation process consisted of open-ended questions and various questioning methods to unveil, explore, and prove important information (Patton, 1987; Spradley, 1979) about coaching. All coaches’ interviews were transcribed verbatim, and the unstructured qualitative data were inductively analyzed following the procedures and techniques of grounded theory (Strauss &amp; Corbin, 1990). The inductive analysis process allowed the meaning units of the interview transcripts to be regrouped into properties, categories, and components. The components emerging from the analysis consisted of (a) competition, (b) training, (c) organization, (d) coach’s personal characteristics, (e) gymnast’s personal characteristics and level of development, and (f) contextual factors. These components were further developed into a model representing coaches’ knowledge.

The Knowledge of High-Performance Gymnastic Coaches: Methodological Framework

An expert system approach (Buchanan et al., 1983) was used to identify and conceptualize the knowledge of 17 Canadian expert high-performance gymnastic coaches. By using a qualitative research methodology based on the traditions of cognitive anthropology and grounded theory, the first two stages of the knowledge acquisition process for building an expert system (identification and conceptualization) were examined. Open-ended questions and various questioning methods were used to unveil, explore, and probe important information (Patton, 1990; Spradley, 1979) about various coaching situations. All coaches’ interviews were transcribed verbatim, and the unstructured qualitative data were inductively analyzed following the procedures and techniques of grounded theory (Côté, Salmela, Baria, &amp; Russell, 1993; Glaser &amp; Strauss, 1967; Strauss &amp; Corbin, 1990). This article provides the underlying methodological framework used for the entire project. Details about the coaches studied and the methodological framework used to collect and analyze the data are presented.

A Decision-Making Heuristic for the Analysis of Unstructured Qualitative Data

Based on the guidelines suggested by Côté, Salmela, Baria, and Russell in 1993, this article provides a decision-making heuristic addendum to facilitate the organization and interpretation of unstructured qualitative data. At each decision point of the qualitative analysis, criteria are suggested to explore the phenomenon of issue and guide the coding process. The objectives of these methodological procedures are to facilitate the task of organizing and interpreting large bodies of qualitative data and to enhance the reliability and validity of the coding of this inductive process.

Organizing and Interpreting Unstructured Qualitative Data

In the last several years there has been an increase in the amount of qualitative research using in-depth interviews and comprehensive content analyses in sport psychology. However, no explicit method has been provided to deal with the large amount of unstructured data. This article provides common guidelines for organizing and interpreting unstructured data. Two main operations are suggested and discussed: first, coding meaningful text segments, or creating tags, and second, regrouping similar text segments, or creating categories. Furthermore, software programs for the microcomputer are presented as a way to facilitate the organization and interpretation of qualitative data.