Problem Solving In Technology-rich Environments Research Articles

Unsupervised Anomaly Detection in Sequential Process Data

Abstract: In this study, we present three types of unsupervised anomaly detection to identify anomalous test-takers based on their action sequences in problem-solving tasks. The first method relies on the use of the Isolation Forest algorithm to detect anomalous test-takers based on raw action sequences extracted from process data. The second method transforms raw action sequences into contextual embeddings using the Bidirectional Encoder Representations from Transformers (BERT) model and then applies the Isolation Forest algorithm to detect anomalous test-takers. The third method follows the same procedure as the second method, but it includes an intermediary step of dimensionality reduction for the contextual embeddings before applying the Isolation Forest algorithm for detecting anomalous cases. To compare the outcomes of the three methods, we analyze the log files from test-takers in the US sample ( n = 2,021) who completed the problem-solving in technology-rich environments (PSTRE) section of the Programme for the International Assessment of Adult Competencies (PIAAC) 2012 assessment. The results indicated that different groups of test-takers were flagged as anomalous depending on the representation (raw action sequences vs. contextual embeddings) and dimensionality of action sequences. Also, when the contextual embeddings were used, a larger number of test-takers were flagged by the Isolation Forest algorithm, indicating the sensitivity of this algorithm to the dimensionality of input data.

A machine learning-based procedure for leveraging clickstream data to investigate early predictability of failure on interactive tasks

Early detection of risk of failure on interactive tasks comes with great potential for better understanding how examinees differ in their initial behavior as well as for adaptively tailoring interactive tasks to examinees’ competence levels. Drawing on procedures originating in shopper intent prediction on e-commerce platforms, we introduce and showcase a machine learning-based procedure that leverages early-window clickstream data for systematically investigating early predictability of behavioral outcomes on interactive tasks. We derive features related to the occurrence, frequency, sequentiality, and timing of performed actions from early-window clickstreams and use extreme gradient boosting for classification. Multiple measures are suggested to evaluate the quality and utility of early predictions. The procedure is outlined by investigating early predictability of failure on two PIAAC 2012 Problem Solving in Technology Rich Environments (PSTRE) tasks. We investigated early windows of varying size in terms of time and in terms of actions. We achieved good prediction performance at stages where examinees had, on average, at least two thirds of their solution process ahead of them, and the vast majority of examinees who failed could potentially be detected to be at risk before completing the task. In-depth analyses revealed different features to be indicative of success and failure at different stages of the solution process, thereby highlighting the potential of the applied procedure for gaining a finer-grained understanding of the trajectories of behavioral patterns on interactive tasks.

Getting closer to the digital divide: An analysis of impacts on digital competencies based on the German PIAAC sample

This paper takes an intersectional perspective to investigate the effect of socio-demographic variables that may constitute to digital divide. The concept of digital divide emerged from a perspective on unequal access to digital technology and relates nowadays primarily the differences in the competencies necessary to handle this technology. To investigate digital divide, the present paper uses the PIAAC framework of digital competencies which is called problemsolving intechnology-richenvironments (PS-TRE). It introduces the approach of intersectionality that describes persons impaired by multiple inequalities.The paper analyzes the impact of these factors on PS-TRE for three subsamples of the German study: (1) employed people who use computers at work and at home, (2) employed people who use computers only at home, and (3) people that are out of the labor force. It analyzes furthermore contributions to digital divide by a comparison of these impacts with literacy and numeracy scores.While employed people with computer use at work and home only had generation as a factor for constituting digital divide, employed people with computer use only at home had migration background as a further factor. Education and cultural capital showed lower impacts on PS-TRE than on literacy and numeracy.

It’s not what you know but where you come from: Cognitive skills, job autonomy and latent discrimination of ethnic minorities in Israel

Job autonomy – employees’ freedom to schedule and organise their work independently according to their own experience and preferences – is a major factor in job satisfaction. However, it is not granted to many employees in Israel, and the authors of this article were interested in the reasons for this. Based on data from the Programme for the International Assessment of Adult Competencies (PIAAC), they investigated the association between cognitive skills and job autonomy among Israeli-born Jews, Arabs and immigrants from the former Soviet Union (FSU) living in Israel. They considered PIAAC data for cognitive skills in three key domains: literacy, numeracy and problem solving in technology-rich environments. In line with social homophily theory (which explains bonding tendencies among socially similar people), the authors’ findings show different effects of cognitive skills on job autonomy between the three groups. Whereas cognitive skills were positively correlated with job autonomy among Israeli-born Jews and partially among Arabs, these effects were insignificant among FSU immigrants. The findings suggest that in the case of FSU immigrants, a latent form of discrimination is taking place: they are not being rewarded for their credentials and skills with job autonomy. Revealing the discrimination mechanisms affecting job autonomy among ethnic minorities in Israel may help policymakers to overcome ethnic-based inequality and alienation in the labour market. It may also assist the development of new approaches that tap into skills currently being ignored and thereby increase employees’ well-being.

Literacy practices and the Programme for the International Assessment of Adult Competencies (PIAAC): A conceptual critique

The Programme for the International Assessment of Adult Competencies (PIAAC) assesses key skills in literacy, numeracy and problem-solving in technology-rich environments, as well as their relationship to other social outcomes for adults. PIAAC’s developers claimed to better account for adults’ literacy practices than earlier international studies such as the International Assessment of Literacy Study (IALS). Through the sociocultural lens of literacy as social practice, the authors explore the tension between PIAAC’s cognitive orientation and its attempt to factor in meaningful literacy practices. Specifically, they analyse PIAAC’s conceptualisation of literacy practices as instantiated in the background questionnaire given to adult participants. They conclude that PIAAC’s conceptualisation does not align well with current theoretical understandings of literacy practice, as evidenced by (1) its conflation of several key literacy terms, including text, genre and practice, and (2) its erasure of context, purpose and social interaction from literate practice. Thus, the authors found considerable room for improvement in the assessment of adults’ actual literacy practices.

The Effects of Social Characteristics of Jobs on the Cognitive Skills of Adults in the United States: A PIAAC Analysis

There is contrary research around whether collaboration at work fosters individual learning and skill development. This study’s purpose was to examine the relationship between social characteristics of jobs on adults’ cognitive skills as measured by the Program for the International Assessment of Adult Competencies (PIAAC). For select industry sectors, we used a linear regression model to predict scores for PIAAC scales—literacy, numeracy, and problem solving in technology-rich environments—based on the independent variables gender, education level, frequency of workplace collaboration, and frequency of sharing work-related information. Results showed level of collaboration at work is negatively associated with PIAAC scores, contradicting current thinking on the role of social interactions in the workplace. We conclude there may be an overemphasis on the social characteristics in job design in some industries and workplaces, leaving little support and time for other activities known to support workplace learning, like management support and time for reflection.

An exploration of U.S. adults’ information processing skills and political efficacy

As recent debates about the role of social media in election outcomes suggest, understanding the association between information processing skills and political efficacy in the U.S. is a significant inquiry for adult and continuing education. Data from the Programme for the International Assessment of Adult Competencies were used to explore relationships between U.S. participants’ information processing skills and political efficacy. Given the impact of certain demographic characteristics on political efficacy, the study also analyzed whether the relationship between information processing skills and self-reported political efficacy varied across levels of cultural engagement, formal educational attainment, or immigrant status. The results illustrate that higher levels of literacy, numeracy, and problem solving in technology-rich environments are associated with higher political efficacy for U.S. participants. Our research is framed in both cognitive and critical lenses, and we provide implications for practice in adult and continuing education settings.

Examining problem-solving skills in technology-rich environments as related to numeracy and literacy

ABSTRACTThis study sought to a better understanding of the construct of problem solving in technology-rich environments and the effect of literacy and numeracy on problem solving. Data used in this study were drawn from Programme for the International Assessment of Adult Competencies US data which includes 5010 completed cases and a total of 1326 variables. The assessment of literacy, numeracy and problem-solving competencies were administrated using computer-based approaches. The result of the study showed that adults with higher numeracy and literacy competencies were more likely to have higher level of problem-solving skills. The results of the analyses also revealed that solution latency (i.e. time) were an important factor influencing problem-solving skills. This study indicates that basic mathematical skills are essential for solving problems that require interpersonal communication, computer and software knowledge, planning, and organising. The findings from this study provide several implications for researchers, educators, teachers and policymakers.

What predicts literacy, numeracy, and problem solving for incarcerated individuals? A PIAAC examination

The purpose of this article was to investigate relationships between literacy, numeracy, and problem solving ability with characteristics of individuals within the United States prison system. The Programme for the International Assessment of Adult Competencies (PIAAC) dataset was used for analysis, which included information for 1,319 participants ages 16–74. Results indicated that offenders’ prior experiences and prison context variables differentially predicted literacy, numeracy, and problem solving in technology-rich environments. Implications for prison education are discussed along with study limitations.

Mapping Background Variables With Sequential Patterns in Problem-Solving Environments: An Investigation of United States Adults' Employment Status in PIAAC.

Adult assessments have evolved to keep pace with the changing nature of adult literacy and learning demands. As the importance of information and communication technologies (ICT) continues to grow, measures of ICT literacy skills, digital reading, and problem-solving in technology-rich environments (PSTRE) are increasingly important topics for exploration through computer-based assessment (CBA). This study used process data collected in log files and survey data from the Programme for the International Assessment of Adult Competencies (PIAAC), with a focus on the United States sample, to (a) identify employment-related background variables that significantly related to PSTRE skills and problem-solving behaviors, and (b) extract robust sequences of actions by subgroups categorized by significant variables. We conducted this study in two phases. First, we used regression analyses to select background variables that significantly predict the general PSTRE, literacy, and numeracy skills, as well as the response time and correctness in the example item. Second, we identified typical action sequences by different subgroups using the chi-square feature selection model to explore these sequences and differentiate the subgroups. Based on the malleable factors associated with problem-solving skills, the goal of this study is to provide information for improving competences in adult education for targeted groups.

Twenty-First Century Skills for All: Adults and Problem Solving in Technology Rich Environments

The current Information Society requires new skills for personal, labor and social inclusion. Among the so-called Twenty-First Century Skills (Care et al. (eds) in Assessment and teaching of 21st century skills, Springer, New York, 2018) is Problem Solving in Technology Rich Environments (PS-TRE) competence evaluated in PISA and PIAAC tests (OECD in Survey of adult skills (PIAAC). Retrieved from https://goo.gl/cpb3fQ (2016)). This skill, although currently receiving considerable attention in compulsory education, has not received the same level of thought in the case of adult education. In this article, the presence of the PS-TRE skill among adults of working age (25–65 years) in Europe is analysed in relation to the factors that potentially affect a higher score in this skill. This analysis is carried out using structural equations modelling, taking into account socio-personal and educational factors, as well as the use of different skills in work and daily life. The results indicate that educational level and the use of different skills (reading, numerical, related to ICT) at home and at work, as well as participation in non-formal education activities, decisively relate to a higher level of PS-TRE. This result is positively mediated through risk factors such as being older or being a woman. This study concludes that it is necessary to reinforce these skills, not only in children, but also in the adult population, in order to avoid social and labour exclusion.

Problem Solving in Technology-Rich Environments and Self-Rated Health Among Adults in the U.S.: An Analysis of the Program for the International Assessment of Adult Competencies

Problem-solving skills in the context of technologically complex modern societies have become increasingly important to health management in later life. This study is designed to investigate the associations between problem-solving skills in technology-rich environments (PSTRE) and health, and to explore whether age differences exist. Using data from the 2012/2014 Program for the International Assessment of Adult Competencies (PIAAC), we used logistic regression to examine the relationship between PSTRE and self-reported health among a representative sample of American adults aged 35 years and older (N = 3,260). Overall, greater PSTRE (odds ratio [OR] = 1.012, p < .001) was significantly associated with better self-rated health even after adjusting for the sociodemographic characteristics. Yet, PSTRE was only partially predictive of health in some age groups. Our findings highlight the potential of PSTRE to reduce health disparities among middle-aged and older adults living in modern technology and information-rich societies.

Miten aikuisten perustaidot määritellään?

Analysoin artikkelissa aikuisten perustaitojen määrittelyä diskurssin eri tasoilla eli instituutioiden ja käytäntöjen muokkaamina puheina sekä teksteinä, jotka vaikuttavat sosiaalisen todellisuutemme ymmärtämiseen. Tarkastelun kohteena ovat yhteiskuntapoliittisen ja toiminnallisen tason diskurssit, joita edustavat Taloudellisen kehityksen ja yhteistyön järjestön OECD:n teettämä kansainvälinen PIAAC-aikuistutkimus, Suomen opetus- ja kulttuuriministeriön Taito-ohjelma (2014–2020) sekä ohjelmaan kuuluvien hanketoimijoiden ja kohderyhmien haastattelut. Huomioni diskurssissa on kiinnittynyt erityisesti aikuisten perustaitojen määrittelyyn, niiden heikkouksiin ja kehittämisehdotuksiin. Analyysini ei pyri tulkitsemaan perustaitojen tilannetta sellaisenaan vaan sen argumentointia. Kansainvälisen OECD:n asiantuntijadiskurssin vaikutus perustaitojen määrittelyyn on ilmeinen. OECD nimeää perustaidoksi luku-, numero- ja ongelmanratkaisutaidot teknologisesti kehittyneissä ympäristöissä. Sen tulkinta kansalaisten tarvitsemista perustaidoista ohjaa kansallisen koulutuspolitiikan retoriikkaa, mikä edelleen antaa suuntaa koulutuksen toteuttajille. Tulkinta on kuitenkin yksinkertaistava ilman diskurssin muiden tasojen kuten paikallisten aikuiskoulutuksen ammattilaisten näkemysten huomioimista ja näiden välistä dialogisuutta. Pyrin lisäämään taitopuheen moniäänisyyttä ja nostamaan esiin vaihtoehtoisia ja paikallisia puheenvuoroja. Tarkoitukseni on tuoda diskurssin sisältämiä valtasuhteita näkyviksi ja kyseenalaistaa yksipuolistavia tulkintoja.

Librarians help high school students improve research skills.

Millennials, the generation born from the early 1980s to early 2000s, and anyone born after them are often assumed to be effective information technology users who, because of their experience growing up with technology, are fully prepared to enter today's technology-rich workplaces after completing their education. Yet it is becoming increasingly clear that tech savviness by itself does not guarantee having the essential skills that many employers are looking for—skills that can vary all the way from working with spreadsheets to higher-level problem solving to the ability to perform high-quality research. Goodman found that “Millennials may be on track to be our most educated generation ever, but they consistently score below many of their international peers in literacy, numeracy and problem solving in technology-rich environments” [1]. Since at least 1966, the National Library of Medicine (NLM) has had discussions about reaching out to students in kindergarten through twelfth grade (K–12). In 2006, NLM's 10-year long-range plan recognized the need “for a greatly expanded and more diverse workforce of clinicians, informaticians and librarians” [2]. It recommended that NLM help to meet this need by continuing to adapt its existing educational programs and by “increasing its focus on K–12 education.” Today, NLM's Division of Specialized Information Services provides a robust offering of K–12 science and health education resources to help teachers introduce, reinforce, and supplement education programs [3]. Librarians, working in collaboration with teachers, are in a position to help prepare today's K–12 students to become the health professionals of tomorrow. As a health sciences outreach services librarian who has worked with K–12 students and teachers, I have learned that teachers can greatly benefit from and deeply appreciate having health information professionals work with them in the classroom. One teacher I have been working with for a couple of years mentioned to me that materials such as curricula are very useful yet not sufficient: Often more complex questions arise, and having a health information professional present in the classroom at certain key points in the curriculum to answer intricate questions and to work with the students through parts of the research process is extremely helpful. As the outreach services librarian at the University of Arizona Health Sciences Library, I have been partnering with high school science teachers for five years. I work mainly with juniors and seniors (eleventh and twelfth grades) from about thirteen high schools each year. Included in the students' experience—usually toward the beginning of the school year—is a visit to the university campus and the health sciences library. For many students, visiting a university campus is a new experience. In the library classroom, the students, who have been assigned a research project beforehand, are taught the fundamentals of database searching and are assisted in locating literature relevant to their project. This model teaches students research skills that they can apply to both current and future classroom assignments, while at the same time exposing them to the university environment, demystifying it, and, for many, helping them begin to realize that higher education can be an attainable goal. Recently, I have had conversations with several elementary school teachers and librarians to explore the possibility of working with students and teachers at earlier grade levels. One idea includes working with students in connection with school gardens and seed libraries; helping them learn about food, diet, and nutrition; and maybe learning about some basic research concepts in the process and to do so in such a way that students can take some of these concepts back home with them. Based on this preliminary brainstorming, I think there is potential for health sciences librarians to make meaningful impacts even at the earliest grade levels.

Laypersons’ digital problem solving: Relationships between strategy and performance in a large-scale international survey

This study examines the role of online data as indicators of the cognitive processes involved in problem solving in a technology-rich environment. More specifically, we analyze the relationship between response time, logged action count and task outcomes in a sample of over 23,000 adults from 16 countries who participated in the Problem solving in technology-rich environments (PS-TRE) assessment as part of the Program for the International Assessment of Adult Competencies (PIAAC) survey. Based on a selection of tasks used in the PS-TRE assessment, the results show that while time on task may have a heterogeneous effect on a population level depending on task difficulty, action count is positively linked to task accuracy. The data also reveals a surprisingly varied and task-specific relationship between those variables.

ICT Utilization and Problem Solving Skills: Based on the Analysis of PIAAC Data

Information, Communication and Technology(ICT) has changed every aspect of our life and ways we produce, analyze, utilize, and share information across all industries and occupations. This study analyzed the published data from the Survey of Adult Skills of OECD in order to find out the degree of adult ICT Received (February 10, 2016), Review Result (February 24, 2016) Accepted (March 3, 2016), Published (April 30, 2016) 06639 Dept. Elementary Education, Seoul National University of Education, 1650 Seocho-dong, Seocho-gu, Seoul, Korea email: sylee@snue.ac.kr * 이 논문은 (사) 인문사회과학기술융합학회 2015년 추계학술대회 발표문을 수정·보완한 논문임. ICT Utilization and Problem Solving Skills : Based on the Analysis of PIAAC Data Copyright c 2016 HSST 240 skills of Koreans. OECD measured the levels of information processing skills of adults from 24 countries, and compared the levels of the competencies across participating countries through the Programme for International Assessment for Adult(PIAAC) survey. This study focused on the problem-solving in technology-rich environments(PSTRE) skills among other measured competencies(i.e., literacy and numeracy). The distributions of PSTRE levels by country, age, education and skill level are examined. In addition, the levels of ICT utilization at home and at work are compared by country and age. The percentage of Korean adults whose PSTRE score was higher than Level 2 was 30.4%, which was the lower end of the chart. The degree of ICT utilization of Koreans at work was about the average of all participating countries, while that of ICT utilization at home was the second lowest. The competencies measured in PIAAC survey indicate not only the competitiveness of current human resources in a country but also the potential competitiveness of the future human resources. To this end, the results provide implications for education, life-long learning and work-place training system and related policies.

A comparative analysis of national media responses to the OECD Survey of Adult Skills: policy making from the global to the local?

The Organization for Economic Co-operation and Development’s (OECD) Programme of International Assessment of Adult Competencies (PIAAC) is put forward as a landmark development in the lifelong monitoring and international comparison of education. The first round of PIAAC’s Survey of Adult Skills compared performance in literacy, numeracy and problem solving in technology-rich environments across 24 countries. However, the translation of any OECD agenda into national policies is mediated by many actors, including the media. This paper examines and compares how the national media of Japan, England and France reported on the PIAAC results of their countries and the extent to which these reports mirror key messages from the OECD’s Country Notes. It begins to trace how the OECD PIAAC agendas materialise into national policies. Although their role in this initial period was limited, we argue the roles of the media together with other policy actors must be monitored as they interact to shape possibilities for sustainable adult education policies.

The development and validation of testing materials for literacy, numeracy and digital skills in a Dutch context

Besides work-oriented training, most Dutch adult learning courses of formal and non-formal education focus on three basic skills: literacy, numeracy and problem solving in technology-rich environments. In the Netherlands, the Ministry of Education, Culture and Science recently initiated the development of a new adult education framework concerning literacy, numeracy and digital skills. In order to monitor the progress of literacy, numeracy and digital competencies, it is necessary to develop and validate testing materials for specific competencies. This study validates the testing materials which were developed to assess learners’ proficiency in literacy (reading and writing), numeracy and digital skills based on the new Dutch framework. The outcome is that the materials proved valid and can be used in different courses referring to basic skills and adult learning, though there are still some limitations. Besides adult education professionals (such teachers and trainers), policy makers can also use the results of these tests in order to describe and monitor the impact of adult education on the lives of adult learners.

Digitalna pismenost in reševanje problemov v tehnološko bogatih okoljih

Hiter razvoj in vedno večja vloga informacijsko-komunikacijskih tehnologij zahtevata bolj razvite digitalne spretnosti posameznikov. Prispevek se osredotoča na koncepta digitalnih spretnosti in reševanja problemov v tehnološko bogatih okoljih, ki bosta obravnavana tudi na podlagi empiričnih podatkov, pridobljenih v mednarodni študiji PIAAC. V uvodnem delu je podan pregled literature in so predstavljeni rezultati predhodnih raziskav s področja merjenja digitalnih spretnosti ter podatki o uporabi storitev informacijske družbe med članicami Evropske unije. V drugem delu članka analiziramo rezultate, pridobljene v študiji PIAAC. Ti rezultati so pokazali, da imata največji vpliv na uspešnost reševanja problemov v tehnološko bogatih okoljih starost in izobrazba, kar potrjujejo tudi rezultati drugih raziskav. Prispevek zaključujemo s predlogi za izboljšanje trenutnega stanja vključenosti vseh skupin prebivalstva v programe usposabljanja na področju digitalnih spretnosti.

Socio-demographic and practice-oriented factors related to proficiency in problem solving: a lifelong learning perspective

This article explores the relative importance of different socio-demographic and practice-oriented factors that are related to proficiency in problem solving in technology-rich environments (PSTREs) and by extension may be related to complex problem solving (CPS). The empirical analysis focuses on the proficiency measurements of PSTRE made available by the Programme for the International Assessment of Adult Competencies, which is relevant for gaining insight on some of the factors related to CPS. The purpose was to examine the relationship between the broad information processing experience that individuals gather in different contexts over the lifespan, and the chances to develop problem-solving skills that adults receive in different socio-demographic profiles. Results reveal that socio-demographic factors such as age, education and immigration status as well as practice-oriented factors such as ICT use and reading practice at and outside work are strongly related to proficiency.