Saber Pro Research Articles

Spatial analysis of academic performance in the areas of systems and telematics in the Saber Pro tests in the Colombian Caribbean

Objective: To conduct a study based on spatial data analysis to characterize academic performance in the five competencies of the Saber Pro tests, carried out by students of Systems Engineering and related programs in the departments of the Colombian Caribbean. Methodology: For the development of this research, a four-phase adaptation of the CRISP-DM methodology was utilized: F1. Business and data understanding, F2. Data preparation, F3. Modeling, F4. Evaluation and analysis. Results: As a result, the analysis of quantiles identified the departments in the Caribbean coast with the highest and lowest performance across the five areas of the test. Additionally, spatial regression analysis was used to determine the correlation among the five competencies of the test. Finally, spatial clustering analysis identified groups of departments that collectively achieved the best and worst results. Conclusions: the results obtained in this study are intended to guide strategic decision-making aimed at improving the quality of education in the Colombian Caribbean coast. Furthermore, the proposed study serves as a foundation for extrapolating spatial data analysis to other regions and specific levels of the Saber tests.

Strengthening Generic Skills in Undergraduate Students in Physical Education. Perception of University Professors

Purpose: Demonstrate the effectiveness of the proposal for strengthening generic and specific competencies, which are evaluated in the Saber Pro tests in students of the Bachelor's program in Physical Education at a Colombian university, through the perception of participating teachers. said proposal. Methodology: A qualitative methodology was implemented, with the purpose of trying to understand the reality and behaviors of individuals who belong to a certain social context (Vílchez, 2022), in addition to understanding and identifying various characteristics associated with the qualities and descriptions of a social phenomenon without resorting to the collection of specific numerical data (Morenos, 2021). Results and Conclusions: They indicate the development of innovative teaching strategies in academic spaces in order to strengthen competencies. Likewise, it demonstrates the viability of the proposal in relation to the teaching strategies that make a theoretical and practical approach, since they adjust to the focus and thematic contents of the subjects, therefore, they facilitate the assimilation and appropriation of the knowledge that requires a future graduate, thus concluding the need to incorporate innovative tools that promote the appropriation of knowledge to train better professionals in the area of Physical Education. Implications: This research provides implications for educational and government policy to continue to strengthen standardized testing policies in higher education. Originality/value: This research offers novelty by glimpsing the impact of the formation of the pedagogical model in higher education.

A Gender Gap Analysis on Academic Performance in Engineering Students on Admission and Exit Standardized Tests

This paper describes a gender-based research work on academic performance in engineering students. This study is based on the results of 9 469 students from Universidad Nacional de Colombia in the college`s Admission and exit standard tests (the latter known as Saber Pro Tests). Tools such as descriptive statistics, regression analysis, and data mining are used to estimate both gaps and leaps in scores per gender. These tools are not only used to estimate said gaps, but also to determine whether these gaps are broadened or closed throughout the university education process. The results show that there are still gender gaps in favor of men in the area of mathematics, even in STEM programs. On the contrary, a gap in favor of women is noted in the area of writing, although a decline is also observed in reading comprehension skills for both genders. On the other hand, in terms of the global scores, women improve more than men as a result of their undergraduate experience. Finally, purely disciplinary competencies exhibit a notorious gender gap in favor of men, which should lead to future reforms in this type of programs.

Perception on Saber Pro tests results in the bachelor’s degree in basic education students

Introduction. The study addresses the competencies evaluated, that is, the Generic competencies and the Specific competencies, within the framework of the normativ estructure established by the Colombian Institute for the Evaluation of Education-ICFES, about the examination presented by future professionals in education. Objective. Carry out a study on the perception of the analysis of the results of the Saber Pro tests that students of the bachelor’s degree in Basic Education with emphasis on Mathematics, Humanities, and Spanish Language obtainedin 2019. Methodology. For this purpose, the official information of the results that are published in the ICFES databases is used to determine the level of comparison between the results of the related competencies with the data obtained in the institution and at thenational level. The study is developed undera non-experimental research approach of a descriptive type, where the percentage relationships of systematic comparison are established between the results of the program, the institution, branch, and the national average. Conclusions. Every student takes these standardized tests as a mandatory requirement to obtain their professional diploma, as contemplated by the policies in Colombia. In addition, it is worth mentioning that the individual and institutional efforts made to demonstrate excellent outcomes in a homogeneous test aimed at a heterogeneous population directed by State agencies. They leave out of the radar the different contextual, methodological conditions, and opportunities students have, to access information. Here we talk about future teachers.

Engineering design, complexity in its teaching

The trend toward increasing the design component in engineering curricula is part of an effort to better prepare graduates for engineering practice. The objective of the article is to review and contrast some methodological strategies of engineering design teaching at the university level to analyze the psychological theories in teaching practice and the temporal location of these courses within 5 years of the program. The results of the bibliographic review made allowed us to infer psychological postures, preferably cognitive and constructivist, and the location of design courses in the first semesters of engineering curricula. However, engineering design courses are recommended throughout the degree (taking as a reference some results of the Saber PRO tests) in which students participate in applied projects, and get closer to the industry, as more favorable strategies for the adaptation of future engineers to the particular demands of the national productive sector.

Value added in hierarchical linear mixed models with error in variables

In this paper, we propose a methodology to evaluate the effects on the value-added estimators when there is a measurement error in variables. These errors lead to biased estimators; in most cases, these estimators have a large variance. The proposed methodology is illustrated under two scenarios: First, simulation studies for discussing the effects on value-added estimates for some values of reliability. Second, an application for real data using a Colombian database that contains the scores of two standardized tests: Saber 11 and Saber Pro. In the first test, the students are evaluated in the 11th grade of high school. In the second test, the students are evaluated in the last two years of the undergraduate program. We also implement the estimation method taking into account measurement error and adapt the Bootstrap procedure to Hierarchical Linear Mixed Model with error in variables. The results show that measurement error affects the higher-education value-added and the estimator variance. This implies universities can be incorrectly classified. Therefore, we may assert that any given university is contributing to students’ progress when, in reality, this is not so.

Impact of COVID-19 on Students' Generic Skills - A Case Study in a University from a Developing Country

The education sector was considerably affected by the COVID-19 pandemic, risking the learning process and forcing governments to pull out contingency actions to ensure the students' development of generic and specific skills and guarantee education quality. These actions include the shift to online and hybrid (i.e., online and in-person) classes. This work assesses the pandemic effect on the generic skills of undergraduate students in a Colombian university. The study aims to determine the effect of COVID- 19 on the development of generic skills quantitatively. Two datasets were retrieved: i) A dataset with the scores obtained by the students in an institutional Generic Skills test and in the midterm tests; ii) A dataset with the students' scores in the Colombian standardized test for undergraduate students, called the Saber Pro test. Three analysis stages were performed: i) Univariate exploratory analysis; ii) Differential analysis to compare the No COVID vs. COVID scenarios; iii) A correlation analysis. Results showed that the scores of the Generic Skills and the Midterm tests increased significantly when comparing the two scenarios, except for the Written Communication. As for the Saber Pro test, only the scores for Written Comprehension, Quantitative Reasoning, and the global scores increased significantly. On the other hand, the correlation analysis showed a strong correlation only between the scores obtained at the Generic Skills and the Saber Pro tests for the English Proficiency skill. In addition, the analysis elucidated a weak correlation between the Generic Skills test's average and the Saber Pro's global score. Finally, the results prove that online education is a feasible alternative that offers students more flexibility to ensure the development of generic and specific skills.

A Multinomial Logistic Model for the Higher Education Value-Added with K-means Clustering

The implementation of state tests at the end of the different academic cycles, to pursue high standard education, is used by many countries in the world. For the Colombian case there are two classic exams: one of them is for all the students that are about to finish high school, called Saber 11, and the other one is for people that are finishing the university cycle, called Saber Pro. Both are measured by quantitative scores. This case study was considered because it was possible to know the Saber 11 and Saber Pro scores for the same individual. The machine learning k-means technique was used for defining different groups of Saber Pro scores. Then a multinomial logistic model was applied to estimate the probability of a student to fall in every one of those groups, given his own characteristics. In other words, the distribution probability of the Saber Pro results was produced through the different defined groups using k-means. This splitting technique is far better than the standard even subdivisions (like quartiles or quintiles), because it produces optimal distance-wise clusters that allows more natural statistical results. This work gives numerical insights regarding the performance for different student profiles, and shows that the educational achievement at the end of an academic program, is influenced by the initial academic level, the socioeconomic condition, and the university status, but also that more input variables must be analyzed and included. In particular, for the considered samples, the research concluded that the individuals with the least favorable socioeconomic condition have a tendency to underperform when compared with the rest of the population, and the students of the non-accredited institutions tend to obtain lower educational achievements than the accredited ones. Of course, in all the investigated groups, students with a high result in Saber 11 tend to achieve an outstanding performance in Saber Pro, regardless of their socioeconomic status or the accreditation status of the institution. On the other hand, the gender shows no general behavior or whatsoever.

Forecasting model with machine learning in higher education ICFES exams

<span>In this paper, we proposed to make different forecasting models in the University education through the algorithms K-means, K-closest neighbor, neural network, and naïve Bayes, which apply to specific exams of engineering, licensed and scientific mathematical thinking in Saber Pro of Colombia. ICFES Saber Pro is an exam required for the degree of all students who carry out undergraduate programs in higher education. The Colombian government regulated this exam in 2009 in the decree 3963 intending to verify the development of competencies, knowledge level, and quality of the programs and institutions. The objective is to use data to convert into information, search patterns, and select the best variables and harness the potential of data (average 650.000 data per semester). The study has found that the combination of features was: women have greater participation (68%) in Mathematics, Engineering, and Teaching careers, the urban area continues to be the preferred place to apply for higher studies (94%), Internet use increased by 50% in the last year, the support of the family nucleus is still relevant for the support in the formation of the children.</span>

Using Decision Trees to Predict Critical Reading Performance.

In Colombia, all undergraduate students, regardless of the professional training program they take, must complete the general competencies sections of the Saber Pro exam that include Critical Reading, Quantitative Reasoning, Citizen Competencies, Written Communication, and English. This paper presents the application of the classification technique based on decision trees in the prediction of the performance in the Critical Reading section presented by the students of the Pontificia Universidad Javeriana Cali in the years 2017 and 2018. The CRISP methodology was used. From the socioeconomic, academic and institutional data stored in the ICFES databases, a data repository was built, cleaned and transformed. A mineable view composed of 2052 records and 17 attributes was obtained. The J48 algorithm of the Weka tool was used to build the decision tree. The score obtained in the Critical Reading section of the Saber Pro exam was taken as a class. According to the results obtained, the Philosophy, Applied Mathematics, and Medicine programs stood out for having the best performance in this test. Among the predictive variables associated with performance in the Critical Reading skill are the faculty, the age group and the student's transportation index, as three important variables related to the good or low academic performance of the students of the Universidad Javeriana Cali. The knowledge generated in this research is constituted in quality information to support the decision-making process of the university directives in order to improve the quality of the higher education offered in this institution.

Exploring the Relationship between Saber Pro Test Outcomes and Student Teacher Characteristics in Colombia: Recommendations for Improving Bachelor’s Degree Education

This explanatory sequential mixed methods study explores the perceptions of academic and administrative managers responsible for teacher training at a public university in Colombia, as well as their views on improving such training after learning about the performance of teachers student teachers in the 2019 Saber Pro test, the differences in their test scores, and the relationships and statistical correlations between these outcomes and the students’ personal, family, socioeconomic and academic characteristics. Our findings show significant differences in the student teachers’ mean scores and performance when data are grouped according to personal, socioeconomic and academic conditions; a significant relationship between performance and student teacher characteristics; and correlations between critical reading scores and the other competencies assessed. Our data also highlight the lack of knowledge among academic and administrative managers about students’ life circumstances and the diversity of factors that may impact their performance; the importance of correlational data; the difference between expected and true outcomes; the inequity under which students seem to pursue their education; the limitations in access to resources; the training required for teachers to be able to analyze quantitative data and use specific software; the impact of teachers’ critical reading skills on student outcomes; the importance of data-driven decision-making; and the need for teachers to engage in quantitative research practices.

Modelo estadístico para conocer el nivel en Competencias Genéricas y Pensamiento Numérico en los Egresados del Ciclo Técnico Profesional en Barranquilla

El objeto de este proyecto es diseñar un Modelo Estadístico que permita conocer cuánto valor agrega la Institución Universitaria ITSA de la ciudad de Barranquilla (Colombia) en la transferencia de conocimientos en Competencias Genéricas y Pensamiento Numérico a Egresados del Ciclo Técnico Profesional mediante la predicción de resultados de pruebas estandarizadas profesionales a partir de las pruebas a nivel secundario versus los resultados reales desde el año 2013 al 2016. Se recolecto la información de un total de mil seiscientos (1600) egresados de la Institución Universitaria ITSA que presentaron la prueba Saber Pro en los periodos 2012-1 al 2015-3 de estos datos se tomó una muestra representativa de trescientos veintiséis (326) egresados de los programas del Ciclo Técnico Profesional de la Institución. Esta actividad se desarrolló mediante un requerimiento a la Oficina de Sistema y Egresados, también se utilizó la información suministrada por las plataformas virtuales que ofrece el ICFES en su página web. Este modelo permitió revisar e identificar falencias en los métodos y estrategias implementadas en la transferencia de conocimiento.

Perceptions of Students and Teachers in an English Course: Reconstructing a Syllabus

This initial research report aimed to explore the impact on students’ and teachers’ learning and teaching expectations of an English Course containing English for Specific Purposes (ESP), Saber Pro samples, and the use of graded audiobooks by interpreting students and teachers’ perceptions. Data was gathered through a survey and a focus group interview. The initial findings revealed that the current syllabus was designed efficiently in terms of layout, but certain aspects concerning content, language level, methodology, and evaluation needed to be adjusted. The interpretation of this academic community’s perceptions resulted in the reconstruction of the course syllabus.

Analysis of the scores obtained in the Saber Pro test by the students of the University of Pamplona, Colombia, during the year 2020

El Examen Nacional para la Calidad de la Educación Superior de Colombia Saber Pro, se propone como una herramienta para la evaluación de la calidad, este examen está destinado a estudiantes que completan estudios de pregrado y deben cumplir con este requisito para su graduación, consta de un conjunto de competencias genéricas que evalúa lectura crítica, razonamiento cuantitativo, competencias ciudadanas, comunicación escrita e inglés. Diversos estudios relacionan la capacidad intelectual, los factores sociodemográficos y académicos de los estudiantes de alto y bajo rendimiento en el examen. La investigación planteó como objetivo la realización de un estudio comparativo de los promedios de los puntajes en las competencias genéricas de las pruebas Saber Pro del año 2020 de las siete facultades de la Universidad de Pamplona, se incluyeron los datos de 3905 estudiantes. Se realizó un ANOVA para comparar los puntajes promedios de cada uno de los módulos, encontrándose diferencias estadísticamente significativas en los promedios analizados. Tanto la prueba a posteriori que se usó (prueba de Duncan) así como los promedios globales obtenidos por facultad, develan que la facultad de ciencias básicas fue la que mejor se destacó, seguida de la facultad de salud, luego le siguen las facultades de ingenierías y arquitectura, artes y humanidades, y ciencias agrarias, siendo las facultades de ciencias de la educación y ciencias económicas y empresariales las que obtuvieron los menores puntajes promedio, la facultad de ciencias básicas, se destaca en razonamiento cuantitativo con un puntaje de (163,26) seguido de competencias ciudadanas, lectura crítica e inglés, siendo comunicación escrita el módulo donde se obtuvieron los menores puntajes (137,09).

Level of English in Colombian Higher Education: A Decade of Stagnation

This article analyzes the results of the Saber Pro, the state exam for students completing higher education, during 2007–2017 concerning the English language section. This analysis uses the reports and databases from the Instituto Colombiano para la Evaluación de la Educación (ICFES) repository and explains the policy in its historical context. The results warn of a quite worrying picture between the goals established by the Ministry of Education and the final achievements. The level of English of future Colombian professionals is not only very low but also without improvement from its beginnings in 2007 to 2017. As a conclusion, it would be necessary to review, from the universities’ perspective, the language educational policy and propose bottom-up structural alternatives that allow a sustained impulse in teacher training, methodology, and curricular and pedagogical organization.

Efficiency of university education: A partial frontier analysis

This article investigates the efficiency of the university education using two linked databases (Saber Pro and Saber 11) from the Colombian Institute for Evaluation of Education (ICFES) corresponding to 2014. We use a non-parametric frontier approach that combines the “order m” technique with the concept of a meta-frontier to disaggregate students’ total efficiency in generic skills in quantitative reasoning, critical reading, and written communication, into the parts attributable to the students themselves and the university. The analysis is performed by academic programme and by education sector (public vs. private). Results indicate that most of the inefficiency of students in the assessment of generic skills in higher education is attributable to the students themselves and a significant number of students could improve their performance in the assessment in each of the academic programmes if they performed as efficiently as those located on the frontier. Furthermore, the inefficiency share of students varies between academic programmes and university sectors, with students in the private sector more inefficient than those in the public sector in some and less inefficient in others. This research constitutes the first application of the technique of “order m” with the approach of the meta-frontier for the analysis of educational efficiency using data at the student and university levels.

Quantitative reasoning and written communication. Competencies that evidence scientific thinking in university graduates

The purpose of this manuscript was to characterize and analyse both the differences and similarities of the results between the different careers in a public university in eastern Colombia, in the state tests Saber Pro 2017, in the modules quantitative reasoning and written communication, skills needed to evidence the level of scientific thinking, very useful for professionals in engineering and science, especially for professionals in mathematics and physics. In this research, a univariate descriptive analysis was carried out, followed by a principal component analysis. Engineering programs tend to perform better in quantitative reasoning and those of social sciences and education in written communication. These results indicate that graduates of engineering and science programs have a level of scientific thinking more related to the exact and natural sciences.

Does Quality Make a Difference for Higher Education Graduates in Colombia? Measuring Differences in Accredited and Non-accredited Institutions With Similar Financial Characteristics

In this paper we explore the difference of quality in higher education between accredited and non-accredited higher education institutions (HEIs) in Colombia. For that purpose we test if the difference of the National Exam of Student Performance (Saber Pro), which is our proxy for quality between institutional accredited and non-accredited institutions is statistically significant. The hypothesis is that indeed those HEIs that have been granted the institutional quality accreditation by the Colombian ministry of education should have better results in the national exam than those that not. We argue that for robustness of the results, it is necessary to control for confounding effects among comparable HEIs. For this purpose, we employ a propensity matching score approach based on common financial characteristics in order to avoid issues of selection bias. Our results find that the difference in performance between accredited and non-accredited institutions is positive and statistically significant. This difference between performances can be attributable to the substantial differences in the pattern of expenditures and asset investment of non-accredited HEIs.

Teaching strategies for strengthening written communication competence in university students

Las evaluaciones externas se han convertido en uno de los retos de los sistemas educativos en el mundo, Colombia no es la excepción, razón que ha llevado a investigadores desde diferentes áreas del conocimiento a indagar y gestionar estrategias pedagógicas innovadoras que fortalezcan procesos de aprendizaje, especialmente en lo relacionado con la lectura y escritura. La presente investigación se fundamentó en el análisis estadístico de los resultados de las Pruebas Saber 2014-2017 presentadas por estudiantes de una institución de educación superior de Bucaramanga, insumo relevante para la generación de estrategias que posibiliten el fortalecimiento del proceso lectoescritor a partir de la investigación acción en el aula, teniendo en cuenta el proceso de diseño, implementación, evaluación, reflexión y rediseño en estudiantes desde el primer semestre académico. Los resultados de la investigación apuntan a relacionar las deficiencias en el proceso lectoescritor desde la educación inicial y las dificultades en la gestión de sus emociones con el rendimiento académico en la educación superior.

Potencial De La Educación a Distancia Para Reducir Brechas De Aprendizaje En Educación Superior: Una Mirada Al Caso Colombiano

ABSTRACT One of the main needs of Latin America, besides education coverage, is to reduce learning gaps in higher education. In order to respond to such need, many strategies have been designed, worldwide, to impact the demand and provision of higher education programs. Among these strategies we can find grants and scholarships, financial aids, and others. Many have been the options that have been proposed to raise the provision of higher education programs, such as offering technology-supported education modalities like blended-learning, Massive Open Online Courses (MOOCs), and distance and online education. This paper aims to explore the potential of the provision of distance education programs to close learning gaps in Colombia’s higher education. In order to do so, we present an estimatation of the effects that students and programs characteristics can have on the results of the test “Saber PRO” (a Colombian standardized test that measures the level of competences developed by students in different higher education programs). This estimation was done applying a mixed model (Hierarchical Linear Modeling-HLM) based on the information of 451,358 students who were assessed in their last phase of their university major. Data were collected from 2016 to 2018. The econometrics estimations show that controlling individuals’ backgrounds and the characteristics of different academic programs, the variables that were highly associated with the reduction of learning gaps among individuals registered in distance programs compared to face-to-face programs are the students’ age and their cognitive backgrounds (grades in standardized tests taken before their undergraduate studies). The results show that we could expect distance education to become an efficient mechanism to contribute to the reduction of learning gaps in higher education. The quantitative findings suggest a series of research questions that invite to reflect about the potential of distance education in medium-income countries.