Traditional Evaluation Methods Research Articles

Automated Writing Evaluation: Users’ Perception and Expectations

Automated Writing Evaluation (AWE) is a great tool for writing evaluation and correction due to its ability to analyze writing. Recent debates in writing have focused on the distinctions between the traditional method of teaching writing and the implementation of AWE in the teaching of writing and choosing the better of the two. The objective of this study is to explore the learners’ perception of AWE among university students as well as their expectation of AWE implementation. The design of the study is descriptive in nature, employing the quantitative method with 72 university students participating in the experiment. This group underwent an intervention of 10 weeks being exposed to the AWE use. In this study, a questionnaire was developed as a tool for data collection. The implementation of AWE was viewed favourably by respondents. It was discovered that respondents consider AWE to be successful in terms of improving vocabulary use, language use, spelling, and punctuation. Respondents however, consider AWE to be less helpful in improving the substance and organisation of students’ ESL writing. The findings indicate that using technology in this digital learning environment cannot sufficiently support the process of ESL writing assessment and still necessitates traditional evaluation methods such as educators explaining and helping their students organize their writing and develop successful material in ESL writing.

Study on Contribution of Different Journal Evaluation Indicators to Impact Factor Based on Machine Learning

Sci-Tech journals have long served as platforms for academic communication and the collision of ideas, facilitating advanced inventions and major discoveries in science. The speed of development and future prospects of a field in the current era can often be reflected by the quality and quantity of cutting-edge papers published in Sci-Tech journals within that field. Currently, the impact factor of Sci-Tech journals is a widely recognized journal evaluation index that comprehensively reflects the quality and influence of the journals under evaluation. However, traditional journal evaluation methods based on statistical formulas, while relatively simple and fast, have certain limitations. They are not comprehensive enough and do not support the comparison between journals from different disciplines. In recent times, researchers have delved into using multiple suitable indicators for comprehensive journal evaluation, attempting to understand the role each indicator plays in the evaluation process, such as the rank sum ratio. Our paper presents a new dataset constructed from data from journals across various fields obtained from the China Wanfang Literature Platform. We endeavor to explore a series of novel journal evaluation methods based on machine learning, including deep learning models. With these 9 methods, we aim to determine the contribution of 17 journal evaluation indicators to the impact factor and identify important factors that can further enhance the quality and influence of Sci-Tech journals, which has great guiding significance for the future development of journals.

Evaluation of Site Suitability for Photovoltaic Power Plants in the Beijing–Tianjin–Hebei Region of China Using a Combined Weighting Method

As the construction of photovoltaic power plants continues to expand, investors have placed great importance on the suitability assessment of site selection. In this study, we have developed a multi-level evaluation system and proposed an AHP–XGBoost–GIS comprehensive evaluation model for assessing site suitability in the Beijing–Tianjin–Hebei region. The findings revealed that approximately 48,800 km2 (21.59% of the total area) constituted a suitable area in this region, surpassing previous research results. Generally suitable areas accounted for the largest proportion at 35.89%. Suitable areas in Beijing and Tianjin were relatively limited, with most of them concentrated in Baoding City, Zhangjiakou City, and Chengde City on the northwest side of the “Yanshan-Taihang Mountains”, where there are ample wastelands with gentle slopes conducive to constructing photovoltaic power plants southeast of the “Yanshan-Taihang Mountains”, and solar radiation conditions were better. However, ecological and economic factors restricted suitable areas resulting in smaller coverage including cities such as Handan, Xingtai, Qinhuangdao, and Tangshan, among others. This study successfully overcame limitations associated with traditional subjective evaluation methods by providing decision support for regional energy allocation planning and spatial planning efforts, as well as environmental protection endeavors.

Risk Evaluation of Elevators Based on Fuzzy Theory and Machine Learning Algorithms

Elevators have become an integral part of modern buildings, and technological advances have enabled the monitoring of their operational status through sensor technology. In response to the development of the elevator industry and the need for practical elevator operation risk evaluation, this paper proposes an elevator risk evaluation method based on fuzzy theory and machine learning methods. The method begins by establishing an elevator operation risk evaluation index system. The traditional fuzzy comprehensive evaluation method is then employed to evaluate the risk levels of the 50 elevators studied. The collected index data and labels (fuzzy comprehensive evaluation results) are used as inputs to train the support vector machine (SVM) model. To optimize the SVM model, the maximum information coefficient method, enhanced by the correlation-based feature selection (MIC-CFS) method, is employed to select features for the index input to the SVM model. The improved gray wolf algorithm (IGWO) method optimizes the SVM. Finally, the model’s performance is verified using new index data. The experimental results demonstrate that introducing machine learning methods for elevator risk evaluation saves time and effort while providing good accuracy compared to the traditional expert evaluation method. The optimization of the SVM model by IGWO and feature selection by the MIC-CFS method results in a more concise SVM model that converges faster during training, exhibits better stability, and achieves higher accuracy.

Gas-in-place content and occurrence state of deep shale gas in the Luzhou area, Sichuan Basin, China

The exploration and development of deep shale gas has made significant breakthroughs, but due to the characteristics of deep shale gas, there is no recognized and effective method of evaluating gas-bearing. To effectively determine the gas-in-place (GIP) content and occurrence state of deep shale gas, a total of 909 shales were collected for the study utilizing pressure-holding and conventional coring technologies from Wufeng-Longmaxi Formations in the Luzhou area, Sichuan Basin, China. The GIP content of the above shales was evaluated using various methods, including the USBM, polynomial fitting, ACF, MCF, process analysis and carbon isotope fractionation (CIF) methods. The accuracy of the above methods was verified through comparison with the gas content measured by pressure-holding coring. The results demonstrate that (1) the degassing content obtained from conventional coring ranges 0.06–2.13 cm3/g, averaging 0.64 cm3/g; the measured gas content obtained from pressure-holding coring ranges 2.19–11.91 cm3/g, averaging 7.16 cm3/g. The gas content obtained is significantly higher because pressure-holding coring samples have excellent sealing and minimal lost time. (2) The GIP content is underestimated by the USBM method, while the polynomial fitting, ACF, and MCF methods overestimate it. The process analysis and CIF methods can evaluate the GIP content of deep shales more accurately. Based on the comparative results of the adsorbed gas ratio calculated using the process analysis and the CIF methods (33.49% and 24.53%, respectively), it is recommended that the CIF method be given priority when evaluating deep shale gas-bearing. (3) The GIP content of deep shales in the Luzhou area calculated by the CIF method ranges 2.52–11.99 cm3/g, averaging 7.19 cm3/g; the adsorbed gas ratio accounts for 3.41%–53.32%, averaging 24.53%; and the lost gas ratio accounts for 54.01%–97.45%, averaging 86.38%. Compared with medium and shallow shale gas reservoirs in other areas, deep shale gas reservoirs in the Luzhou area have gas-bearing characteristics of high GIP content, low adsorbed gas ratio and high lost gas ratio, which restricts the applicability of traditional evaluation methods.

The cross-efficiency evaluation model with judgmental information: Cooperative game and optimal coordination perspective

In the actual efficiency evaluation process, it frequently occurs that the quantitative input and output data of each decision-making unit is difficult to obtain due to the limitation in data measurement and the absence of prior information. Instead, the form of judgment information can be selected to express the input–output state of the evaluated decision-making units compared with the reference level. Traditional cross-efficiency evaluation methods do not involve this case and cannot be well used in this situation. Based on the above problems, a multiplicative cross-efficiency evaluation model with judgmental information is constructed, and the optimal adjustment model is provided by introducing deviation variables to deal with the inconsistent issue of data features. In addition, for the case of multiple optimal solutions, the maximizing and minimizing secondary objective models of the multiplicative cross-efficiency are established respectively to reasonably identify the corresponding optimal weights. Simultaneously, in order to effectively depict the interaction between decision-making units, the characteristic functions of JM-cross efficiency cooperative game and the multiplicative Shapley value are established based on benevolent and aggressive cross-efficiency matrices. Moreover, in order to avoid the extreme ranking results that are oriented by high impact decision-making units, a series of optimization coordination models are developed, and an optimal efficiency ranking vector generation method with strong applicability and decision flexibility is proposed. The optimal efficiency ranking vector not only creates a precise match with the given Shapley index of the multiplicative cross-efficiency game, but also bounds between the minimum and the maximum average efficiency. Finally, the validity and rationality of the discussed models are illustrated by a concrete example.

A methodology for evaluating the relative pollution level of metal pollution in surface sediments of rivers based on the statistical results of relevant literatures covering world-wide rivers

Due to the intervention of human activities, the background values of riverbed sediment exhibit spatiotemporal variability, which can affect the accuracy of risk assessment results. Using risk assessment that do not rely on background values is an executable alternative to avoid such problems. In this study, a relative pollution level assessment (RPLA) method which was based on the statistical results of relevant literatures was proposed. This method includes a four-step data processing procedure to extract the evaluation indexes of relative pollution degree of pollutants in environment and a series of relative pollution status assessment methods to evaluate the overall relative pollution level and regional difference of world-wide rivers. To demonstrate how to use RPLA method, 310 relevant literatures covering world-wide rivers were selected. And the ambient background value (x̅), the world-wide threshold values (WWTV) and the relative pollution grades (LEVEL I ∼ IV) of 9 target metals (Cr, Ni, Cu, Zn, As, Cd, Pb, Sb and Tl) in riverbed surface sediments of world-wide rivers were extracted and used for evaluation. Moreover, the stability and applicability of RPLA method were evaluated. Results show that the evaluation results of RPLA method are robust and comparable with traditional evaluation method.

An Intelligent Chinese Driver Road Performance Assessment Model (RPAM) for Future Licensing Examinations

As the demand for private vehicles rises, there has been a gradual increase in the number of motor vehicles on the roads, leading to a growing concern about addressing traffic safety. Currently, China’s approach to assessing driver capabilities remains rooted in traditional, non-intelligent, and standardized evaluation methods based on examination subjects. The traditional model often falls short in providing constructive feedback on a driver’s real-world vehicle handling abilities, as many of the examination subjects can be practiced in advance to achieve a mere passing result, which, undoubtedly, increases the likelihood of underqualified drivers on the road. To address the issues of the current examination-oriented driver evaluation system in China, we propose a road performance assessment model (RPAM) that assesses drivers comprehensively by evaluating their road environment perception and vehicle operation abilities based on an in-vehicle and out-vehicle perception system. The model leverages patterns of the driver’s head posture, along with real-time information on the vehicle’s behavior and the road conditions, to quantify various performance metrics related to reasonable operation processes. These metrics are then integrated to generate a holistic assessment of the driving capabilities. This paper ultimately conducted tests of the RPAM on one actual examination route in Beijing. Two drivers were randomly selected for the examination. The model successfully computed the overall ability scores for each driver, validating the effectiveness.

A novel three-dimensional surface reconstruction method for fabric appearance evaluation

Traditional fabric appearance evaluation method mainly relies on subjective and manual operation, which is not uniform and time-consuming. In this paper, one set of self-developed 3D reconstruction system and method based on photometric stereo is proposed for objective fabric appearance evaluation. A photometric stereo image acquisition system equipped with multi-directional lighting sources is developed for the 3D surface reconstruction. Lighting and camera calibration is conducted to obtain the position of light sources and the intrisic parameters of digital camera. Our experimental results show that 3D fabric surface could be reconstructed effectively based on photometric stereo vision method, it has the potential to be used for fabric appearance evaluation.

A Comprehensive Study on Pain Assessment from Multimodal Sensor Data.

Pain assessment is a critical aspect of healthcare, influencing timely interventions and patient well-being. Traditional pain evaluation methods often rely on subjective patient reports, leading to inaccuracies and disparities in treatment, especially for patients who present difficulties to communicate due to cognitive impairments. Our contributions are three-fold. Firstly, we analyze the correlations of the data extracted from biomedical sensors. Then, we use state-of-the-art computer vision techniques to analyze videos focusing on the facial expressions of the patients, both per-frame and using the temporal context. We compare them and provide a baseline for pain assessment methods using two popular benchmarks: UNBC-McMaster Shoulder Pain Expression Archive Database and BioVid Heat Pain Database. We achieved an accuracy of over 96% and over 94% for the F1 Score, recall and precision metrics in pain estimation using single frames with the UNBC-McMaster dataset, employing state-of-the-art computer vision techniques such as Transformer-based architectures for vision tasks. In addition, from the conclusions drawn from the study, future lines of work in this area are discussed.

An Improved Gradient Enhancement Regression Tree Method to Evaluate the Innovation Capability of Energy Companies

In recent years, with the introduction of the innovation-driven development strategy, audit departments across the country have actively responded by arranging audit experts to assess the innovation capabilities of enterprises based on their information. They have also studied the impact of implementation on innovation indicators in energy enterprises, aiming to accurately implements to support enterprises and drive regional development. Traditional manual evaluation methods are inefficient and prone to human interference. By using a gradient boosting regression tree model to construct a scoring prediction model, instead of manual evaluation methods, both accuracy and efficiency can be ensured. Experimental results show that this prediction model outperforms other models such as random forest regression and can guarantee prediction accuracy.

Discordant pair analysis for sample efficient model evaluation

We present a new technique for assessing the effectiveness of a classification algorithm using discordant pair analysis. This method utilizes a known performance baseline algorithm and a large unlabeled dataset with an assumed class distribution to obtain overall performance estimates by only assessing the subset of examples that the algorithms classify discordantly. Our approach offers an efficient way to evaluate the performance of an algorithm that minimizes the human adjudications needed while also maintaining precision in the evaluation and in some cases improving the evaluation quality by reducing human adjudication errors. This approach is a computationally efficient alternative to the traditional exhaustive method of performance evaluation and has the potential to improve the accuracy of performance estimates. Simulation studies show that the discordant pair method reduces the number of adjudications by over 90%, while maintaining the same level of sensitivity and specificity.

Characterizing the fault performance of real-time precise satellite orbit and clock correction products

The a priori fault probability of the real-time precise satellite orbit and clock correction products is the critical parameter for integrity monitoring of precise point positioning (PPP). The traditional fault probability evaluation methods use the worst-case instantaneous user ranging error (IURE) as the conservative test statistic. However, the systematic biases of IURE contained in the worst-case IURE barely affect the PPP accuracy, which will undermine the statistical distribution of test statistic and reduce the sensitivity of fault detection. The fault probability will be estimated over-conservatively for the traditional methods. By clarifying the sources of the systematic biases, a new test statistic is constructed by deliberately removing the systematic biases of IURE originated from satellite orbit and clock errors. One-year Global Positioning System correction products evaluation results have demonstrated that the constructed test statistic follows the Gaussian distribution with the decreased uncertainty and the improved fault detection sensitivity. The real-world data experiments have shown that the a priori probabilities of the satellite fault and the constellation fault are at the order of 10−4 and 10−5 levels, respectively.

The Application and Potential of Artificial Intelligence Participating in College Teaching Evaluation

The dominance could be applied to many more areas as new artificial intelligence (AI) advances. There are great defects in the traditional evaluation methods of higher education. The evaluation of people is dominant, resulting in excessively subjective results and wasting human resources. These issues adversely affect many stakeholders. Therefore, the combination of the objectivity of artificial intelligence and the traditional method of teaching evaluation has an exploratory significance for the development of teaching and emphasizes the technical ability that the new artificial intelligence can realize. The purpose of this study is to explore the potential of a new method for artificial intelligence in higher education evaluation. Based on the analysis of existing experimental case reports and related studies by the method of literature synthesis, it is certain that the new artificial intelligence will help improve the quality of teaching evaluation in higher education. This has a positive impact on all fields, including schools, teachers, and students. The teaching objectives of the school can be better and more efficiently accomplished under the evaluation system. This study confirms the feasibility of incorporating artificial intelligence into university evaluation.

Process-driven susceptibility assessment of glacial lake outburst debris flow in the Himalayas under climate change

Global warming is causing glaciers to retreat and glacial lakes to expand in the Himalayas, which amplifies the risk of glacial lake outburst debris flows (GLODFs) and poses a significant threat to downstream lives and infrastructures. However, the complex interplay between GLODF occurrences and associated indicators, coupled with the lack of a comprehensive susceptibility indicator system that considers the entire GLODF process, presents a substantial challenge in assessing GLODF susceptibility in the Himalayas. This study proposes a process-driven GLODF susceptibility assessment indicator system responding to climate change that considers the complete process of GLODF formation, incorporating relevant parameters about upstream, themselves, and downstream of glacial lakes. Furthermore, to mitigate subjective factors associated with traditional evaluation methods, we developed three novel hybrid machine-learning models by integrating classic machine-learning algorithms with the whale optimization algorithm (WOA) to delineate the distribution of GLODF susceptibility in the Himalayas. All the hybrid models effectively predicted the GLODFs occurrence, with the WOA-SVC model demonstrating the highest prediction accuracy. Approximately 34% of the catchments exhibit high and very high susceptibility levels, primarily concentrated along the north and south sides of the Himalayan ridge, particularly in the eastern and central Himalayas. Indicators capturing the physical formation process of hazards, such as topographic potential (highest relative importance value of 40%), can precisely identify GLODF. A total of 128 catchments pose potential transboundary threats, with 24 classified as having a very high susceptibility level and 25 as having a high susceptibility level. Notably, the border region between China and Nepal is a prominent hotspot for transboundary threats of GLODF. These findings can provide valuable clues for disaster prevention, mitigation, and cross-border coordination in the Himalayas.

Intercultural Communication Competence Improvement of Students under the International Education Management Model

With the acceleration of globalization, international education has gradually become the core content of higher education. In this context, students have encountered lots of intercultural challenges, making the evaluation and improvement of their intercultural communication competence (ICC) a key issue. It is difficult for traditional evaluation methods, such as questionnaire surveys, interviews, and teacher evaluations, to meet the needs of current educational institutions because they are subjective, time-consuming, and inefficient. After describing the ICC evaluation process of students under the international education management model, this study proposed a new automatic evaluation method based on the dual-scale convolutional neural network (CNN) model and the bi-directional long-short-term memory (Bi-LSTM) model of characters and words. The new method evaluated students’ ICC more accurately and objectively, providing a new perspective and tool for future research and educational training.

Estimation of Fracture Size and Probability Density Function by Setting Scanlines in Rectangular Sampling Window

Abstract Rock masses are very important materials in geotechnical engineering. In engineering rock mass, fracture is the relatively weak part of mechanical strength in rock mass and is the most important factor controlling the deformation, damage, and permeability of rock mass. Therefore, investigating fractures is very important for characterizing rock mass. This paper proposed a new approach by using uniformly equidistant orthogonal scanlines. Within the study context, the solution formula of fracture size is derived by establishing the space intersection model of arbitrary fracture and scanline, rectangular window, and a rectangular box with a rectangular window. Then, fractures were randomly generated in a certain size cube and compared with the traditional Kulatilake trace length integral evaluation method. The study results have shown that the proposed method is more reasonable and accurate. Then, this method was applied to an adit of Songta Hydropower Station. Finally, a new fracture diameter probability density estimation method was proposed, the fracture diameter of the normal distribution was verified, and the parameters of the probability density function obtained by the scanlines method were in agreement with the initial set parameters. In summary, the proposed scanlines method can well estimate the mean value of the fracture diameter and the probability density function of the fracture size.

Tor network anonymity evaluation based on node anonymity

In order to address the shortcomings of traditional anonymity network anonymity evaluation methods, which only analyze from the perspective of the overall network and ignore the attributes of individual nodes, we proposes a dynamic anonymity model based on a self-built anonymous system that combines node attributes, network behavior, and program security monitoring. The anonymity of evaluation nodes is assessed based on stable intervals and behavior baselines defined according to their normal operating status. The anonymity of the network is evaluated using an improved normalized information entropy method that refines anonymity evaluation to the anonymity of each node and expands the dimensionality of evaluation features. This paper compares the effectiveness of our proposed method with static framework information entropy and single indicator methods by evaluating the degree of anonymity provided by a self-built Tor anonymous network under multiple operating scenarios including normal and under attack. Our approach utilizes dynamically changing network anonymity based on multiple anonymous attributes and better reflects the degree of anonymity in anonymous systems.

The mediating role of AKT/ERK/JNK signaling on the malignant phenotype of microcystin-LR in gastric adenocarcinoma cells

Microcystin-leucine arginine (MC-LR), a widely distributed and highly toxic environmental pollutant, plays crucial roles in cancer malignancy by activating characteristically toxic signaling pathways. Traditional animal-based toxicity evaluation methods have proven insufficient for identifying the specific role of these signaling pathways. Therefore, this study aimed to uncover the regulatory relationship between the toxic pathways and the progression of gastric cancer (GC). The findings provide novel avenues for conducting in vitro toxicity tests based on the investigated pathways. We found that MC-LR promoted the migration and invasion of SGC-7901 cells while simultaneously inhibiting their apoptosis in a dose-dependent manner. This observed cytotoxicity was primarily mediated through the AKT, JNK, and ERK signaling pathways. By using a mediation analysis model, we determined that AKT and ERK exhibited competitive effects in MC-LR-treated GC malignancy, while AKT and JNK acted independently from one another. This study establishes an in vitro toxicity test model of MC-LR based on toxicity-related pathways and underscores the pivotal roles of AKT, ERK, and JNK signaling in MC-LR toxicity. The findings offer a novel, fundamental framework for conducting chemical toxicity risk assessment.

Application of fuzzy analytic hierarchy process – multi-layer fuzzy inference system in product design evaluation

To enhance profitability of production cycle, any manufacturer needs effective product design and evaluation procedures. This study proposed a novel approach combining fuzzy analytic hierarchy process (FAHP) and multi-layer fuzzy inference system (MFIS). It is based on consumer online comments to improve product design. This method possesses several advantages over traditional design evaluation methods. It can quickly acquire consumer preferences, effectively handle multi-criteria decision problems and integrate uncertain and fuzzy information. The Fuzzy Analytic Hierarchy Process–Multi-layer Fuzzy Inference System (FAHP-MFIS) involves the following steps: screening of factors, hierarchical modeling, quantification of qualitative factors, and conversion of these factors into quantitative values. It is a knowledge-based system that uses logical rules. The quantity and levels of input variables directly correlates with the quantity of logical rules. However, with multi-factor and multi-level inference, the establishment of a rule base becomes impractical due to the overwhelming number of rules. To address this issue, the Taguchi orthogonal table is applied to reduce the number of logical rules. Taking a household oxygen generator for medical devices as an example, the proposed model is applied in real-time. In the first stage, web crawlers are used to collect user reviews of the household oxygen generators on large e-commerce platforms. Latent Dirichlet Allocation (LDA) models are used to screen for principal and sub-factors in the second stage. Then, sub-factors of the FAHP screening are used as inputs, and the principal factors are used as outputs. In the third stage, priority indicators are established based on principal factors such as Appearance, Basic Function, and Advanced Function. Established evaluation models are then used to rank the selected designs. The results show that the higher the priority index value of the product design scheme, the better the scheme, and vice versa. This study holds significant reference value in aiding enterprises to enhance the efficiency of their manufacturing cycle and determining the direction of product design and innovation with improved pace and accuracy. Moreover, it can be applied to other fields such as supply chain management, risk assessment, and investment decisions.