Limited Number Of Criteria Research Articles

Using Fleiss’ Kappa Coefficient to Measure the Intra and Inter-Rater Reliability of Three AI Software Programs in the Assessment of EFL Learners’ Story Writing

Story writing is a valuable skill for EFL learners, as it allows them to express their creativity and practice their language proficiency. However, assessing story writing can be challenging and time-consuming for teachers, especially when they have to deal with large classes and multiple criteria. Therefore, some researchers have explored the use of artificial intelligence (AI) tools to automate the assessment of story writing and provide feedback to learners. However, the reliability of these tools is still questionable. This study aimed to compare the intra- and inter-rater reliability of three AI tools for assessing EFL learners' story writing: Poe.com, Bing, and Google Bard. The study utilized quantitative tools to answer the research questions, namely, calculating the Fleiss' Kappa coefficient using the Datatab software program (available on datatab.com). The study sampled 14 written pieces by EFL Libyan adult learners, the pieces used were stories built around a prompt provided by the teacher. The assessment was done using two criteria, one including the measurement of students' creativity, and the second was done focusing only on the linguistic aspect of the students' writings. With the creativity criterion, the results of the study show that Poe's intra-rater reliability was 0.01 (slight), while Bing's was 0.2 (fair), Bard's was 0.2 (fair). This shows that Poe is the least reliable assessment tool among the three. For the inter-rater reliability, there were three assessments done to the same 14 sampled pieces to check the consistency of the results. In the first attempt the inter-rater reliability was 0.04 (slight), the second assessment it was 0.01 (slight), on the third time it was -0.03 (no agreement). There was a decrease in the consistency and reliability of scores over time. Without the creativity criterion, the results show that Poe's inter-rater reliability level was 0.05 (slight), while Bing's was -0.02 (no agreement), and Bard's was 0.01 (slight). Here, it is shown that Bing was the least reliable. For the inter-rater reliability, the three assessments made by the three software applications were compared. There were three assessments done on the same 14 sampled pieces to check the consistency of the results. In the first attempt, the inter-rater reliability was 0 (slight), the second assessment it was -0.1 (no agreement), on the third time it was -0.13 (no agreement). There was a decrease in the consistency and reliability of scores over time. The three applications performed in a reliable way to a certain extent without the exclusion of the creativity criterion, this goes against the common belief that AI software cannot assess creativity. Still, the results of the reliability measurements with the creativity criterion show that the assessment scores are not statistically significant, and there's a high probability that the observed agreement is due to random chance. Some limitations of this study were the small sample size, the limited number of criteria, and the lack of human raters for comparison. Future research could involve more participants, more criteria, more AI tools, and human raters to provide a more comprehensive and reliable evaluation of AI tools for assessing EFL story writing.

Making Room for Everyone’s Talent

See RECORDING. Current societal challenges demand more cooperation and a multidisciplinary approach from scientists. Putting into practice the shared ambition in Dutch academia requires a modernisation of the system of Recognition & Rewards. The modernisation should improve the quality of each of the key areas in academia: education, research, impact, leadership and (for university medical centres) patient care. Why do we think a change in recognition and rewards is needed? We see a mismatch between what we deem important in academic work and what we reward academic staff for. Careers depend heavily on research performance, which is measured by a limited number of criteria. We need a better balance in how we recognize and reward academics to help us achieve excellent education, research, impact, and leadership, as well as the highest level of patient care in our university medical centres. We want to make Room for everyone’s talent! In the Dutch Recognition & Rewards programme we work in cooperation with all Dutch research universities, university medical centres, reputable research institutes, the Royal Academy, and research funders. We aim for a healthy and inspiring environment for all our staff, where all talents are valued. In this presentation I will explain what we want to change in academic career assessment in the Netherlands, and how we hope to initiate the desired cultural change. This change is a fundamental change of beliefs, not just a change in the rules of the game. To achieve this, a broad dialogue in academia is needed. We think that sharing best practices and experimenting will initiate the desired movement. Every university and research organisation in the Netherlands has set up a high-level Recognition & Rewards committee. The various committees are working enthusiastically to stimulate the intended culture change at an institutional level. There is a great and inspiring diversity of approaches. Experimenting, inspiring, co-creating, sharing best practices, and learning from each other are central in the joint programme. At a national level, we also monitor the commonality and ensure connections between the developments at the universities, university medical centres, research institutes, and research funders. We are aware that the Netherlands is only a very small country. We cannot change academic career assessment on our own. We need to work together all over the world to change the recognition and rewards of academics. We hope you are curious, critical, and open about your concerns and become involved!

A proposed multi criteria indexing and ranking model for documents and web pages on large scale data

Due to the expansion of data, search engines encounter different obstacles for retrieving better relevant content to user’s search queries. Consequently, various retrieval and ranking algorithms have been applied to satisfy the result’s relevancy according to user’s needs. Unfortunately, indexing and ranking processes face several challenges to achieve highly accurate results, since most of the existing indexes and ranking algorithms crawl documents and web pages based on limited number of criteria that satisfy user needs. So, this research studies and observes how search engines work and which factors contribute to higher rankings results. The research also proposes a Multi Criteria Indexing and Ranking Model (MCIR) based on weighted documents and pages which depend on one or more ranking factors, aiming at building a model that achieves high performance, better relevant pages, and the ability to index and rank both online/offline pages and documents. The MCIR model was applied on three different experiments to compare documents and pages results in terms of ranking scores, based on one or more criteria of user’s preferences. The results of applying MCIR model proved that final pages ranking results depending on multi-criteria are better than using only one criterion, and some criteria have great effect on ranking results than other criteria. It was also observed that the MCIR model achieved high performance on indexing and ranking dataset up to 100 gigabytes.

Control as part of the management system of internal processes

Enterprise management – is the process of planning, organization, motivation, control and regulation of personnel actions, setting strategic goals and tactical objectives of the enterprise, making management decisions and ensuring their implementation.
 Control, in the process of managing the activities of the enterprise, is an extremely important element in the system of mechanisms that provide the efficiency of the enterprise. Control is a process that helps an organization achieve its goals. Control continues the planning process and monitors the implementation of plans. One of the essential functions of control is to determine the reserves for new management decisions.
 The authors in the article define the organizational and legal forms of ensuring the implementation of internal control procedures. The authors present one of the elements of the management system of the relevant processes (control) and describe its factors that affect them.
 The authors give a clear definition of the concept of control system at the enterprise (internal control) as part of the management system of internal processes and characterize the components of the internal control structure, methods of their analysis and analyze its quality and feasibility in the current realities of enterprises.
 Enterprise financial management includes the organization and control of cash inflows from sales of products or collection of amounts under previous agreements for services rendered, as well as cash inflows from securities, and etc. The most important tasks of financial management are also the payment of supplies of raw materials and materials intended for production; payments on invoices payable for previously purchased goods; payments for operating costs (advertising, insurance, etc.); payment of wages to employees of the enterprise; payment of taxes and other payments to the budget and extrabudgetary funds.
 The article determines that the effectiveness of control of such an organizational system as the enterprise management system should be characterized by special indicators. These indicators are influenced by a large number of external and internal factors, and it is desirable to assess them with a limited number of criteria.

Methodical approaches to choosing the sites for underground energy facilities

The work is aimed at studying approaches and developing principal regulations when choosing the sites intended for constructing underground nuclear small power plants in geological formations. The research methodology provides for analysis and summarizing international and Russian experience in choosing the sites for the placement of radiation hazardous facilities. Results. The work gives the approaches when choosing the sites intended for radiation hazardous facilities proposed by IAEA in Russia and abroad. The formation of criteria complete, their specific weight and an integrated assessment of impacts of an energy object on the environment is of importance in the site choosing procedure. The several criteria characterising the site are different for the surface and underground placement of a nuclear plant. The underground placement of a nuclear plant is a determining factor in protecting against undesired external and internal impacts. The authors have proposed an algorithm of choosing the sites for underground small nuclear power plants based on a limited number of criteria. Conclusions. The approaches used for choosing the sites have similar algorithms. The site choosing procedure pays a lot of attention to formation of criteria and an integrated assessment of impacts of an energy object on the environment. The underground placement of energy objects improves their security that allows decreasing a number of observed criteria when choosing the sites.

Patients with high left ventricular filling pressure may be missed applying 2016 echo guidelines: a pilot study.

2016 guidelines for the echographic evaluation of left ventricular filling pressure (LVFP) proposed a single algorithm with limited number of criteria (E/A ratio, tricuspid regurgitation velocity, left atrial volume index and average E/e') mainly related to left atrial pressure. Pulmonary venous flow analysis, evaluating more specifically left ventricular end diastolic pressure (LVEDP) has been withdrawn. We aim to evaluate the proportion of patients diagnosed with normal LVFP according to 2016 recommendations, despite an abnormal pulmonary venous flow profile suggesting high LVEDP. We prospectively studied patients with stable ischemic cardiomyopathy and aortic stenosis, before cardiac surgery. Extensive echocardiography was performed including pulmonary and mitral A wave durations. We included 76 patients (mean age 72 ± 10years, 78% were men), 37 (49%) with aortic stenosis and 22 (29%) with ischemic cardiomyopathy. Mean left ventricular ejection fraction was 67 ± 11%. Applying recommendations, 58 patients had normal LVFP and 15 patients had high LVFP. Among the 58 patients with normal LVFP, 26 patients had Apd-Amd duration > 30ms highly suggestive of high LVEDP. These patients had higher LV mass (112 ± 30g/m2 vs. 86 ± 20g/m2, p = 0.004) and shorter A wave duration (120 ± 13.6ms vs. 132 ± 16.5ms, p = 0.006) as compared to the remaining 15 patients with concordant evaluation (normal LVFP and normal Apd-Amd). In the present study, we found that 26/58 patients with low LVFP according to the 2016 recommendations had Apd-Amd suggestive of high LVEDP. Pulmonary venous flow should be added to the algorithm, particularly in patients with unexplained symptom, high LV mass or truncated mitral A wave.

Adaptive Routing Protocol for Lifetime Maximization in Multi-Constraint Wireless Sensor Networks

Routing in wireless sensor networks plays a crucial role in deploying and managing an efficient and adaptive network. Ensuring efficient routing entails an ever-increasing necessity for optimized energy consumption and reliable resource management of both the sensor nodes and the overall sensor network. An efficient routing solution is characterized by its ability to increase network lifetime, enhance efficiency, and ensure the appropriate quality of service. Therefore, the routing protocols need to be designed with an ultimate objective by considering and efficiently managing many characteristics and requirements, such as fault tolerance, scalability, production costs, and others. Unfortunately, many of the existing solutions lead to higher power consumption and communication control overheads, which not only increase network congestion but also decrease network lifetime. In addition, most of these protocols consider a limited number of criteria, in contrast to the complexity and the evolution of WSNs. This paper presents a new adaptive and dynamic multi-criteria routing protocol. Our protocol operates in multi-constraint environments, where most of the current solutions fail to monitor successive and continuous changes in network state and user preferences. This approach provides a routing mechanism, which creates a routing tree based on the evaluation of many criteria. These criteria can cover the topological metrics of neighboring nodes (the role of the nodes in intracommunications, connections between different parts of the network, etc.), the estimated power consumption to reach each direct neighbor, the path length (number of hops to the sink), the remaining energy of individual sensor nodes, and others. These criteria are controlled and supervised dynamically through a detection scheme. In addition, a dynamic selection mechanism, based on multi-attribute decision-making methods, is implemented to build and update the routing tree. In response to changes in the network state, user preferences, and application-defined goals, the election mechanism provides the best routing neighbor between each node and the sink.

Using Partial Least Squares Structural Equation Modeling in Tourism Research

Although the number of empirical applications of partial least–squares structural equation modeling (PLS-SEM) in tourism has increased in the last two years, Assaker, Huang, and Hallak have conducted the only assessment on the use of PLS-SEM in four studies and with a limited number of criteria. Thus, this study aims to critically analyze how the PLS-SEM method has been applied in 44 articles published in 11 leading tourism journals from 2000 to 2014 in terms of four key criteria: (1) themes explored and main motivations for using PLS-SEM; (2) characteristics of proposed models; (3) how the models were evaluated; and (4) the use of more advanced analyses within the method. The findings revealed that although applications in tourism have improved in recent years, problematic aspects in the application of PLS-SEM in tourism research still exist. The article provides suggestions on how to improve the use of PLS-SEM in future tourism applications.

Selecting Natural Fibers for Bio-Based Materials with Conflicting Criteria

Proper material selection enhances both desired physical and meta-physical properties of the end user products. Due to the growing awareness of the environmental issues as well as the industrial sustainability, the Natural Fiber Composites (NFCs) have recently emphasized to be implemented in various industrial applications. The final product features of such composites depend on the integrated characteristics of both matrix and fillers properties. Selecting an appropriate natural fiber type to form an NFC is affected by several factors and criteria. Up-to-date, natural fibers are evaluated regarding limited number of criteria. This study, introduces a proper integrated evaluation and selection scheme for the natural fibers and their capabilities considering integrated evaluation. Results demonstrate that better evaluation of natural fibers regarding wide range of criteria would lead to better decisions regarding selecting the suitable NFCs for industrial applications and enhance achieving better performance. Such evaluations should consider combined economic and environmental characteristics as well as technical ones. New potential fiber types can be discovered and utilized through better evaluations using combined desired criteria.

Combined Multi-criteria Evaluation Stage Technique as an Agro Waste Evaluation Indicator for Polymeric Composites: Date Palm Fibers as a Case Study

The final features of natural fiber composites (NFCs) depend on the integrated characteristics of their constituents. In the industry today, natural agro waste fibers are evaluated using a limited number of criteria. In this work, a combined multi-criteria evaluation stage technique (CMCEST) is introduced as a simple efficient systematic indicator to enhance evaluation of the available natural agro wastes for polymeric composites. In this proposed technique, criteria affecting the proper selection of natural agro waste fibers were combined and divided into sequence stages as follows: single-evaluation-criterion (SEC), combined-double-evaluation-criterion (CDEC), combined-triple-evaluation-criterion (CTEC), etc. These stages are based on combined physical, mechanical, and economic evaluation criteria and can be extended to several further stages to include other beneficial characteristics. The effectiveness of this technique was demonstrated by evaluating coir, date palm, jute, hemp, kenaf, and oil palm fibers simultaneously. This combined evaluation criteria can lead to more informative decisions regarding selection of the most suitable fiber type for polymeric composites. The CMCEST enhancements can reveal new potential fiber types through better evaluation schemes, help achieve clearer indications of the capabilities of available agro wastes to enhance composites, and determine proper ecological waste management practices. Utilizing the proposed technique, the date palm fiber type was found to be quite promising due to beneficial characteristics revealed in CTEC, which provides a reasonable, cheap, and eco-friendly alternative material suitable for different applications.

The Case For Yet <italic>Another</italic> Digital Preservation Evaluation Tool

Early 2013 DEN and PACKED launched a score model for the preservation of digital collections (scoremodel.org). The model is intended as a self-evaluation tool: organizations that host and/or manage digital collections can use the Score model to identify the potential threats to the long-term viability and accessibility of digital collections. In contrast to OAIS-oriented tools such as ISO 16363, the Score model is aimed towards smaller institutions, who have limited technical and organizational competencies. The criteria in this checklist are grouped around seven sections, in each of which a limited number of criteria are given, using as little technical jargon as possible. The resulting report can be used as a planning tool to systematically tackle the threats to digital preservation. By introducing risk levels, organizations are triggered to prioritize actions and to focus on measurements that are most important to take.

Can we simplify the hospital accreditation process? Predicting accreditation decisions from a reduced dataset of focus priority standards and quality indicators: results of predictive modelling

ObjectivesAccreditation in France relies on a mandatory 4-year cycle of self-assessment and a peer review of 82 standards, among which 14 focus priority standards (FPS). Hospitals are also required to...

Using ANP and AHP for the supplier selection in the construction and civil engineering companies; Case study of Iranian company

The experts of the construction and civil engineering companies are responsible for selecting an appropriate supplier. Evaluation and selection of appropriate suppliers may decrease quality deficiency, cost overrun, environmental problems and etc. In this research a decision support system is developed to select appropriate supplier for construction and civil engineering companies. The effective criteria on the supplier selection process have been investigated and accommodated to internal structure of the Iranian construction and civil engineering companies by questionnaire survey. For this purpose, the current research carries out among logistics managers of the first grade Iranian construction and civil engineering companies or those with having ISO certification. After completion of questionnaires, the effectiveness of criteria is evaluated using t-student test. Five criteria are excluded, and 18 criteria are selected as effective criteria. To select appropriate supplier, the AHP and ANP approaches are utilized simultaneously. Since, these two methods use a limited number of criteria, it is essential to prioritize criteria. So, Friedman test is applied and finally five of the most important criteria have been employed. Unfortunately environmental issues are not attended by construction and civil engineering companies. To illustrate concept and performance of proposed decision support system three candidate suppliers are ranked using AHP and ANP models. Although the final results of AHP and ANP methods are the similar, the values of variation between suppliers are different. Therefore, it is recommended to employ the ANP method considering internal relations between criteria.

The Use of Partial Least Squares Structural Equation Modeling in Strategic Management Research: A Review of Past Practices and Recommendations for Future Applications

Every discipline needs to frequently review the use of multivariate analysis methods to ensure rigorous research and publications. Even though partial least squares structural equation modeling (PLS-SEM) is frequently used for studies in strategic management, this kind of assessment has only been conducted by Hulland (1999) for four studies and a limited number of criteria. This article analyzes the use of PLS-SEM in thirty-seven studies that have been published in eight leading management journals for dozens of relevant criteria, including reasons for using PLS-SEM, data characteristics, model characteristics, model evaluation and reporting. Our results reveal several problematic aspects of PLS-SEM use in strategic management research, but also substantiate some improvement over time. We find that researchers still often do not fully make use of the method's capabilities, sometimes even misapplying it. Our review of PLS-SEM applications and recommendations on how to improve the use of the method are important to disseminate rigorous research and publication practices in the strategic management discipline.

A Statistical Approach to the Validation and Optimisation of Geoheritage Assessment Procedures

The assessment of geoheritage is a crucial task for the establishment of ranks according to their value, an important step for the elaboration of catalogues and the implementation of protection and use programmes. One important problem that permeates the different stages of geoheritage assessment is subjectivity in the selection and application of criteria. Evaluation methods frequently used are based on the assignment of values through the expert’s judgement (direct methods). In the last few decades, parametric methods, based on the measurement of specific features of geomorphosites have been increasingly applied. This type of procedure allows different operators to obtain similar results if certain criteria are accepted, but it requires more time and effort. A parametric method is proposed here based on three sets of criteria: intrinsic quality, potential for use and protection needs. The method has been tested on a series of coastal geomorphosites. Specific criteria, parameters to express them and value ranks are proposed. A statistical approach has been applied to the criteria to identify the most significant ones and their relative weights. The analysis enabled reduction of the number of criteria, eliminating redundancies. The new set of criteria was applied to assess the value of geomorphosites located in the north of Spain. The classification was compared with the ones obtained by both the parametric method using a larger number of parameters and a direct evaluation applied to the same sites. Results show a quite good coincidence. It is concluded that this approach makes it possible to obtain reliable classifications of geomorphosites based on clearly defined criteria and replicable procedures, but with a much lower effort, using a limited number of criteria.

Eurocode 8 Compliant Real Record Sets for Seismic Analysis of Structures

Among all the possible options to define the seismic input for structural analysis, natural recordings are emerging as the most attractive. Easily accessible waveform databases are available and evidence shows that only a relatively limited number of criteria has to be considered in selection and scaling to get an unbiased estimation of seismic demand. Like many codes worldwide, Eurocode 8 (EC8) allows the use of real ground-motion records for the seismic assessment of structures. The main condition to be satisfied by the chosen set is that the average elastic spectrum does not underestimate the code spectrum, with a 10% tolerance, in a broad range of periods depending on the structure's dynamic properties. The EC8 prescriptions seem to favour the use of spectrum-matching records, obtained either by simulation or manipulation of real records. The study presented herein investigates the European Strong-Motion Database with the purpose of assessing whether it is possible to find real accelerogram sets complying with the EC8 spectra, while accounting for additional constraints believed to matter in the seismic assessment of buildings, as suggested by the current best practice. Original (un-scaled) accelerogram sets matching EC8 criteria were found, for the case of one-component (P-type) and spatial sets (S-type), for the spectra anchored to the Italian peak acceleration values. The average spectra for these sets tend to be as close as possible to the code spectrum. Other sets, requiring scaling, have been found to match the non dimensional (country-independent) EC8 spectral shape. These sets have also the benefit of reducing, in respect to the un-scaled sets, the record-to-record variability of spectra. Combinations referring to soft soil, stiff soil, and rock are presented here and are available on the internet at http://www.reluis.it/

Definition of criteria for overall assessment of animal welfare

AbstractWelfare is multidimensional, comprising good health, comfort, expression of behaviour, and so on. Its overall assessment therefore requires a multicriteria evaluation. The set of criteria shall be exhaustive (no missing item), minimal (only necessary items), agreed by stakeholders, and legible (a limited number of criteria). Furthermore, the interpretation from one criterion shall not depend on that from another. We propose a set of 12 subcriteria grouped into four criteria: feeding, housing, health and optimised emotional states. This work will assist in developing measures to be used on-farm to form a European standard for overall assessment of animal welfare.

Biological and ecological characteristics of invasive species: a gammarid study

Knowledge of characteristics helpful in screening potential invaders and in elaborating strategies to limit their success is highly desirable. We focused on gammarid amphipods from Western Europe and North America to discover biological and/or ecological traits that may explain successful invasion by these species. Two typologies were considered: an analytical one, with groups built on the basis of biological or ecological similarities, and an empirical one, with groups constituted a priori according to a species’ invasive status and its fresh or brackish water origin. The results obtained are discussed in the light of three hypotheses that may influence invasiveness: biotic potential, species size and euryoeciousness. The analysis revealed a particular ecological profile for invaders, with a strong influence of salinity tolerance, but no typology was found based on biological characteristics. Invasiveness cannot be predicted from a limited number of criteria, and is the result of a combination of several characteristics. Invasive species therefore exhibit a particular ecological profile rather than a biological one, contrary to most classical explanations.

Scholarly use of internet‐based electronic resources

AbstractThis research examines the use of Internet‐based electronic resources (e‐sources) by a group of library and information science (LIS) scholars. It focuses particularly on how scholars use, cite, and evaluate e‐sources during the research process. This research also explores the problems scholars encounter and concerns they have when using e‐sources for research. The following approaches were used to collect data for the investigation: (a) a longitudinal analysis of e‐source citations in eight LIS journals from 1991 through 1998; (b) a survey of editors of the eight journals; and (c) a survey of 201 authors with articles to be published in the eight journals. The longitudinal analysis of e‐source citations shows that there has been a notable increase in the number and proportion of authors who cite e‐sources in their research articles over the 8‐year period, although at the time of this study, e‐sources were still cited much less frequently than print sources. This result provides empirical evidence that e‐sources are increasingly used among scholars. Complementing the citation data, the results from the author survey show that e‐sources are becoming an important component in scholars' research, and are serving a wide range of purposes and functions. The number of access points and self‐perceived overall ability to use the Internet are identified as the two significant variables affecting frequency of e‐source use. The results of this study also suggest that a limited number of criteria can be implemented in practice for scholars to evaluate electronic sources and systems. When citing e‐sources, scholars consider some factors that are unique to e‐sources, in addition to the factors they consider for print sources. Although the advantages of e‐sources promote citing, some drawbacks of e‐sources at this stage serve as a barrier. The survey of editors also reveals a lack of clearly stated editorial policies regarding citing e‐sources. The major problems and concerns reported by scholars regarding using e‐sources are summarized, and both the theoretical implications and practical applications of the findings are discussed.

Scholarly use of Internet-based electronic resources

This research examines the use of Internet-based electronic resources (e-sources) by a group of library and information science (LIS) scholars. It focuses particularly on how scholars use, cite, and evaluate e-sources during the research process. This research also explores the problems scholars encounter and concerns they have when using e-sources for research. The following approaches were used to collect data for the investigation: (a) a longitudinal analysis of e-source citations in eight LIS journals from 1991 through 1998; (b) a survey of editors of the eight journals; and (c) a survey of 201 authors with articles to be published in the eight journals. The longitudinal analysis of e-source citations shows that there has been a notable increase in the number and proportion of authors who cite e-sources in their research articles over the 8-year period, although at the time of this study, e-sources were still cited much less frequently than print sources. This result provides empirical evidence that e-sources are increasingly used among scholars. Complementing the citation data, the results from the author survey show that e-sources are becoming an important component in scholars' research, and are serving a wide range of purposes and functions. The number of access points and self-perceived overall ability to use the Internet are identified as the two significant variables affecting frequency of e-source use. The results of this study also suggest that a limited number of criteria can be implemented in practice for scholars to evaluate electronic sources and systems. When citing e-sources, scholars consider some factors that are unique to e-sources, in addition to the factors they consider for print sources. Although the advantages of e-sources promote citing, some drawbacks of e-sources at this stage serve as a barrier. The survey of editors also reveals a lack of clearly stated editorial policies regarding citing e-sources. The major problems and concerns reported by scholars regarding using e-sources are summarized, and both the theoretical implications and practical applications of the findings are discussed.