Quality Of Wikipedia Articles Research Articles

How article category in Wikipedia determines the heterogeneity of its editors

Collaboration is essential to advancing knowledge and, ultimately, entire societies. With the development of Web 2.0, the possibilities have risen to unprecedented levels and allowed for the collaborative creation of the world’s largest compendium of knowledge that ever existed – Wikipedia. Collaboration is not a safeguard of quality per se, however. Rather, the quality of Wikipedia articles rises with the number of editors per article as well as a greater diversity among them. Here, we address a not yet documented potential threat to those preconditions: self-selection of Wikipedia editors to articles. Specifically, we expected articles with a clear-cut link to a specific country (e.g., about its highest mountain, “national” article category) to attract a larger proportion of editors of that nationality when compared to articles without any specific link to that country (e.g., “gravity”, “universal” article category), whereas articles with a link to several countries (e.g., “United Nations”, “international” article category) should fall in between. Across several language versions, hundreds of different articles, and hundreds of thousands of editors, we find the expected effect within Wikipedia: The more exclusively an article topic is linked to a particular nation, the higher the proportion of editors from that country is among the contributors.

Automatic Quality Assessment of Wikipedia Articles—A Systematic Literature Review

Wikipedia is the world’s largest online encyclopedia, but maintaining article quality through collaboration is challenging. Wikipedia designed a quality scale, but with such a manual assessment process, many articles remain unassessed. We review existing methods for automatically measuring the quality of Wikipedia articles, identifying and comparing machine learning algorithms, article features, quality metrics, and used datasets, examining 149distinct studies, and exploring commonalities and gaps in them. The literature is extensive, and the approaches follow past technological trends. However, machine learning is still not widely used by Wikipedia, and we hope that our analysis helps future researchers change that reality.

Asian American Female Composers and Digital Memory

Asian American Female Composers and Digital Memory

Verhütung in der deutschsprachigen Wikipedia: Eine Inhalts- und Qualitätsanalyse

ZusammenfassungHintergrundJugendliche und Erwachsene beziehen Informationen über Verhütung heute oft per Internet, vor allem über die Online-Enzyklopädie Wikipedia, da Google-Suchen meist Wikipedia-Einträge als Toptreffer liefern.Ziel der ArbeitVor diesem Hintergrund ist es Ziel des vorliegenden Beitrags, erstmals Inhalte und Qualität von Wikipedia-Artikeln über Verhütungsmethoden systematisch zu analysieren. Geprüft werden dabei 5 zentrale Qualitätsdimensionen: die Ausprägung der Korrektheit (Forschungsfrage F1), der Vollständigkeit (F2), der Neutralität (F3), der Verständlichkeit (F4) und der Aktualität (F5) der Verhütungsinformationen sowie auf dieser Basis auch ihre Gesamtqualität (F6).Material und MethodenEs wurde eine Stichprobe aller deutschsprachigen Wikipedia-Artikel zu allen Verhütungsmethoden gebildet (N = 25). Diese Artikel wurden mittels eines auf der Basis des Forschungsstandes entwickelten und reliabilitätsgeprüften Codebuchs von 3 unabhängigen, geschulten Codierenden analysiert. Die Datenanalyse erfolgte mit SPSS. Die Studie ist präregistriert und alle Daten, Materialien und Analyseskripte sind öffentlich verfügbar.ErgebnisseEs zeigte sich, dass die 25 Wikipedia-Artikel zu Verhütungsmethoden in ihrer inhaltlichen Qualität stark variierten. Während sie hinsichtlich Korrektheit (F1) und Neutralität (F3) im Mittel gute Qualität aufwiesen, erreichten sie hinsichtlich Vollständigkeit (F2), Verständlichkeit (F4) und Aktualität (F5) nur mittelmäßige Werte, woraus sich dann auch eine moderate Gesamtqualität ergab (F6).DiskussionWeitere Forschung sowie Praxismaßnahmen sind notwendig, um die Qualität von Verhütungsinformationen in der Wikipedia und in anderen sozialen Medien noch besser einschätzen und zielgerichteter verbessern zu können.

Quality Change: Norm or Exception? Measurement, Analysis and Detection of Quality Change in Wikipedia

Wikipedia has been turned into an immensely popular crowd-sourced encyclopedia for information dissemination on numerous versatile topics in the form of subscription free content. It allows anyone to contribute so that the articles remain comprehensive and updated. For enrichment of content without compromising standards, the Wikipedia community enumerates a detailed set of guidelines, which should be followed. Based on these, articles are categorized into several quality classes by the Wikipedia editors with increasing adherence to guidelines. This quality assessment task by editors is laborious as well as demands platform expertise. As a first objective, in this paper, we study evolution of a Wikipedia article with respect to such quality scales. Our results show novel non-intuitive patterns emerging from this exploration. As a second objective we attempt to develop an automated data driven approach for the detection of the early signals influencing the quality change of articles. We posit this as a change point detection problem whereby we represent an article as a time series of consecutive revisions and encode every revision by a set of intuitive features. Finally, various change point detection algorithms are used to efficiently and accurately detect the future change points. We also perform various ablation studies to understand which group of features are most effective in identifying the change points. To the best of our knowledge, this is the first work that rigorously explores English Wikipedia article quality life cycle from the perspective of quality indicators and provides a novel unsupervised page level approach to detect quality switch, which can help in automatic content monitoring in Wikipedia thus contributing significantly to the CSCW community.

Measuring Quality of Wikipedia Articles by Feature Fusion‐based Stack Learning

AbstractOnline open‐source knowledge repository such as Wikipedia has become an increasingly important source for users to access knowledge. However, due to its large volume, it is challenging to evaluate Wikipedia article quality manually. To fill this gap, we propose a novel approach named “feature fusion‐based stack learning” to assess the quality of Wikipedia articles. Pre‐trained language models including BERT (Bidirectional Encoder Representations from Transformers) and ELMo (Embeddings from Language Models) are applied to extract semantic information in Wikipedia content. The feature fusion framework consisting of semantic and statistical features is built and fed into an out‐of‐sample (OOS) stacking model, which includes both machine learning and deep learning models. We compare the performance of proposed model with some existing models with different metrics extensively, and conduct ablation studies to prove the effectiveness of our framework and OOS stacking. Generally, the experiment shows that our method is much better than state‐of‐the‐art models.

A hybrid approach to classifying Wikipedia article quality flaws with feature fusion framework

Article quality has always been a major concern for Wikipedia. To improve article quality, it is critical to first identify defects. Thus, flaw classification has attracted considerable attention. To achieve this, several machine-learning-based approaches are available, including deep learning models based on either manually constructed or autoextracted features. However, adopting only features of either single type may not ensure a comprehensive description of articles. To improve flaw classification, we propose a feature fusion framework combining both handcrafted and autoextracted features. In this research, we first use a rule-based method from a previously proposed framework to extract handcrafted features. Additionally, we obtain autoextracted features using Bidirectional Encoder Representations from Transformers (BERT) and various deep learning models, including bidirectional long short-term memory (Bi LSTM), bidirectional gated recurrent unit (Bi GRU), bidirectional recurrent neural network (Bi RNN), and multihead self-attention models. Finally, the handcrafted features are standardized and concatenated with the autoextracted features. Then, the concatenated features are fed into a feedforward neural network for classification. A detailed comparison of different classifiers is conducted. We compare 12 different classifiers in terms of training performance, classification performance, and model training time. The experiments show that the proposed feature fusion framework can notably improve the effectiveness of quality flaw classification for Wikipedia articles. In particular, a Bi GRU model based on the proposed framework achieves excellent classification accuracy.

Readability and quality of Wikipedia articles on pelvic floor disorders.

This study is aimed at evaluating the readability and quality of Wikipedia articles on pelvic floor disorders (PFD) and comparing their content with International Urogynecological Association patient education leaflets. Readability was assessed using six different readability scales, including the Simple Measure of Gobbledygook (SMOG) Index, which is considered superior for scoring healthcare information. Quality was assessed by three female pelvic medicine and reconstructive surgery fellows using the modified DISCERN instrument. DISCERN is validated to evaluate the quality of written consumer health information; it was subsequently modified by health education researchers to enable the evaluation of Wikipedia articles. We evaluated 30 Wikipedia articles that correlated with 29 International Urogynecological Association leaflets. The mean SMOG score of the Wikipedia articles was 12.0 ± 2.1 (12th-grade reading level) whereas the mean SMOG score of the International Urological Association (IUGA) leaflets was 3.4 ± 0.3 (third-grade reading level, p< 0.001). The mean modified DISCERN score of the Wikipedia articles was 34.43 ± 5.90 (moderate quality); however, the mean modified DISCERN score of the IUGA literature was 45.02 ± 1.36 (good quality, p< 0.001). Wikipedia articles on PFD are neither readable nor reliable: they require a 12th-grade-level education for comprehension and are merely rated moderate in quality. In comparison, IUGA leaflets require a third-grade education for comprehension and are rated good in quality. Urogynecological providers should provide appropriate health education materials to patients, as Wikipedia is both a popular and sometimes inaccurate resource for patients.

Relating Wikipedia article quality to edit behavior and link structure

Currently, the relation between edit behavior, link structure, and article quality is not well-understood in our community, notwithstanding that this relationship may facilitate editing processes and content quality on Wikipedia. To shed light on this complex relation, we classify article edits and perform an in-depth analysis of editing sequences for 4941 articles. Additionally, we build a network of internal Wikipedia hyperlinks between articles. Using this data, we compute parsimonious metrics to quantify editing and linking behavior. Our analysis unveils that conflicted articles differ substantially from others in almost all metrics, while we also detect slight trends for high-quality articles. With our network analysis we find evidence indicating that controversial and edit war articles frequently span structural holes in the Wikipedia network. Finally, in a prediction experiment we demonstrate the usefulness of edit behavior patterns and network properties in predicting conflict and article quality. With our work, we assist online collaboration communities, especially Wikipedia, in long-term improvement of content quality by offering valuable insights about the interplay of article quality, controversies and edit wars, editing behavior, and network properties via sequence-based edit and network-based article metrics.

A General Approach to Multimodal Document Quality Assessment

&#x0D; &#x0D; &#x0D; The perceived quality of a document is affected by various factors, including grammat- icality, readability, stylistics, and expertise depth, making the task of document quality assessment a complex one. In this paper, we explore this task in the context of assessing the quality of Wikipedia articles and academic papers. Observing that the visual rendering of a document can capture implicit quality indicators that are not present in the document text — such as images, font choices, and visual layout — we propose a joint model that combines the text content with a visual rendering of the document for document qual- ity assessment. Our joint model achieves state-of-the-art results over five datasets in two domains (Wikipedia and academic papers), which demonstrates the complementarity of textual and visual features, and the general applicability of our model. To examine what kinds of features our model has learned, we further train our model in a multi-task learning setting, where document quality assessment is the primary task and feature learning is an auxiliary task. Experimental results show that visual embeddings are better at learning structural features while textual embeddings are better at learning readability scores, which further verifies the complementarity of visual and textual features.&#x0D; &#x0D; &#x0D;

Wikipedia in Vascular Surgery Medical Education: Comparative Study.

BackgroundMedical students commonly refer to Wikipedia as their preferred online resource for medical information. The quality and readability of articles about common vascular disorders on Wikipedia has not been evaluated or compared against a standard textbook of surgery.ObjectiveThe aims of this study were to (1) compare the quality of Wikipedia articles to that of equivalent chapters in a standard undergraduate medical textbook of surgery, (2) identify any errors of omission in either resource, and (3) compare the readability of both resources using validated ease-of-reading and grade-level tools.MethodsUsing the Medical Council of Canada Objectives for the Qualifying Examination, 8 fundamental topics of vascular surgery were chosen. The articles were found on Wikipedia using Wikipedia’s native search engine. The equivalent chapters were identified in Schwartz Principles of Surgery (ninth edition). Medical learners (n=2) assessed each of the texts on their original platforms to independently evaluate readability, quality, and errors of omission. Readability was evaluated with Flesch Reading Ease scores and 5 grade-level scores (Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, Simple Measure of Gobbledygook Index, and Automated Readability Index), quality was evaluated using the DISCERN instrument, and errors of omission were evaluated using a standardized scoring system that was designed by the authors.ResultsFlesch Reading Ease scores suggested that Wikipedia (mean 30.5; SD 8.4) was significantly easier to read (P=.03) than Schwartz (mean 20.2; SD 9.0). The mean grade level (calculated using all grade-level indices) of the Wikipedia articles (mean 14.2; SD 1.3) was significantly different (P=.02) than the mean grade level of Schwartz (mean 15.9; SD 1.4). The quality of the text was also assessed using the DISCERN instrument and suggested that Schwartz (mean 71.4; SD 3.1) had a significantly higher quality (P=.002) compared to that of Wikipedia (mean 52.9; SD 11.4). Finally, the Wikipedia error of omission rate (mean 12.5; SD 6.8) was higher than that of Schwartz (mean 21.3; SD 1.9) indicating that there were significantly fewer errors of omission in the surgical textbook (P=.008).ConclusionsOnline resources are increasingly easier to access but can vary in quality. Based on this comparison, the authors of this study recommend the use of vascular surgery textbooks as a primary source of learning material because the information within is more consistent in quality and has fewer errors of omission. Wikipedia can be a useful resource for quick reference, particularly because of its ease of reading, but its vascular surgery articles require further development.

The quality of Wikipedia articles regarding concepts in the field of microbiology and parasitology

The quality of Wikipedia articles regarding concepts in the field of microbiology and parasitology

Wisdom of crowds: the effect of participant composition and contribution behavior on Wikipedia article quality

PurposeThis paper aims to explore the effect of participant composition and contribution behavior of the different types of participants on the quality of knowledge generation in online communities.Design/methodology/approachThis study samples all the featured articles in Chinese Wikipedia and performs a Cox regression to reveal how participant composition and contribution behavior affect the quality of articles in different contexts.FindingsThe results show that an increase in the number of participants increases the possibility of either enhancing or reducing the article quality. In most cases, the greater the proportion of core members (people who frequently participate in editing), the higher the possibility of enhancing the article quality. Occasional participants’ editorial behavior hinders quality promotion, this negative effect weakens when such editorial behavior becomes more frequent.Practical implicationsThe findings help to better leverage the role of online communities in practice and to achieve knowledge collaboration in a more efficient manner. For example, an appropriate centralized organizational form should be established in online communities to improve the efficiency of crowd contributions. And it is worth developing mechanism to encourage participants to frequently participate in editing the article.Originality/valueThis study contributes to the research on the organizational forms of online communities by showing the effect of participant composition and behavior in the new form of organizing on knowledge generation. This study also contributes to the research on wisdom of crowds by revealing who in a group of participants, in what context, and by what means influence knowledge generation.

Indicator of quality for environmental articles on Wikipedia at the higher education level

Wikipedia is important in higher education because students and scholars often use it. Nevertheless, the issue of Wikipedia’s quality is an obstacle for its use at the higher education level. In order to contribute to this discussion, we have proposed ‘Verifiability by respected sources’ as an indicator for assessing the quality of Wikipedia articles at the higher education level and conducted an analysis of the most frequently visited articles in the category of Environment on Wikipedia. Results show that these articles contain many unreferenced statements, so their usage at the higher education level is problematic. Therefore, we also propose specific steps for relevant actors that could help to improve the quality of Wikipedia.

Assessing the quality of information on wikipedia: A deep‐learning approach

Currently, web document repositories have been collaboratively created and edited. One of these repositories, Wikipedia, is facing an important problem: assessing the quality of Wikipedia. Existing approaches exploit techniques such as statistical models or machine leaning algorithms to assess Wikipedia article quality. However, existing models do not provide satisfactory results. Furthermore, these models fail to adopt a comprehensive feature framework. In this article, we conduct an extensive survey of previous studies and summarize a comprehensive feature framework, including text statistics, writing style, readability, article structure, network, and editing history. Selected state‐of‐the‐art deep‐learning models, including the convolutional neural network (CNN), deep neural network (DNN), long short‐term memory (LSTMs) network, CNN‐LSTMs, bidirectional LSTMs, and stacked LSTMs, are applied to assess the quality of Wikipedia. A detailed comparison of deep‐learning models is conducted with regard to different aspects: classification performance and training performance. We include an importance analysis of different features and feature sets to determine which features or feature sets are most effective in distinguishing Wikipedia article quality. This extensive experiment validates the effectiveness of the proposed model.

Perceived Quality of Online Encyclopedias

This study illustrates how different user groups perceive and evaluate the content quality of Wikipedia articles as compared to entries of a traditional encyclopedia. Therefore, an experimental set-up was used with blinded articles of different topic fields from the German Wikipedia and Brockhaus online, evaluated by experts with different backgrounds (university vs. practice) and by students of the field. The findings showed that the quality of both encyclopedias was assessed similarly (intra-group evaluations), although more faults and mistakes were criticized in the Wikipedia sample. However, the inter-group comparison revealed differences in the groups' quality perceptions. This partly applied to the comparison of the expert groups, and especially to the comparison of expert and (non-expert) student evaluations. Students tended to give better ratings, especially within the Wikipedia sample. Most noticeable, they did not detect any content-related faults in both sets, highlighting that further training is needed to improve their information literacy.

Using big data and network analysis to understand Wikipedia article quality

The research reported in this paper focuses on the question of why Wikipedia articles are different in quality. Since these articles are developed in an open and social environment, our work investigates if the social capital of contributors plays a role in determining the quality of the articles. We focus on three major types of social capital with respect to teams of contributors working on Wikipedia articles: internal bonding, external bridging and functional diversity. Through a social network analysis of these articles based on a dataset extracted from its edit history, our research finds that all three types of social capital have a significant impact on their quality. In addition, we found that internal bonding interacts positively with external bridging resulting in a multiplier effect on article quality. The findings of our research have implications for developing automated techniques for quality assessment of Wikipedia and also provide insights into improving quality of these articles.

Knowledge categorization affects popularity and quality of Wikipedia articles.

The existence of a shared classification system is essential to knowledge production, transfer, and sharing. Studies of knowledge classification, however, rarely consider the fact that knowledge categories exist within hierarchical information systems designed to facilitate knowledge search and discovery. This neglect is problematic whenever information about categorical membership is itself used to evaluate the quality of the items that the category contains. The main objective of this paper is to show that the effects of category membership depend on the position that a category occupies in the hierarchical knowledge classification system of Wikipedia—an open knowledge production and sharing platform taking the form of a freely accessible on-line encyclopedia. Using data on all English-language Wikipedia articles, we examine how the position that a category occupies in the classification hierarchy affects the attention that articles in that category attract from Wikipedia editors, and their evaluation of quality of the Wikipedia articles. Specifically, we show that Wikipedia articles assigned to coarse-grained categories (i. e., categories that occupy higher positions in the hierarchical knowledge classification system) garner more attention from Wikipedia editors (i. e., attract a higher volume of text editing activity), but receive lower evaluations (i. e., they are considered to be of lower quality). The negative relation between attention and quality implied by this result is consistent with current theories of social categorization, but it also goes beyond available results by showing that the effects of categorization on evaluation depend on the position that a category occupies in a hierarchical knowledge classification system.

Improving the visibility of library resources via mapping library subject headings to Wikipedia articles

PurposeLinking libraries and Wikipedia can significantly improve the quality of services provided by these two major silos of knowledge. Such linkage would enrich the quality of Wikipedia articles and at the same time increase the visibility of library resources. To this end, the purpose of this paper is to describe the design and development of a software system for automatic mapping of FAST subject headings, used to index library materials, to their corresponding articles in Wikipedia.Design/methodology/approachThe proposed system works by first detecting all the candidate Wikipedia concepts (articles) occurring in the titles of the books and other library materials which are indexed with a given FAST subject heading. This is then followed by training and deploying a machine learning (ML) algorithm designed to automatically identify those concepts that correspond to the FAST heading. In specific, the ML algorithm used is a binary classifier which classifies the candidate concepts into either “corresponding” or “non-corresponding” categories. The classifier is trained to learn the characteristics of those candidates which have the highest probability of belonging to the “corresponding” category based on a set of 14 positional, statistical, and semantic features.FindingsThe authors have assessed the performance of the developed system using standard information retrieval measures of precision, recall, and F-score on a data set containing 170 FAST subject headings manually mapped to their corresponding Wikipedia articles. The evaluation results show that the developed system is capable of achieving F-scores as high as 0.65 and 0.99 in the corresponding and non-corresponding categories, respectively.Research limitations/implicationsThe size of the data set used to evaluate the performance of the system is rather small. However, the authors believe that the developed data set is large enough to demonstrate the feasibility and scalability of the proposed approach.Practical implicationsThe sheer size of English Wikipedia makes the manual mapping of Wikipedia articles to library subject headings a very labor-intensive and time-consuming task. Therefore, the aim is to reduce the cost of such mapping and integration.Social implicationsThe proposed mapping paves the way for connecting libraries and Wikipedia as two major silos of knowledge, and enables the bi-directional movement of users between the two.Originality/valueTo the best of the authors’ knowledge, the current work is the first attempt at automatic mapping of Wikipedia to a library-controlled vocabulary.

Improving information quality of Wikipedia articles with cooperative principle

PurposeThe purpose of this paper is to investigate the impact of cooperative principle on the information quality (IQ) by making objects more relevant for consumer needs, in particular case Wikipedia articles for students.Design/methodology/approachThe authors performed a quantitative study with participants being invited to complete an online survey. Each rater evaluated three selected and re-written articles from Wikipedia by four IQ dimensions (accuracy, completeness, objectivity, and representation). Grice’s maxims and submaxims were used to re-write articles and make them more relevant for student cognitive needs. The results were analyzed with statistical methods of mean, standard deviation, Cronbach’s α, and ICC (two-way random model of single measure).FindingsThe study demonstrates that Wikipedia articles can be made more relevant for student needs by using cooperative principle with increase in IQ and also achieving higher consistency of students’ scores as recent research. In particular, students in the research perceived the abstract, constructed with cooperative principle, more objective and complete as reported in recent research.Practical implicationsThe work can benefit encyclopedia editors to improve IQ of existing articles as well as consumers that would obtain more relevant information in less reading time.Originality/valueThis is one of the first attempts to empirically investigate the application of cooperate principle to make objects more relevant for consumer needs and impact of this on IQ. IQ improvement evidence is provided and impacts on IQ dimensions such as objectivity, completeness, accuracy, and representation for research community to validate and compare results.