Systematic Literature Reviews In Software Engineering Research Articles

Performance Evaluation of the Plagiarism Systems A Systematic Review

Paraphrase plagiarism occurs when text is deliberately obfuscated to evade detection, deliberate alteration increases the complexity of plagiarism and the difficulty in detecting paraphrase plagiarism., Plagiarism systems play an important role in educational and research institutions. In the field of education, the use of these systems has increased in universities to help researchers select new research, which is why it has been important to ensure that universities have effective citation screening systems. Different metrics were used to evaluate the performance of these systems. All of these metrics affect the performance and usability of the impersonation system. This paper present a systematic literature review (SLR) to obtain an overview of the existing studies of distinct criteria for evaluating the performance of textual plagiarism systems and each criterion is briefly explained to give the basic idea behind it. Overall, the aim of this paper is to provide a modern introduction and short review of these criteria. A systematic literature review is performed according to the guidelines for conducting SLR in software engineering. The study relied on a comprehensive set of research papers collected from electronic libraries published during a time period from 2013 to 2023. The results of this SLR include information about metrics for evaluating the performance of plagiarism systems according to previous research. During a specific period it can assist researchers in this field by providing an overview of current researches in this field. Moreover, it may serve as a first step towards a great SLR-assisted explanation of the subject.

A Quality Assessment Instrument for Systematic Literature Reviews in Software Engineering

Context: Systematic literature reviews (SLRs) have become standard practise as part of software engineering research, although their quality varies. To build on the reviews, both for future research and industry practice, they need to be of high quality. Objective: To assess the quality of SLRs in software engineering, we put forward an appraisal instrument for SLRs. The instrument is intended for use by appraisers of reviews, but authors may also use it as a checklist when designing and documenting their reviews. Method: A well-established appraisal instrument from research in healthcare was used as a starting point to develop a quality assessment instrument. It is adapted to software engineering using guidelines, checklists, and experiences from software engineering. As a validation step, the first version was reviewed by four external experts on SLRs in software engineering and updated based on their feedback. Results: The outcome of the research is an appraisal instrument for the quality assessment of SLRs in software engineering. The instrument intends to support the appraiser in assessing the quality of an SLR. The instrument includes 16 items with different options to capture the quality. The item is assessed on a two or three-grade scale, depending on the item. The instrument also supports consolidating the items into groups, which are then used to assess the overall quality of a systematic literature review. Conclusion: It is concluded that the presented instrument may be helpful support for an appraiser in assessing the quality of SLRs in software engineering.

Big Data Maturity Assessment Models: A Systematic Literature Review

Big Data and analytics have become essential factors in managing the COVID-19 pandemic. As no company can escape the effects of the pandemic, mature Big Data and analytics practices are essential for successful decision-making insights and keeping pace with a changing and unpredictable marketplace. The ability to be successful in Big Data projects is related to the organization’s maturity level. The maturity model is a tool that could be applied to assess the maturity level across specific key dimensions, where the maturity levels indicate an organization’s current capabilities and the desirable state. Big Data maturity models (BDMMs) are a new trend with limited publications published as white papers and web materials by practitioners. While most of the related literature might not have covered all of the existing BDMMs, this systematic literature review (SLR) aims to contribute to the body of knowledge and address the limitations in the existing literature about the existing BDMMs, assessment dimensions, and tools. The SLR strategy in this paper was conducted based on guidelines to perform SLR in software engineering by answering three research questions: (1) What are the existing maturity assessment models for Big Data? (2) What are the assessment dimensions for Big Data maturity models? and (3) What are the assessment tools for Big Data maturity models? This SLR covers the available BDMMs written in English and developed by academics and practitioners (2007–2022). By applying a descriptive qualitative content analysis method for the reviewed publications, this SLR identified 15 BDMMs (10 BDMMs by practitioners and 5 BDMMs by academics). Additionally, this paper presents the limitations of existing BDMMs. The findings of this paper could be used as a grounded reference for assessing the maturity of Big Data. Moreover, this paper will provide managers with critical insights to select the BDMM that fits within their organization to support their data-driven decisions. Future work will investigate the Big Data maturity assessment dimensions towards developing a new Big Data maturity model.

Reducing efforts of software engineering systematic literature reviews updates using text classification

Systematic Literature Reviews (SLRs) are frequently used to synthesize evidence in Software Engineering (SE), however replicating and keeping SLRs up-to-date is a major challenge. The activity of studies selection in SLR is labor intensive due to the large number of studies that must be analyzed. Different approaches have been investigated to support SLR processes, such as: Visual Text Mining or Text Classification. But acquiring the initial dataset is time-consuming and labor intensive. In this work, we proposed and evaluated the use of Text Classification to support the studies selection activity of new evidences to update SLRs in SE. We applied Text Classification techniques to investigate how effective and how much effort could be spared during the studies selection phase of an SLR update. Considering the SLRs update scenario, the studies analyzed in the primary SLR could be used as a classified dataset to train Supervised Machine Learning algorithms. We conducted an experiment with 8 Software Engineering SLRs. In the experiments, we investigated the use of multiple preprocessing and feature extraction tasks such as tokenization, stop words removal, word lemmatization, TF-IDF (Term-Frequency/Inverse-Document-Frequency) with Decision Tree and Support Vector Machines as classification algorithms. Furthermore, we configured the classifier activation threshold for maximizing Recall, hence reducing the number of Missed selected studies. The techniques accuracies were measured and the results achieved on average a F-Score of 0.92 and 62% of exclusion rate when varying the activation threshold of the classifiers, with a 4% average number of Missed selected studies. Both the Exclusion rate and number of Missed selected studies were significantly different when compared to classifier which did not use the configuration of the activation threshold. The results showed the potential of the techniques in reducing the effort required of SLRs updates.

When to update systematic literature reviews in software engineering

[Context] Systematic Literature Reviews (SLRs) have been adopted by the Software Engineering (SE) community for approximately 15 years to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially outdated, and there are no systematic proposals on when to update SLRs in SE. [Objective] The goal of this paper is to provide recommendations on when to update SLRs in SE. [Method] We evaluated, using a three-step approach, a third-party decision framework (3PDF) employed in other fields, to decide whether SLRs need updating. First, we conducted a literature review of SLR updates in SE and contacted the authors to obtain their feedback relating to the usefulness of the 3PDF within the context of SLR updates in SE. Second, we used these authors’ feedback to see whether the framework needed any adaptation; none was suggested. Third, we applied the 3PDF to the SLR updates identified in our literature review. [Results] The 3PDF showed that 14 of the 20 SLRs did not need updating. This supports the use of a decision support mechanism (such as the 3PDF) to help the SE community decide when to update SLRs. [Conclusions] We put forward that the 3PDF should be adopted by the SE community to keep relevant evidence up to date and to avoid wasting effort with unnecessary updates.

Scholarly publication venue recommender systems

PurposeThe purpose of this investigation is to identify, evaluate, integrate and summarize relevant and qualified papers through conducting a systematic literature review (SLR) on the application of recommender systems (RSs) to suggest a scholarly publication venue for researcher's paper.Design/methodology/approachTo identify the relevant papers published up to August 11, 2018, an SLR study on four databases (Scopus, Web of Science, IEEE Xplore and ScienceDirect) was conducted. We pursued the guidelines presented by Kitchenham and Charters (2007) for performing SLRs in software engineering. The papers were analyzed based on data sources, RSs classes, techniques/methods/algorithms, datasets, evaluation methodologies and metrics, as well as future directions.FindingsA total of 32 papers were identified. The most data sources exploited in these papers were textual (title/abstract/keywords) and co-authorship data. The RS classes in the selected papers were almost equally used. DBLP was the main dataset utilized. Cosine similarity, social network analysis (SNA) and term frequency–inverse document frequency (TF–IDF) algorithm were frequently used. In terms of evaluation methodologies, 24 papers applied only offline evaluations. Furthermore, precision, accuracy and recall metrics were the popular performance metrics. In the reviewed papers, “use more datasets” and “new algorithms” were frequently mentioned in the future work part as well as conclusions.Originality/valueGiven that a review study has not been conducted in this area, this paper can provide an insight into the current status in this area and may also contribute to future research in this field.

On Using Grey Literature and Google Scholar in Systematic Literature Reviews in Software Engineering

Context: The inclusion of grey literature (GL) is important to remove publication bias while gathering available evidence regarding a certain topic. The number of systematic literature reviews (SLRs) in Software Engineering (SE) is increasing but we do not know about the extent of GL usage in these SLRs. Moreover, Google Scholar is rapidly becoming a search engine of choice for many researchers but the extent to which it can find the primary studies is not known. Objective: This tertiary study is an attempt to i) measure the usage of GL in SLRs in SE. Furthermore this study proposes strategies for categorizing GL and a quality checklist to use for GL in future SLRs; ii) explore if it is feasible to use only Google Scholar for finding scholarly articles for academic research. Method: We have conducted a systematic mapping study to measure the extent of GL usage in SE SLRs as well as to measure the feasibility of finding primary studies using Google Scholar. Results and conclusions: a) Grey Literature: 76.09% SLRs (105 out of 138) in SE have included one or more GL studies as primary studies. Among total primary studies across all SLRs (6307), 582 are classified as GL, making the frequency of GL citing as 9.23%. The intensity of GL use indicate that each SLR contains 5 primary studies on average (total intensity of GL use being 5.54). The ranking of GL tells us that conference papers are the most used form 43.3% followed by technical reports 28.52%. Universities, research institutes, labs and scientific societies together make up 67.7% of GL used, indicating that these are useful sources for searching GL. We additionally propose strategies for categorizing GL and criteria for evaluating GL quality, which can become a basis for more detailed guidelines for including GL in future SLRs. b) Google Scholar Results: The results show that Google Scholar was able to retrieve 96% of primary studies of these SLRs. Most of the primary studies that were not found using Google Scholar were from grey sources.

Scrutinizing Systematic Literature Review Process in Software Engineering

Performing the Systematic Literature Review (SLR) in the turbulent field of Software Engineering (SE) brings different obstacles and uncertainties. The commonly used guidelines for performing the SLR in this field are adapted from health sciences and presented by Kitchenham and Charters in 2007. This paper follows the Kitchenham’s three-phases-review-process and fulfils it with the findings, observations and recommendations from other influential authors in the field. The process of SLR is observed from the perspective of appliance in the field of SE and supplemented by the important precautious measures that should be undertaken by the authors performing it. Thus, this paper aims to present the state-of-the-art in performing the SLR in SE.

Protocol adaptations to conduct Systematic Literature Reviews in Software Engineering: a chronological study

<div class="page" title="Page 1"><div class="section"><div class="layoutArea"><div class="column"><p><span>Systematic literature reviews (SLR) have reached a considerable level of adoption in Software Engineering (SE), how-ever protocol adaptations for its implementation remain tangentially addressed. This work provides a chronological framework for the use and adaptation of the SLR protocol, including its current status. A systematic literature search was performed, reviewing a set of twelve articles being selected in accordance with the inclusion and exclusion criteria between 2004 and 2013, using digital data sources recognized by the SE community. A chronological framework is provided that includes the current state of the protocol adaptations to conduct SLR in SE. The results indicate areas where the quantity and quality of investigations needs to be increased and the identi- fication of the main proposals providing adaptations for the protocol conducting SLR in SE. </span></p></div></div></div></div>

Systematic reviews in software engineering: An empirical investigation

Systematic Literature Reviews (SLRs) have gained significant popularity among Software Engineering (SE) researchers since 2004. Several researchers have also been working on improving the scientific and methodological infrastructure to support SLRs in SE. We argue that there is also an apparent and essential need for evidence-based body of knowledge about different aspects of the adoption of SLRs in SE. The main objective of this research is to empirically investigate the adoption, value, and use of SLRs in SE research from various perspectives. We used mixed-methods approach (systematically integrating tertiary literature review, semi-structured interviews and questionnaire-based survey) as it is based on a combination of complementary research methods which are expected to compensate each others’ limitations. A large majority of the participants are convinced of the value of using a rigourous and systematic methodology for literature reviews in SE research. However, there are concerns about the required time and resources for SLRs. One of the most important motivators for performing SLRs is new findings and inception of innovative ideas for further research. The reported SLRs are more influential compared to the traditional literature reviews in terms of number of citations. One of the main challenges of conducting SLRs is drawing a balance between methodological rigour and required effort. SLR has become a popular research methodology for conducting literature review and evidence aggregation in SE. There is an overall positive perception about this relatively new methodology to SE research. The findings provide interesting insights into different aspects of SLRs. We expect that the findings can provide valuable information to readers about what can be expected from conducting SLRs and the potential impact of such reviews.

A Federated Search Approach to Facilitate Systematic Literature Review in Software Engineering

To impact industry, researchers developing technolo gies in academia need to provide tangible evidence of the advantages of using them. Nowadays, Systematic Literature Review (SLR) has become a prominent methodology in evidence-based researches. Although adopting SLR in software engineering does not go fa r in practice, it has been resulted in valuable resea rches and is going to be more common. However, digi tal libraries and scientific databases as the best rese arch resources do not provide enough mechanism for SLRs especially in software engineering. On the oth er hand, any loss of data may change the SLR result s and leads to research bias. Accordingly, the search process and evidence collection in SLR is a critic al point. This paper provides some tips to enhance the SLR process. The main contribution of this work is presenting a federated search tool which provides a n automatic integrated search mechanism in wellknown Software Engineering databases. Results of ca se study show that this approach not only reduces required time to do SLR and facilitate its search p rocess, but also improves its reliability and resul ts in the increasing trend to use SLRs.

Identifying relevant studies in software engineering

Context Systematic literature review (SLR) has become an important research methodology in software engineering since the introduction of evidence-based software engineering (EBSE) in 2004. One critical step in applying this methodology is to design and execute appropriate and effective search strategy. This is a time-consuming and error-prone step, which needs to be carefully planned and implemented. There is an apparent need for a systematic approach to designing, executing, and evaluating a suitable search strategy for optimally retrieving the target literature from digital libraries. Objective The main objective of the research reported in this paper is to improve the search step of undertaking SLRs in software engineering (SE) by devising and evaluating systematic and practical approaches to identifying relevant studies in SE. Method We have systematically selected and analytically studied a large number of papers (SLRs) to understand the state-of-the-practice of search strategies in EBSE. Having identified the limitations of the current ad-hoc nature of search strategies used by SE researchers for SLRs, we have devised a systematic and evidence-based approach to developing and executing optimal search strategies in SLRs. The proposed approach incorporates the concept of ‘quasi-gold standard’ (QGS), which consists of collection of known studies, and corresponding ‘quasi-sensitivity’ into the search process for evaluating search performance. Results We conducted two participant–observer case studies to demonstrate and evaluate the adoption of the proposed QGS-based systematic search approach in support of SLRs in SE research. Conclusion We report their findings based on the case studies that the approach is able to improve the rigor of search process in an SLR, as well as it can serve as a supplement to the guidelines for SLRs in EBSE. We plan to further evaluate the proposed approach using a series of case studies on varying research topics in SE.