What is a fatal flaw? A guide for authors & reviewers

Abstract

In our first editorial (Richey & Davis-Sramek, 2020), we made two important commitments to the community: a faster review cycle and a larger number of manuscripts published every year. For a higher quality and more expedient review process, we revised the editorial review structure and process to create a more influential Senior Editor (SE) role in addition to a requirement for two reviewers per manuscript. Meeting the second commitment is more challenging because it is not as simple as accepting more papers. It requires increasing the number of quality submissions to maintain a low acceptance rate—an important signal of journal prestige. To accomplish this, we have moved to enhance the reach of the Journal of Business Logistics (JBL) through (1) an enhanced social media presence, (2) an ongoing invitation for Special Topic Forums, (3) a request for manuscripts that are more inclusive of our international colleagues, and (4) a call for research on a broader set of boundary-spanning topics. As JBL editors, we appreciate the role of our reviewers and SEs in meeting these goals. In a nutshell, we cannot accomplish either without their effort and commitment to creating a timely review process. We also depend on their expertise and insights to move papers through the review process to publication. Likewise, we need them to inform us when manuscripts do not meet the conceptual or methodological standards of rigor, as well as when they do not address a relevant issue that impacts practice. Perhaps most importantly, we all depend on one another to support scholars in our community and give them the best opportunity for advancement and promotion. To understand the perceptions of fatal flaws in supply chain management and logistics (SCML) research, we developed a basic survey and sent it to JBL's Senior Editors (SEs) and members of the Editorial Advisory Board (EAB). This sample resulted in forty-one responses and a 54% response rate. In asking about their own submissions, two-thirds of the respondents answered “yes” to having a paper rejected from JBL when it did not have a fatal flaw. Additionally, 90% of them reported “yes” to having a paper rejected from a journal other than JBL without a fatal flaw. This could be interpreted to mean: (1) Review team members are not explicitly pointing out what they assess are fatal flaws, or (2) papers with correctable, albeit perhaps major issues, are being prematurely rejected. If it is the former, we are working to remedy the issue by asking both reviewers and SEs to provide a concrete rationale for rejecting a manuscript. The latter issue, however, requires more reflection, and it concerns us that valuable work could be rejected for reasons other than fatally flawed scholarship. With this in mind, it makes sense to provide a grounded and specific understanding of fatal flaws. This is where question three takes us. Although standard definitions of fatal flaws are readily available, the respondents offered specific and sometimes detailed explanations. The open-ended responses were analyzed and categorized using Word Stat.1 Given the small sample, our intent was general description rather than statistical significance. Figure 1 illustrates the word cloud produced in the analysis that summarizes and highlights the characteristics of a fatal flaw, with larger words corresponding to more frequent use in the responses. With the caveat that our tenure as JBL editors is relatively short, the findings are consistent with our own observations. An academic text contains a fatal flaw if it contains an ethical, paradigmatic, theoretical, or methodological issue that is so substantial that even a major revision could not sufficiently solve this issue. – Tenured Associate Professor and JBL SE Authors and reviewers are reminded to take note of the four areas addressed—ethical, paradigmatic, theoretical, or methodological issues. These areas address the scientific approach taken, the authors' understanding of the literature, the conceptualization (of the model), and the methodological execution of the study. Reviewers are commonly asked to examine the existence of fatal flaws in each of these areas. A significant error in research design, data collection or analysis/findings that cannot be remedied and which invalidates the findings to the point where the research is not publishable. – Chaired Full Professor and JBL SE An aspect of the research design that cannot be corrected now that the study is completed. It cannot be salvaged as is. – Chaired Full Professor and JBL EAB I define fatal flaw as an aspect of the research design that cannot be corrected now that the study is completed. Fatal flaws can occur via several avenues. One source of fatal flaw is a critical mistake in the research design (e.g., there isn't a control group yet a differences-in-differences approach was attempted, using a regression discontinuity design where assignment to treatment and control group isn't random). A second source is poor measurement of the underlying theoretical construct being operationalized. Poor measurement can arise because the observed indicator(s) don't map to the construct's domain (e.g., using R&D intensity to measure asset specificity) or research design (e.g., many single-informant surveys result in measures having unnaturally high correlations). Some things that I would not deem fatal flaws are (i) poor theoretical positioning, (ii) an inadequate literature review, and (iii) choice of statistical estimator. So long as the research design and measures are adequate, theory, literature review, and statistical approaches can be fixed. – Tenured Associate Professor and JBL SE While varied, there is a common theme across these explanations: Fatal flaws reflect issues in the research that cannot be remedied or corrected. It means that something in the conceptualization, design, or execution of the study leads to questions about the credibility of the contribution, or that the contribution does not add new knowledge about the phenomenon of interest. Reviewers are asked to be supportive and to help authors find a path to publication, but many flaws make finding that path impossible and often result in invalid findings. To get a more detailed understanding of the responses, the data were coded to uncover higher order themes. Nine themes emerged, and Table 1 provides these themes along with selected corresponding examples from the data. Limited understanding of the theory and literature Significant mismatch between research question, theory, and design Failure to rule out reasonable competing explanations for observations Naive positivism—every rationally justifiable assertion can be scientifically verified or is capable of logical or mathematical proof Significant amount of the relevant literature excluded Theoretical grounding ignored No research questions/No hypotheses Assumptions not grounded in realistic understanding of the phenomenon Real-world system cannot be operated in the way assumed Data is abstracted in a way that is not representative of reality Research purpose, design choice, or concepts not clear Disorganized flow Poor/convoluted writing Lack of positioning of the study to the journal audience Did not ask the right question(s) Outdated or poorly designed measures Context is overly simplified/too narrow Biasing questions in qualitative interviews Measures/items that lack face validity of concepts/constructs do not actually measure the intended concept/construct Inappropriate sample sizes/populations Bad/incorrect sampling frame Subjects not representative of population to the results are being generalized Strong evidence of common method variance Secondary data that do not really address the question being asked Deviation from standard/best practice and methodologies Deviation from methodological standard/best practices Relevant control variables are missing Presence of a confounding variable Inductive approach but deductive logic Contribution to the field incremental or not clear Who cares? What problem is solved? Lack of fit to the audience of the journal Insufficient motivation of future research Context too narrow Results/conclusions not actionable Focus on the pure mathematical model Operationalization of issues the manager or firm cannot influence Use of outcome variables that extend beyond the managerial context HARKing—hypothesizing after the results are known Plagiarism/Self-plagiarism Artificial data We largely agree with these themes, and we address most fatal flaws via desk rejections. Many manuscript submissions that conclude with desk rejections stem from the fatal style issues listed above. If a paper is disorganized and incoherent so much that its purpose is not clear, we will not waste the valuable time of reviewers (or the authors). It is also important to note that we will not request a review if the manuscript is a poor fit for the journal. We desk reject manuscripts when the references in a manuscript are not at all reflective of the SCML literature, or when the paper is focused solely on the development of a mathematical model or the application of a tool—to the exclusion of making a larger theoretical contribution. Finally, a paper is almost sure to be desk rejected when the research context is too narrow, although we distinguish between when a narrow context reflects a phenomenon versus when the context is the phenomenon. As a hypothetical example, we would desk reject a paper that surveys kiwi farmers in Alabama when its stated purpose is to understand retail kiwi distribution in Alabama.2 Alternately, we welcome manuscripts that use a narrow context to bring nuance and understanding to a SCML phenomenon. For example, Rao et al. (2021) examined two serialized blockchain projects, but they used this narrow context to provide novel insight about supply chain visibility and transparency. Overall, when we send a manuscript out for review, several hurdles have already been cleared. For many of the other fatal flaw issues listed in Table 1, we depend on reviewers to use their judgment and research capabilities to help us reach a decision. While the examples noted in the table are not comprehensive, they do represent thinking from our most senior editorial board members. It is important that researchers proactively take measures to eliminate fatal flaws in their work. We hope that this will help JBL authors in both designing and executing research studies and in writing manuscripts. Let us face the hard truth—it is much easier to unearth flaws in a manuscript and write it off as unsalvageable than to guide authors to a path forward. But given that all research is flawed (McGrath, 1981), the question should be, “Is the manuscript fatally flawed?” Our expectation is that a recommendation for rejecting a manuscript will be based on insurmountable flaws, rather than on a personal disinterest of the phenomenon or dislike of the research method employed. Perhaps even the narrowest path would involve repositioning, a different theoretical lens, better construct alignment, inclusion of other variables, or a different statistical approach. However, when there are indeed cases that flaws cannot be remedied or corrected, we ask that they are prioritized and explained when reviewers and SEs make the recommendation to reject a manuscript. We would like to see the review process put significant focus on what authors can do to move the paper toward publication, even if it does not meet the standards required for publication in JBL. We challenge reviewers to counter every constructive criticism with a constructive suggestion. Likewise, it behooves authors to make the reviewers' jobs easier! Suggestions include presenting the research at conferences, getting “friendly reviews” from colleagues, and hiring a professional copy editor. This will move a manuscript past the development or “stream of consciousness” stage, which can be distracting and frustrating for reviewers and editors alike. Perhaps the worst mistake that authors can make is when obfuscation (intentional or unintentional convoluted writing) leads reviewers to render the manuscript as fatally flawed. In sum, it is hardly new or novel to suggest, “Reviewers should treat manuscripts the way they would like their own submitted manuscripts to be treated.” To advance JBL and to meet the commitments we make to the community, we need reviewers and SEs to help us build a system of supportive reviewing. Diamonds in the rough often shine the brightest, but only after an abundant polishing. This requires that reviewers and SEs not only serve as trusted gatekeepers, but they also must be “diamond cutters” (Saunders, 2005) who will work with authors to transform rough manuscripts into impactful gems of scholarship. This issue includes four peer-reviewed articles, one original manuscript by Wallenburg et al. (2021) and three that come as a result the Special Topic Forum (STF) on Managing Extended Supply Chains, co-edited by Thomas Choi, Sriram Narayanan, David Novak, Jan Olhager, Jiuh-Biing Sheu, and Frank Wiengarten. This issue also includes an invited editorial by the STF editorial team (Choi et al., 2021). Early conceptualizations of supply chain management emphasized creating value for the customer (Mentzer et al., 2001), and this emphasis is no less, and perhaps significantly more important for understanding SCM today (Min et al., 2019). In a recent retrospective article written as reflections by previous JBL editors (Goldsby et al., 2019), the evolution of the Journal reflects the shift from creating customer value by managing the firm's transportation and logistics functions to managing a complex network of firms that often span the globe. To meet the increasing demands of customers for service, convenience, and speed, companies are challenged to improve both capabilities in their downstream order fulfillment processes and visibility in their upstream network. Several recent JBL articles have examined order fulfillment processes involving digital capabilities (Ambra et al. 2020; Ballot et al. 2021; Ishfaq et al. 2021), inventory management approaches (Muir et al. 2019; Singh & Ardjmand, 2020), and consumer responses to order fulfillment options (Nguyen et al., 2019; Tokar et al. 2020). In this issue, Wallenburg et al. (2021) contribute to this growing stream of research by investigating a challenging issue for firms—product returns. They take a novel perspective by using signaling theory and utilizing a natural experiment to examine how product packaging can minimize purchasing returns. They find that returns are reduced with premium packaging. Importantly, this suggests that firms should shift their functional view of packaging and think about how to use packaging for a more esthetic appeal. The rest of this issue focuses on the management of extended supply chains. As the global landscape drives companies to be part of complex global supply chains, the need for theoretical calibration (Craighead et al., 2019) suggests theory development and data collection should reflect a multi-tiered research approach. This is easier said than done, of course, with a small number of studies capturing even upstream dyadic data (e.g., Schwieterman et al., 2020). This STF pushes the field to examine and analyze extended upstream supply chains. In introducing the STF, Choi et al. (2021) elaborate on the issues and challenges that firms should consider. Li et al. (2021) investigate a phenomenon that happens often in automotive supply chains—financial squeezing. When companies force suppliers to reduce their prices, what is the ripple effect across the extended supply chain? This study finds that this common practice can lead to instability across the network, with suppliers at the bottom suffering the most. Nath et al. (2021) take an interesting perspective by exploring institutional pressures that call for companies throughout the apparel supply chain to implement more sustainable SCM practices. They find that both first- and second-tier suppliers are subject to more coercive pressure from buyers, third-party auditors, and governments. Finally, Wiedmer and Griffis (2021) offer a rare and comprehensive empirical analysis of “real-world” supply chain networks. Analyzing 21 five-tier extended supply chains, they examine whether these real-world supply chains align with previously developed network typologies. Interestingly, their analysis did not show evidence of widely touted “coopetition” characteristics. In sum, the articles in this issue demonstrate cutting-edge research that promotes the importance of having a holistic approach to logistics and supply chain phenomena. Collectively, these articles offer broad implications for the discipline, and we hope that the JBL community will continue to push the boundaries of knowledge into these kinds of important and impactful areas. Happy reading! Join is on LinkedIn for implications and updates!