Real-world Decision Support Research Articles

Evaluating the Accuracy of Artificial Intelligence(AI)-Generated Synopses for Plasma Cell Disorder Treatment Regimens

BACKGROUND Concise, accurate, and real time synopses of the evidence base are critical to support treatment decision-making in hematology, especially in a very rapidly evolving space such as the treatment of plasma cell disorders (PCDs). Synopses are used in clinical practice guidelines, educational settings, and general clinical practice. This process of knowledge curation is time consuming and cumbersome, even when performed by a clinical expert. Artificial intelligence (AI) - specifically, Large Language Models (LLMs) have promise in this context; however, they are prone to hallucination and may provide inaccurate and out of date information. Morever, the degree to which individual LLMs perform in summarizing the clinical evidence base may vary widely. We objectively assessed the abilities of four LLMs to generate accurate, coherent, and relevant synopses for six widely used PCD regimens. METHODS We compared the performance of four popular LLMs: 1) Claude 3.5 (“Claude”); 2) ChatGPT 4.0 (“ChatGPT”); 3) Gemini; and 4) Llama-3.1 (“Llama”). Each LLM was prompted exactly as follows: “write a synopsis for the development and evolution of therapy with [regimen name] for PCD, using citations from the literature”, where [regimen name] was replaced with: 1) “Dara-RVd”; 2) “KRd”; 3) “VDT-PACE”; 4) “Dara-CyBorD”; 5) “Elranatamab”; and 6) “Talquetamab.” The generated synopses were assessed by two PCD physician specialists using Likert scales on six criteria: accuracy, completeness, relevance, clarity, hallucinations, and coherence; lower scores correspond to poor performance and higher scores to excellent performance. The reviewers were blinded to LLMs to minimize bias and conducted their evaluations independently. The evaluation process was recorded using REDCap. Mean scores with 95% confidence intervals (CI) were calculated across all domains. RESULTS There were marked differences in LLM performance across the six criteria. Claude demonstrated the highest performance in all domains, notably outperforming the other LLMs in accuracy: mean Likert score 3.92 (95% CI 3.54-4.29); ChatGPT 3.25 (2.76-3.74); Gemini 3.17 (2.54-3.80); Llama 1.92 (1.41-2.43); completeness: mean Likert score 4.00 (3.66-4.34); GPT 2.58 (2.02-3.15); Gemini 2.58 (2.02-3.15); Llama 1.67 (1.39-1.95); and lack of hallucinations: mean Likert score 4.00 (4.00-4.00); ChatGPT 2.75 (2.06-3.44); Gemini 3.25 (2.65-3.85); Llama 1.92 (1.26-2.57). Llama performed considerably poorer across all the studied domains, frequently providing inaccurate information and misinterpreted results. ChatGPT and Gemini had intermediate performance. Notably, none of the LLMs including Claude registered perfect accuracy, completeness, or relevance. CONCLUSION To our knowledge, this is the first rigorous evaluation of widely available LLMs for the task of evidence summarization in the PCD domain. In our evaluation of four LLMs for six PCD regimens, meaningful differences were evident across the LLMs. While Claude performed at a notably higher level, evidence synopses need to be as close to perfectly accurate and comprehensive to be usable for real-world clinical decision support. By this standard, even the best performing LLMs continue to require careful editing from a domain expert to become usable. Inaccurate and incoherent synopses could lead to suboptimal patient care if taken literally. We plan to repeat this experiment with newer generations of LLMs to evaluate their potential improvement over time.

A flexible deep learning framework for liver tumor diagnosis using variable multi-phase contrast-enhanced CT scans

BackgroundLiver cancer is a significant cause of cancer-related mortality worldwide and requires tailored treatment strategies for different types. However, preoperative accurate diagnosis of the type presents a challenge. This study aims to develop an automatic diagnostic model based on multi-phase contrast-enhanced CT (CECT) images to distinguish between hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and normal individuals.MethodsWe designed a Hierarchical Long Short-Term Memory (H-LSTM) model, whose core components consist of a shared image feature extractor across phases, an internal LSTM for each phase, and an external LSTM across phases. The internal LSTM aggregates features from different layers of 2D CECT images, while the external LSTM aggregates features across different phases. H-LSTM can handle incomplete phases and varying numbers of CECT image layers, making it suitable for real-world decision support scenarios. Additionally, we applied phase augmentation techniques to process multi-phase CECT images, improving the model’s robustness.ResultsThe H-LSTM model achieved an overall average AUROC of 0.93 (0.90, 1.00) on the test dataset, with AUROC for HCC classification reaching 0.97 (0.93, 1.00) and for ICC classification reaching 0.90 (0.78, 1.00). Comprehensive validation in scenarios with incomplete phases was performed, with the H-LSTM model consistently achieving AUROC values over 0.9.ConclusionThe proposed H-LSTM model can be employed for classification tasks involving incomplete phases of CECT images in real-world scenarios, demonstrating high performance. This highlights the potential of AI-assisted systems in achieving accurate diagnosis and treatment of liver cancer. H-LSTM offers an effective solution for processing multi-phase data and provides practical value for clinical diagnostics.

Fake it till you make it: Synthetic data for emerging carsharing programs

Carsharing is an integral part of the transformation toward flexible and sustainable mobility. New carsharing programs are entering the market to challenge large operators by offering innovative services. This study investigates the use of generative machine learning models for creating synthetic data to support carsharing decision–making when data access is limited. To this end, it explores the evaluation, selection, and implementation of leading-edge methods, such as generative adversarial networks (GANs) and variational autoencoders (VAEs), to generate synthetic tabular transaction data of carsharing trips. The study analyzes usage data of an emerging carsharing program that is expanding its services to include free-floating electric vehicles (EVs). The results show that augmenting real training data with synthetic samples improves predictive modeling of upcoming trips by up to 4.63%. These results support carsharing researchers and practitioners in generating and leveraging synthetic mobility data to develop solutions to real-world decision support problems in carsharing.

Polysocial Risk Scores: Implications for Cardiovascular Disease Risk Assessment and Management.

To review current evidence, discuss key knowledge gaps and identify opportunities for development, validation and application of polysocial risk scores (pSRS) for cardiovascular disease (CVD) risk prediction and population cardiovascular health management. Limited existing evidence suggests that pSRS are promising tools to capture cumulative social determinants of health (SDOH) burden and improve CVD risk prediction beyond traditional risk factors. However, available tools lack generalizability, are cross-sectional in nature or do not assess social risk holistically across SDOH domains. Available SDOH and clinical risk factor data in large population-based databases are under-utilized for pSRS development. Recent advances in machine learning and artificial intelligence present unprecedented opportunities for SDOH integration and assessment in real-world data, with implications for pSRS development and validation for both clinical and healthcare utilization outcomes. pSRS presents unique opportunities to potentially improve traditional "clinical" models of CVD risk prediction. Future efforts should focus on fully utilizing available SDOH data in large epidemiological databases, testing pSRS efficacy in diverse population subgroups, and integrating pSRS into real-world clinical decision support systems to inform clinical care and advance cardiovascular health equity.

Overview of the Problem List Summarization (ProbSum) 2023 Shared Task on Summarizing Patients' Active Diagnoses and Problems from Electronic Health Record Progress Notes.

The BioNLP Workshop 2023 initiated the launch of a shared task on Problem List Summarization (ProbSum) in January 2023. The aim of this shared task is to attract future research efforts in building NLP models for real-world diagnostic decision support applications, where a system generating relevant and accurate diagnoses will augment the healthcare providers' decision-making process and improve the quality of care for patients. The goal for participants is to develop models that generated a list of diagnoses and problems using input from the daily care notes collected from the hospitalization of critically ill patients. Eight teams submitted their final systems to the shared task leaderboard. In this paper, we describe the tasks, datasets, evaluation metrics, and baseline systems. Additionally, the techniques and results of the evaluation of the different approaches tried by the participating teams are summarized.

Novel Distance-Measures-Based Extended TOPSIS Method under Linguistic Linear Diophantine Fuzzy Information

The advantages of the intuitionistic fuzzy set, Pythagorean fuzzy set, and q-rung orthopair fuzzy set are all carried over into the linear Diophantine fuzzy set by extending the restrictions on the grades. Linear Diophantine fuzzy sets offer a wide range of practical applications because the reference parameters allow evaluation andto express their judgments about membership and nonmembership degrees in a variety of ways. Linguistic-valued information cannot be described by linear Diophantine fuzzy numbers since precise numbers are used in linear Diophantine fuzzy systems. In this paper, we first present the novel idea of a linguistic linear Diophantine fuzzy set, which is the hybrid structure of the linear Diophantine fuzzy set and the linguistic term set. Furthermore, some basic operational rules with novel distance measures, namely, Hamming, Euclidean, and Chebyshev distance measures, are established. Based on the newly defined concept of distance measure, an extended TOPSIS technique is presented to tackle the linguistic uncertainty in real-world decision support problems. A numerical example is illustrated to support the applicability of the proposed methodology and to analyze symmetry of the optimal decision. A comparison analysis is constructed to show the symmetry, validity, and effectiveness of the proposed method over the existing decision support techniques.

Tasks as needs: reframing the paradigm of clinical natural language processing research for real-world decision support.

Electronic medical records are increasingly used to store patient information in hospitals and other clinical settings. There has been a corresponding proliferation of clinical natural language processing (cNLP) systems aimed at using text data in these records to improve clinical decision-making, in comparison to manual clinician search and clinical judgment alone. However, these systems have delivered marginal practical utility and are rarely deployed into healthcare settings, leading to proposals for technical and structural improvements. In this paper, we argue that this reflects a violation of Friedman’s “Fundamental Theorem of Biomedical Informatics,” and that a deeper epistemological change must occur in the cNLP field, as a parallel step alongside any technical or structural improvements. We propose that researchers shift away from designing cNLP systems independent of clinical needs, in which cNLP tasks are ends in themselves—“tasks as decisions”—and toward systems that are directly guided by the needs of clinicians in realistic decision-making contexts—“tasks as needs.” A case study example illustrates the potential benefits of developing cNLP systems that are designed to more directly support clinical needs.

An analytical model for information gathering and propagation in social networks using random graphs

In this paper, we propose an analytical model for information gathering and propagation in social networks using random sampling. We represent the social network using the Erdos–Renyi model of the random graph. When a given node is selected in the social network, information about itself and all of its neighbors are obtained and these nodes are considered to be discovered. We provide an analytical solution for the expected number of nodes that are discovered as a function of the number of nodes randomly sampled in the graph. We use the concepts of combinatorics, probability, and inclusion–exclusion principle for computing the number of discovered nodes. This is a computationally-intensive problem with combinatorial complexity. This model is useful when crawling and mining of the social network graph is prohibited. Our work finds application in several important real-world decision support scenarios such as survey sample selection, construction of public directory, and crowdsourced databases using social networks, targeted advertising, and recommendation systems. It can also be used for finding a randomized dominating set of a graph that finds applications in computer networks, document summarization, and biological networks. We have evaluated the performance both analytically as well as by means of simulation, and the results are comparable. The results have an accuracy of around 96% for random graphs and above 87% for the power-law graphs.

Software Agent-Centric Semantic Social Network for Cyber-Physical Interaction and Collaboration

Considerable research has recently focused on integrating cyber-physical systems in a social context. However, several challenges remain concerning appropriate methodologies, frameworks and techniques for supporting socio-cyber-physical collaboration. Existing systems do not recognize how cyber-physical resources can be socially connected so that they interact in collaborative decision-making like humans. Furthermore, the lack of semantic representations for heterogeneous cyber-social-collaborative networks limits integration, interoperability and knowledge discovery from their underlying data sources. Semantic Web ontology models can help to overcome this limitation by semantically describing and interconnecting cyber-physical objects and human participants in a social space. This research addresses the establishment of both cyber-physical and human relationships and their interactions within a social-collaborative network. We discuss how nonhuman resources can be represented as socially connected nodes and utilized by software agents. A software agent-centric Semantic Social-Collaborative Network (SSCN) is then presented that provides functionality to represent and manage cyber-physical resources in a social network. It is supported by an extended ontology model for semantically describing human and nonhuman resources and their social interactions. A software agent has been implemented to perform some actions on behalf of the nonhuman resources to achieve cyber-physical collaboration. It is demonstrated within a real-world decision support system, GRiST (www.egrist.org), used by mental-health services in the UK.

Transparency and accountability in AI decision support: Explaining and visualizing convolutional neural networks for text information

Proliferating applications of deep learning, along with the prevalence of large-scale text datasets, have revolutionized the natural language processing (NLP) field, thereby driving the recent explosive growth. Nevertheless, it is argued that state-of-the-art studies focus excessively on producing quantitative performances superior to existing models, by playing “the Kaggle game.” Hence, the field requires more effort in solving new problems and proposing novel approaches and architectures. We claim that one of the promising and constructive efforts would be to design transparent and accountable artificial intelligence (AI) systems for text analytics. By doing so, we can enhance the applicability and problem-solving capacity of the system for real-world decision support. It is widely accepted that deep learning models demonstrate remarkable performances compared to existing algorithms. However, they are often criticized for being less interpretable, i.e., the “black box.” In such cases, users tend to hesitate to utilize them for decision-making, especially in crucial tasks. Such complexity obstructs transparency and accountability of the overall system, potentially debilitating the deployment of decision support systems powered by AI. Furthermore, recent regulations are emphasizing fairness and transparency in algorithms to a greater extent, turning explanations more compulsory than voluntary. Thus, to enhance the transparency and accountability of the decision support system and preserve the capacity to model complex text data at the same time, we propose the Explaining and Visualizing Convolutional neural networks for Text information (EVCT) framework. By adopting and ameliorating cutting-edge methods in NLP and image processing, the EVCT framework provides a human-interpretable solution to the problem of text classification while minimizing information loss. Experimental results with large-scale, real-world datasets show that EVCT performs comparably to benchmark models, including widely used deep learning models. In addition, we provide instances of human-interpretable and relevant visualized explanations obtained from applying EVCT to the dataset and possible applications for real-world decision support.

Bidirectional approximate reasoning-based approach for decision support

Fuzzy rule-based systems are widely applied for real-world decision support, such as policy formation, public health analysis, medical diagnosis, and risk assessment. However, they face significant challenges when the application problem at hand suffers from the “curse of dimensionality” or “sparse knowledge base”. Combination of hierarchical fuzzy rule models and fuzzy rule interpolation offers a potentially efficient and effective approach to dealing with both of these issues simultaneously. In particular, backward fuzzy rule interpolation (B-FRI) facilitates approximate reasoning to be performed given a sparse rule base where rules do not fully cover all observations or the observations are not complete, missing antecedent values in certain available rules. This paper presents a hierarchical bidirectional fuzzy reasoning mechanism by integrating hierarchical rule structures and forward/backward rule interpolation. A computational method is proposed, building on the resulting hierarchical bidirectional fuzzy interpolation to maintain consistency in sparse fuzzy rule bases. The proposed techniques are utilised to address a range of decision support problems, successfully demonstrating their efficacy.

Using Machine Learning Applied to Real-World Healthcare Data for Predictive Analytics: An Applied Example in Bariatric Surgery

ObjectivesLaparoscopic metabolic surgery (MxS) can lead to remission of type 2 diabetes (T2D); however, treatment response to MxS can be heterogeneous. Here, we demonstrate an open-source predictive analytics platform that applies machine-learning techniques to a common data model; we develop and validate a predictive model of antihyperglycemic medication cessation (validated proxy for A1c control) in patients with treated T2D who underwent MxS. MethodsWe selected patients meeting the following criteria in 2 large US healthcare claims databases (Truven Health MarketScan Commercial [CCAE]; Optum Clinformatics [Optum]): underwent MxS between January 1, 2007, to October 1, 2013 (first = index); aged ≥18 years; continuous enrollment 180 days pre-index (baseline) to 730 days postindex; baseline T2D diagnosis and treatment. The outcome was no antihyperglycemic medication treatment from 365 to 730 days after MxS. A regularized logistic regression model was trained using the following candidate predictor categories measured at baseline: demographics, conditions, medications, measurements, and procedures. A 75% to 25% split of the CCAE group was used for model training and testing; the Optum group was used for external validation. Results13 050 (CCAE) and 3477 (Optum) patients met the study inclusion criteria. Antihyperglycemic medication cessation rates were 72.9% (CCAE) and 70.8% (Optum). The model possessed good internal discriminative accuracy (area under the curve [AUC] = 0.778 [95% CI = 0.761-0.795] in CCAE test set N = 3527) and transportability (external AUC = 0.759 [95% CI = 0.741-0.777] in Optum N = 3477). ConclusionThe application of machine learning techniques to real-world healthcare data can yield useful predictive models to assist patient selection. In future practice, establishment of prerequisite technological infrastructure will be needed to implement such models for real-world decision support.

An empirical study of Bayesian network parameter learning with monotonic influence constraints

Learning the conditional probability table (CPT) parameters of Bayesian networks (BNs) is a key challenge in real-world decision support applications, especially when there are limited data available. A conventional way to address this challenge is to introduce domain knowledge/expert judgments that are encoded as qualitative parameter constraints. In this paper we focus on a class of constraints which is naturally encoded in the edges of BNs with monotonic influences. Experimental results indicate that such monotonic influence constraints are widespread in practical BNs (all BNs used in the study contain such monotonic influences). To exploit expert knowledge about such constraints we have developed an improved constrained optimization algorithm, which achieves good parameter learning performance using these constraints, especially when data are limited. Specifically, this algorithm outperforms the previous state-of-the-art and is also robust to errors in labelling the monotonic influences. The method is applied to a real world medical decision support BN where we had access to expert-provided constraints and real hospital data. The results suggest that incorporating expert judgments about monotonic influence constraints can lead to more accurate BNs for decision support and risk analysis.

Bicentric diagrams: Design and applications of a graph-based relational set visualization technique

In an era where data on social, economic, and physical networks are proliferating at a rapid pace, the ability to understand the underlying complex structural connections, discover prominent entities, and identify clusters is becoming increasingly important. It is also well-established that interactive visualizations can amplify human cognition and augment decision making. Motivated by a practical need articulated by corporate decision makers and limitations of existing visual representations, this research presents our journey in designing and implementing bicentric diagrams, a novel graph-based set visualization technique. A bicentric diagram enables simultaneous identification of sets, set relationships, and set member reach in integrated egonetworks of two focal entities. Our technique builds on the well-established sociological theory of tie strength to visually group and position nodes. We illustrate the broad applicability of bicentric diagrams with examples from four diverse sample domains: university collaboration, technology co-occurrence, health app purchases, and interfirm alliance networks. We assess the value of our technique using an expert-based evaluation approach. The paper concludes with implications and a discussion of opportunities for implementation in real-world decision support settings.

Empirical decision model learning

One of the biggest challenges in the design of real-world decision support systems is coming up with a good combinatorial optimization model. Often enough, accurate predictive models (e.g. simulators) can be devised, but they are too complex or too slow to be employed in combinatorial optimization.In this paper, we propose a methodology called Empirical Model Learning (EML) that relies on Machine Learning for obtaining components of a prescriptive model, using data either extracted from a predictive model or harvested from a real system. In a way, EML can be considered as a technique to merge predictive and prescriptive analytics.All models introduce some form of approximation. Citing G.E.P. Box [1] “Essentially, all models are wrong, but some of them are useful”. In EML, models are useful if they provide adequate accuracy, and if they can be effectively exploited by solvers for finding high-quality solutions.We show how to ground EML on a case study of thermal-aware workload dispatching. We use two learning methods, namely Artificial Neural Networks and Decision Trees and we show how to encapsulate the learned model in a number of optimization techniques, namely Local Search, Constraint Programming, Mixed Integer Non-Linear Programming and SAT Modulo Theories. We demonstrate the effectiveness of the EML approach by comparing our results with those obtained using expert-designed models.

Meta-heuristic improvements applied for steel sheet incremental cold shaping

In previous studies, a wrapper feature selection method for decision support in steel sheet incremental cold shaping process (SSICS) was proposed. The problem included both regression and classification, while the learned models were neural networks and support vector machines, respectively. SSICS is the type of problem for which the number of features is similar to the number of instances in the data set, this represents many of real world decision support problems found in the industry. This study focuses on several questions and improvements that were left open, suggesting proposals for each of them. More specifically, this study evaluates the relevance of the different cross validation methods in the learned models, but also proposes several improvements such as allowing the number of chosen features as well as some of the parameters of the neural networks to evolve, accordingly. Well-known data sets have been use in this experimentation and an in-depth analysis of the experiment results is included. 5\(\times \)2 CV has been found the more interesting cross validation method for this kind of problems. In addition, the adaptation of the number of features and, consequently, the model parameters really improves the performance of the approach. The different enhancements have been applied to the real world problem, an several conclusions have been drawn from the results obtained.

Never say “not”: Impact of negative wording in probability phrases on imprecise probability judgments

Effective translations between numerical and verbal representations of uncertainty are a concern shared by researchers in cognitive science and psychology, with applications to real-world risk management and decision support systems. While there is a substantial literature on such translations for point-wise probabilities, this paper contributes to the scanty literature on imprecise probability translations. Reanalysis of Budescu et al.’s [1] data on numerical interpretations of the Intergovernmental Panel on Climate Change [2] fourth report’s verbal probability expressions (PEs) revealed that negative wording has deleterious effects on lay judgements. Budescu et al. asked participants to interpret PEs in IPCC report sentences, by asking them to provide lower, “best” and upper estimates of the probabilities that they thought the authors intended. There were four experimental conditions, determining whether participants were given any numerical guidelines for translating the PEs into numbers.The first analysis focuses on twelve sentences in Budescu et al. that used the PE “very likely,” “likely,” “unlikely,” or “very unlikely”. A mixed beta regression modelling the lower, “best” and upper estimates revealed a less regressive mean and less dispersion for positive than for negative wording in all three estimates, for both the “very likely” and “likely” sentence sets. The Budescu et al. data also included a task asking for context-free translations of these PEs, and a similar pattern of results was found for that task. Negative wording therefore resulted in more regressive estimates and less consensus regardless of experimental condition.The second analysis focuses on two statements that were positive–negative duals. Appropriate pairs of responses were assessed for conjugacy and additivity. A large majority of respondents were appropriately super- and sub-additive in their lower and upper probability estimates. A mixed beta regression model of these three variables revealed that respondents were suprisingly close to obeying the conjugacy relationships for lower and upper probabilities.

The assembly line balancing and scheduling problem with sequence-dependent setup times: problem extension, model formulation and efficient heuristics

Assembly line balancing problems (ALBP) consist of distributing the total workload for manufacturing any unit of the products to be assembled among the work stations along a manufacturing line as used in the automotive or the electronics industries. Usually, theory assumes that, within each station, tasks can be executed in an arbitrary precedence-feasible sequence without changing station times. In practice, however, the task sequence may influence the station time considerably as sequence-dependent setups (e.g., walking distances, tool changes) have to be considered. Including this aspect leads to a joint balancing and scheduling problem, which we call SUALBSP (setup assembly line balancing and scheduling problem). In this paper, we modify the problem by modeling setups more realistically, give a new, more compact mathematical model formulation and develop effective heuristic solution procedures. Computational experiments based on existing and new data sets indicate that the new procedures outperform formerly proposed heuristics. They are able to solve problem instances of real-world size with small deviations from optimality in computation times short enough to be accepted in real-world decision support systems.

Skyline and mapping aware join query evaluation

Growing interests in multi-criteria decision support applications have resulted in a flurry of efficient skyline algorithms. In practice, real-world decision support applications require to access data from disparate sources. Existing techniques define the skyline operation to work on a single set, and therefore, treat skylines as an “add-on” on top of a traditional Select-Project-Join query plan. In many real-world applications, the skyline dimensions can be anti-correlated such as the attribute pair {price, mileage} for cars and {price, distance} for hotels. Anti-correlated data are particularly challenging for skyline evaluation and therefore have commonly been ignored by existing techniques. In this work, we propose a robust execution framework called SKIN to evaluate skyline over joins. The salient features of SKIN are: (a) effective in reducing the two primary costs, namely the cost of generating the join results and the cost of dominance comparisons to compute the final skyline of join results, (b) shown to be robust for both skyline-friendly (independent and correlated) as well as skyline-unfriendly (anti-correlated) data distributions. SKIN is effective in exploiting the skyline knowledge in both local within individual data sources and across disparate sources—to significantly reduce the above-mentioned costs incurred during the evaluation of skyline over join. Our experimental study demonstrates the superiority of our proposed approach over state-of-the-art techniques to handle a wide variety of data distributions.

From the Editor: In Memoriam, Rick Rosenthal and Mike Rothkopf

The OR community at large, and the OR practice community in particular, lost two great leaders this year. Rick Rosenthal passed away on January 3, 2008 after a hard-fought battle with cancer. Mike Rothkopf died unexpectedly on February 18, 2008 while swimming at the McCoy Natatorium at Perm State University. In this issue we have two papers to honor these colleagues who were so supportive and involved in OR practice. Rick Rosenthal was a Distinguished Professor at the Naval Postgraduate School in Monterey, California. I knew Rick for over 20 years. He was very much focused on practice and solving real-world prob lems. He was instrumental in getting young faculty involved in the INFORMS Practice Meeting held each spring. Last year I became aware of a talk Rick and Jerry Brown had been giving on practical optimiza tion. After some prodding from me, Rick and Jerry began the process of transforming their talk into an Interfaces paper. Their paper, Optimization Trade craft: Hard-Won Insights from Real-World Decision Support, appears in this issue. I thank Jerry for seeing the paper through to completion. I believe aca demics and practitioners (and those of us who con sider ourselves in the intersection) will find this paper very informative. As someone who teaches optimiza tion, I certainly plan to use it in my classes. Mike past president of INFORMS and former editor in chief of this journal, had long been very active in OR practice. Mike had just joined Penn State in the fall of 2007 after many years at Rut gers. I also knew Mike for a long time. When I became the editor in chief of Interfaces, Mike went out of his way to offer me advice based on his experi ence. He also always had provocative and encourag ing words for academics focused on practice. Mike was probably most famous for his work on auctions, and so it is fitting that his colleagues Ron Harstad and Aleksandar Sasa Pekec have written a paper on auctions in Mike's honor. Their paper, Relevance to Practice and Auction Theory: A Memorial Essay to Michael Rothkopf, appears in this issue. It illustrates just how significant Mike's contributions to auctions were and how he emphasized the need to focus on practice. We mourn the loss of these two great friends and colleagues and remain grateful for their contributions and the impact they have had on the profession.