Live Production Environment Research Articles

Validating, Implementing, and Monitoring Machine Learning Solutions in the Clinical Laboratory Safely and Effectively.

Machine learning solutions offer tremendous promise for improving clinical and laboratory operations in pathology. Proof-of-concept descriptions of these approaches have become commonplace in laboratory medicine literature, but only a scant few of these have been implemented within clinical laboratories, owing to the often substantial barriers in validating, implementing, and monitoring these applications in practice. This mini-review aims to highlight the key considerations in each of these steps. Effective and responsible applications of machine learning in clinical laboratories require robust validation prior to implementation. A comprehensive validation study involves a critical evaluation of study design, data engineering and interoperability, target label definition, metric selection, generalizability and applicability assessment, algorithmic fairness, and explainability. While the main text highlights these concepts in broad strokes, a supplementary code walk-through is also provided to facilitate a more practical understanding of these topics using a real-world classification task example, the detection of saline-contaminated chemistry panels.Following validation, the laboratorian's role is far from over. Implementing machine learning solutions requires an interdisciplinary effort across several roles in an organization. We highlight the key roles, responsibilities, and terminologies for successfully deploying a validated solution into a live production environment. Finally, the implemented solution must be routinely monitored for signs of performance degradation and updated if necessary. This mini-review aims to bridge the gap between theory and practice by highlighting key concepts in validation, implementation, and monitoring machine learning solutions effectively and responsibly in the clinical laboratory.

Optimizing OSPF Cost Value for Best Path Selection Using Metric-Type Variation

Open Shortest Path First (OSPF) is a multi-vendor routing standard. Any routing protocol's goal is to discover the optimum path between source and destination addresses inside a computer network. Other routing protocols include Routing Information Protocol (RIP), Border Gateway Protocol (BGP), Enhanced Interior Gateway Routing Protocol (EIGRP), and IS-IS. which are also used in computer networking. Each routing protocol has different set of rules through which they calculate the best path selection. OSPF is often used with the combination of these routing protocol because of its flexible design. This research focuses on the implementation of OSPF network cost (metric) value for best path calculation using network simulation tool like GNS3 for getting correct outcome just like a real networking device working in a live production environment. This research would use the different combinations of OSPF cost calculation using metric-types for external routes learned from different routing protocols like RIP. Being able to accurately identify cost (metric) values obtain by testing for OSPF when used with other routing protocols, just like a real-world environment.

Scheduling on a budget: Avoiding stale recommendations with timely updates

Recommendation systems usually create static models from historical data. Due to concept drift and changes in the environment, such models are doomed to become stale, which causes their performance to degrade. In live production environments, models are therefore typically retrained at fixed time-intervals. Of course, every retraining comes at a significant computational cost, making very frequent model updates unrealistic in practice. In some cases, the cost is worth it, but in other cases an update could be redundant and the cost an unnecessary loss. The research question then consists of finding an acceptable update schedule for your recommendation system, given a limited budget. This work provides a pragmatic analysis of model staleness for a variety of collaborative filtering algorithms in news and retail domains, where concept drift is a known impediment. We highlight that the rate at which models become stale is highly dependent on the environment they perform in and that this property can be derived from data. These findings are corroborated by empirical observations from four large-scale online experiments. Instead of retraining at regular intervals, we propose an adaptive scheduling method that aims to maximise the accuracy of the recommendations within a fixed resource budget. Offline experiments show that our proposed approach improves recommendation performance while keeping the cost constant. Our findings can guide practitioners to spend their available resources more efficiently.

Systemizing ST 2110 and Simplifying IP-Based Production

In moving to a SMPTE ST 2110 IP platform and the very application-specific benefits such as end-point scale, format agnosticism, flexibility, and ease of integration into wider IP/software workflows that such a move brings to live production, it becomes apparent that full use is not being made of the other benefits that can come in transitioning to an IP-based platform. This article examines some of the technologies and working practices that are used by many mainstream IP adopters—enterprises and cloud providers—and outlines a route for improving the systemization of SMPTE ST 2110 live production environments. When adopted, such improvements can lead to faster network creation, easier facility updates and changes, and greater visibility of the benefits that accrue with such technology.

Integrating the Fairlight CVI into the video workflow

The Fairlight Computer Video Instrument (CVI) is one of the earliest video synthesisers, released in 1984. Over time the aesthetic of its effects has evolved from revolutionary to retro. Although MIDI was developed at around the same time, the CVI is controllable via RS232 rather than MIDI. This paper reviews its history and applications in live and studio-based video production environments. A method is outlined for controlling the CVI that allows sequencing of control data, effect automation, and integration with a digital audio workstation. Results are presented from a system using Logic Pro X for sequencing and Processing for MIDI-serial conversion.

Hatch: Self-distributing systems for data centers

Designing and maintaining distributed systems remains highly challenging: there is a high-dimensional design space of potential ways to distribute a system’s sub-components over a large-scale infrastructure; and the deployment environment for a system tends to change in unforeseen ways over time. For engineers, this is a complex prediction problem to gauge which distributed design may best suit a given environment. We present the concept of self-distributing systems, in which any local system built using our framework can learn, at runtime, the most appropriate distributed design given its perceived operating conditions. Our concept abstracts distribution of a system’s sub-components to a list of simple actions in a reward matrix of distributed design alternatives to be used by reinforcement learning algorithms. By doing this, we enable software to experiment, in a live production environment, with different ways in which to distribute its software modules by placing them in different hosts throughout the system’s infrastructure. We implement this concept in a framework we call Hatch, which has three major elements: (i) a transparent and generalized RPC layer that supports seamless relocation of any local component to a remote host during execution; (ii) a set of primitives, including relocation, replication and sharding, from which to create an action/reward matrix of possible distributed designs of a system; and (iii) a decentralized reinforcement learning approach to converge towards more optimal designs in real time. Using an example of a self-distributing web-serving infrastructure, Hatch is able to autonomously select the most suitable distributed design from among ≈700,000 alternatives in about 5 min.

Generic Architecture for the Automatic Parametrization of Production Machine Assembly Programs

In a high-mix low volume production environment, time to market is a key factor. However, one bottleneck lies in the often times manual parametrization of machines for new or modified designs. A truly flexible manufacturing environment therefore requires a continuous data flow from the design stages to the shop floor. This paper presents a concept for the automated parameterization of machines at a large automotive plant. Therefore, this paper initially discusses the results of a stakeholder analysis. The stakeholders comprise of different departments related to the product design and manufacturing processes. The requirements resulting from the interviews conducted with the stakeholders are grouped and ordered by priority. Secondly a general architecture for the control interface is presented. It includes multiple submodules, which model the continuous data flow between the departments and the production systems. The first main step of the data flow is the transformation of the information which is presented in various styles depending on the source departments to structured and standardized data. Thereafter machine parameters are generated automatically by a submodule using the structured input data and inference rules. Finally, the architecture supports the automatic transfer of the machine readable output data to the assembly line. To test the architecture a prototype comprising of more than 100 robots in a live production environment is implemented. It allows for a continuous data flow from the design and productions planning department to the robots. This enables a flexible process control, which up to today has supported the fast roll out of more than 100 new product variants. In contrast to the conventional manual setup of the machines for operation, the prototype was able to show that the monetary and time expenditure could be reduced by 95 percent.

Requirements for the Hybrid Media Production Facility of the Future

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">People who began their careers in television broadcasting before the 1990s should have seen television and media technical infrastructures endure three significant transitions: standard definition (SD) analog to SD digital; SD digital to high definition (HD) digital; and HD digital to media carried over an Internet Protocol (IP) network in multiple formats. Each transition involved either an infrastructure replacement or a complete rebuild of their technical facilities. Most of the gear and much of the cabling likely had to be replaced, updated, or refined. As changes to the system were made, compressed video, storage, and data management adjusted accordingly. New terminologies evolved, sometimes heightened by “marketing hype,” that drove users to amend workflows, processes, and capital budgets like revolving doors in a hotel. We live in an age of continual transformation where formats, transport methods, and delivery have moved in full strength to yet another dimension—the era of IP. Yet again the industry is being thrust into yet another significant change in infrastructure, which now includes cloud, realtime over-the-top (OTT) streaming, and virtualization. How does one design a facility for these kinds of transitions without needing a forklift upgrade every decade? These are serious topics that impact return on investment (ROI), timing, and capital versus operational alterations. This article examines new hybrid models for media production, explores their components, and gives examples of how to compose the media future for live production environments at the studio and enterprise levels.</i>

Design and Implementation of a Pediatric ICU Acuity Scoring Tool as Clinical Decision Support.

Pediatric in-hospital cardiac arrest most commonly occurs in the pediatric intensive care unit (PICU) and is frequently preceded by early warning signs of clinical deterioration. In this study, we describe the implementation and evaluation of criteria to identify high-risk patients from a paper-based checklist into a clinical decision support (CDS) tool in the electronic health record (EHR). The validated paper-based tool was first adapted by PICU clinicians and clinical informaticians and then integrated into clinical workflow following best practices for CDS design. A vendor-based rule engine was utilized. Littenberg's assessment framework helped guide the overall evaluation. Preliminary testing took place in EHR development environments with more rigorous evaluation, testing, and feedback completed in the live production environment. To verify data quality of the CDS rule engine, a retrospective Structured Query Language (SQL) data query was also created. As a process metric, preparedness was measured in pre- and postimplementation surveys. The system was deployed, evaluating approximately 340 unique patients monthly across 4 clinical teams. The verification against retrospective SQL of 15-minute intervals over a 30-day period revealed no missing triggered intervals and demonstrated 99.3% concordance of positive triggers. Preparedness showed improvements across multiple domains to our a priori goal of 90%. We describe the successful adaptation and implementation of a real-time CDS tool to identify PICU patients at risk of deterioration. Prospective multicenter evaluation of the tool's effectiveness on clinical outcomes is necessary before broader implementation can be recommended.

WSMeter

Evaluating the comprehensive performance of a warehouse-scale computer (WSC) has been a long-standing challenge. Traditional load-testing benchmarks become ineffective because they cannot accurately reproduce the behavior of thousands of distinct jobs co-located on a WSC. We therefore evaluate WSCs using actual job behaviors in live production environments. From our experience of developing multiple generations of WSCs, we identify two major challenges of this approach: 1) the lack of a holistic metric that incorporates thousands of jobs and summarizes the performance, and 2) the high costs and risks of conducting an evaluation in a live environment. To address these challenges, we propose WSMeter, a cost-effective methodology to accurately evaluate a WSC's performance using a live production environment. We first define a new metric which accurately represents a WSC's overall performance, taking a wide variety of unevenly distributed jobs into account. We then propose a model to statistically embrace the performance variance inherent in WSCs, to conduct an evaluation with minimal costs and risks. We present three real-world use cases to prove the effectiveness of WSMeter. In the first two cases, WSMeter accurately discerns 7% and 1% performance improvements from WSC upgrades using only 0.9% and 6.6% of the machines in the WSCs, respectively. We emphasize that naive statistical comparisons incur much higher evaluation costs (&gt; 4 times) and sometimes even fail to distinguish subtle differences. The third case shows that a cloud customer hosting two services on our WSC quantifies the performance benefits of software optimization (+9.3%) with minimal overheads (2.3% of the service capacity).

Dynamic Software Update from Development to Production.

Dynamic Software Update (DSU) solutions update applications while they are executing. These solutions are typically used in production to minimize application downtime, or in integrated development environments to provide live programming support. Each of these scenarios presents different challenges, forcing existing solutions to be designed with only one of these use cases in mind. For example, DSUs for live programming typically do not implement safe point detection or instance migration, while production DSUs require manual generation of patches and lack IDE integration. Also, these solutions have limited ability to update themselves or the language core libraries, and some of them present execution penalties outside the update window. We propose a DSU (gDSU) that works for both live programming and production environments. Our solution implements safe update point detection using call stack manipulation and a reusable instance migration mechanism to minimize manual intervention in patch generation. Moreover, it also offers updates of core language libraries and the update mechanism itself. This is achieved by the incremental copy of the modified objects and an atomic commit operation. We show that our solution does not affect the global performance of the application and it presents only a run-time penalty during the update window. Our solution is able to apply an update impacting 100,000 instances in 1 second. In this 1 second, only during 250 milliseconds the application is not responsive. The rest of the time the application runs normally while gDSU is looking for the safe update point. The update only requires to copy the elements that are modified.

Do fecal and litter microbiomes vary within the major areas of a commercial poultry house, and does this affect sampling strategies for whole-house microbiomic studies?

SUMMARY The microbiota of the live production environment can directly shape the gastrointestinal microbiome of chickens and indirectly influence the health of birds. Therefore, numerous studies have attempted to characterize the microbial communities in litter and chicken feces from commercial poultry houses, but many treat a poultry house as a single sampling unit. Unfortunately, a poultry house has distinct areas (e.g., waterer/feeder, near fans, cooling pads), so effective sampling strategies need to be developed to account for this heterogeneity, especially when costlier microbiomic analyses are used to assess whole-house microbial diversity. Therefore, the goals of this study were to assess the spatial variability of the poultry litter and fecal microbiomes from distinct areas within a poultry house and to compare composite “whole-house” microbiomes pooled from all house areas, either (1) physically prior to DNA extraction or (2) combined in silico after sample processing (during DNA sequence analysis). No significant differences in α-diversity metrics (richness, diversity, evenness) were observed for fecal or litter microbiomes recovered from the different areas of the house, but β-diversity (litter only) and genus-level Operational Taxonomic Units (OTU) relative abundances (for fecal and litter) were found to be significantly different based on sampling area within the house. Additionally, the sample pooling method produced distinct composite fecal microbiomic profiles, but the litter microbiomes were unaffected. These results indicate that sample type, sampling area, and sample pooling method need to be carefully considered when determining appropriate sampling strategies for generating representative composite whole-house microbiomes for future microbiological-based live production studies.

The VRT Sandbox LIVE IP Experience

The awards winning Live IP Project is an exploration of the possibilities and opportunities achievable with today’s IP-enabled broadcast technology in a live production environment. This paper gives an overview of the facts and findings from this hands-on project to share it with the broadcasting community, to advance knowledge of the current state of the technology, and to identify areas where further work is needed. The project has shown that it is possible to build and operate a live IP production studio based on open standards in a multivendor environment.

A Procedure for Collecting and Labeling Man-in-the-Middle Attack Traffic

In this work, we outline a procedure for collecting and labeling Man-in-the-Middle (MITM) attack traffic. Our capture procedure allows for the collection of real-world representative data using a full-scale network environment. MITM attacks are typically performed with the purpose of intercepting information amongst two networked machines. This enables the attacker to gain access to otherwise confidential communications and potentially alter said communications maliciously. MITM attacks are still a very common attack that can be implemented with relative ease across a variety of network environments. Our work establishes experimental procedures for enacting three prevalent MITM attack variants through penetration testing. The process for data collection is defined, along with our approach on gathering real-world, representative data. We also present a novel labeling procedure based on the inherent behaviors of each MITM attack variant. Our work aims to address the challenges associated with collecting such data within a live production environment, as well as identify the impact MITM attacks have on traffic behavior. We also present a case study to provide some quantitative analysis regarding the data collected.

Build me a Ubicomp: bespoke ubiquitous accessibility in live television production environments

Live television production remains driven by platforms which are modelled on systems developed before digitised technology, with specialised components and systems which were designed and developed entirely for a non-disabled plurality. The effect of this is that skilled production staff who become disabled are unable to continue within their roles, in many cases becoming forced into leaving the television industry entirely. This investigation explores the possibility of using bespoke ubiquitous computing systems to circumvent existing practical and strategic restrictions upon reasonable adjustments in production roles. To make our findings, we draw upon twelve criticality-informed interviews with both production specialists and assistive technology experts, and an ethnographic study conducted in a television production environment. This investigation had a particular emphasis upon what practices are (legally) reasonable to adjust in a production environment, and thus allow the realistic targeting of adjustments to particular combinations of roles and disabilities. Through doing so, we describe a space for re-configuring existing user interfaces, practices and workflows in the production environment, introducing a new paradigm of bespoke assistive technologies. We also discuss the novel implications for both disability discrimination law and ubiquitous computing that arise from our investigation.

Automatic Multiple Sensor Data Acquisition System in a Real-time Production Environment

This paper presents a multiple sensor automatic data acquisition system deployed on a CNC turning center in a real- time production environment at Schivo Precision, Waterford, Ireland. The machine has been fitted with a variety of sensors measuring acoustic emission, cutting force & vibration installed at the turret of the machine, coupled with an automatic image acquisition system monitoring the wear on the tools in real time after each operation. The combination of sensors and data acquisition is novel in that it brings together all the currently popular sensoring techniques in the field of tool condition monitoring and tests the validity of these techniques in a live production environment. The sensor data acquisition and optical tool wear measurements are controlled by a computer automatically with no intervention from the machine tool operator in normal production on a variety of component geometries, materials and cutting inserts. All the acquired data is available on-line for the research partners in the various countries. Independent operator feedback on the performance of the process in terms of both product dimensional stability and machined surface stability is used for evaluation of the acquired data applicability for tool condition monitoring.

A random-key encoded harmony search approach for energy-efficient production scheduling with shared resources

When seeking near-optimal solutions for complex scheduling problems, meta-heuristics demonstrate good performance with affordable computational effort. This has resulted in a gravitation towards these approaches when researching industrial use-cases such as energy-efficient production planning. However, much of the previous research makes assumptions about softer constraints that affect planning strategies and about how human planners interact with the algorithm in a live production environment. This article describes a job-shop problem that focuses on minimizing energy consumption across a production facility of shared resources. The application scenario is based on real facilities made available by the Irish Center for Manufacturing Research. The formulated problem is tackled via harmony search heuristics with random keys encoding. Simulation results are compared to a genetic algorithm, a simulated annealing approach and a first-come-first-served scheduling. The superior performance obtained by the proposed scheduler paves the way towards its practical implementation over industrial production chains.

Association of Campylobacter spp. levels between chicken grow-out environmental samples and processed carcasses

Campylobacter spp. have been isolated from live poultry, production environments, processing facilities, and raw poultry products. Environmental sampling in a poultry grow-out house, combined with carcass rinse sampling from the same flock, may provide a relative relationship between pre- and postharvest Campylobacter contamination. Air samples, fecal/litter samples, and feed/drink line samples were collected from 4 commercial chicken grow-out houses in western Virginia between September 2011 and January 2012. Birds from each sampled house were the first flock slaughtered the following day and were then sampled by postchill carcass rinses. Campylobacter, from postenrichment samples, was detected in 27% (32/120) of house environmental samples and 37.5% (45/120) of carcass rinse samples. All environmental sample types from each house included at least one positive sample except the house 2 air samples. The sponge sample method was found to have a significantly higher (P < 0.05) proportion of Campylobacter-positive samples (45%) than the fecal/litter samples (20%) and air samples (15%) when sample types of all the houses were compared. The proportion positive for the fecal/litter samples postenrichment, for each flock, had the highest correlation (0.85) to the proportion of positive carcass rinse samples for each flock. Environmental samples from house 1 and associated carcass rinses accounted for the largest number of Campylobacter positives (29/60). The fewest number of Campylobacter positives, based on both house environmental (4/30) and carcass rinse samples (8/30), was detected from flock B. The results of this study suggest that environmental sampling in a poultry grow-out house, combined with carcass rinse sampling from the same flock, have the potential to provide an indication of Campylobacter contamination and transmission. Campylobacter qualitative levels from house and processing plant samples may enable the scheduled processing of flocks with lower pathogen incidence or concentrations, as a way to reduce postslaughter pathogen transmission.

Nuggets and MXF: Making the Networked Studio a Reality

Nuggets 1 is a European project researching the use of MXF files on IT networks, allowing remote control of cameras in a live production environment. The project partners (Thomson Grass Valley, ZDF, Snell & Wilcox, T-Systems, Enertec, INESC Porto) are creating a local area network/ wide-area network (LAW/WAN)-based workflow that will be demonstrated in 2004. This paper will outline many of the issues discovered in the process of moving from a traditional tape/SDI point-to-point environment to a server/file interchange environment. Details of the proposed network functionality will be given in order to generate audience discussion of the requirements of a file interchange-based workflow.