Intelligent Platform Management Interface Research Articles

MTCA control and data acquisition platform for Plasma Diagnostics

The Advanced Telecommunications Computing Architecture (ATCA) instrumentation platform for fusion control and diagnostics developed at IST, is being extended with a Micro Telecommunications Computing Architecture (MTCA) control and data acquisition platform, still able to maintain performance characteristics in speed, channel density and availability of ATCA, at a cost suitable for smaller systems. Software and firmware have been ported to MTCA using mostly in-house developed hardware, preserving many of ATCA's benefits in performance in a smaller form-factor. The first instance of the proposed platform comprises a MTCA 1U shelf with 6 Advanced Mezzanine Card (AMC) slots, MTCA Carrier Hub (MCH) module, AMC to Field-programmable Gate Array (FPGA) Mezzanine Card (FMC) carrier modules, 500 MSPS to 2 GSPS ADC FMC modules, timing and synchronization input/output (IO) module, or other Commercial Off-The-Shelf (COTS) modules. Data is handled by the MCH, featuring PCIe Gen 3 switching to a host computer, as well as performing clock distribution and Intelligent Platform Management Interface (IPMI) based hardware management for high availability. This compact, flexible and autonomous or scalable platform may either integrate a large-experiment system infrastructure, as well as being configured as a lower-cost, self-contained platform for smaller experiments or development and testbench purposes. The focus of this paper is the development of the MCH MTCA.0 module, which is key to introduce the platform architecture.

Impact of Server Thermal Design on the Cooling Efficiency: Chassis Design

There are various designs for segregating hot and cold air in data centers such as cold aisle containment (CAC), hot aisle containment (HAC), and chimney exhaust rack. These containment systems have different characteristics and impose various conditions on the information technology equipment (ITE). One common issue in HAC systems is a pressure build-up inside the HAC (known as backpressure). Backpressure also can be present in CAC systems in case of airflow imbalances. Hot air recirculation, limited cooling airflow rate in servers, and reversed flow through ITE with weaker fan systems (e.g., network switches) are some known consequences of backpressure. Currently, there is a lack of experimental data on the interdependency between overall performance of ITE and its internal design when backpressure is imposed on ITE. In this paper, three commercial 2-rack unit (RU) servers with different internal designs from various generations and performance levels are tested and analyzed under various environmental conditions. Smoke tests and thermal imaging are implemented to study the airflow patterns inside the tested equipment. In addition, the impact of hot air leakage into the servers through chassis perforations on the fan speed and the power consumption of the servers are studied. Furthermore, the cause of the discrepancy between measured inlet temperatures by the intelligent platform management interface (IPMI) and external sensors is investigated. It is found that arrangement of fans, segregation of space upstream and downstream of fans, leakage paths, the location of baseboard management controller (BMC) sensors, and the presence of backpressure can have a significant impact on ITE power and cooling efficiency.

Taking advantage of the intercommunication features of IPMCs in ATCA CDAQ systems

The PCI Industrial Computer Manufacturers Group (PICMG) 3.x AdvancedTCA Base Specification and Intelligent Platform Management Interface (IPMI) standards specify a modular system for Control and Data Acquisition (CDAQ) where the boards can communicate through the Intelligent Platform Management Bus-0 (IPMB-0).CDAQ systems applied to large physics experiments like ITER, are designed, among other features, for High-Availability (HA). A CDAQ system based on the PICMG 3.x AdvancedTCA Base Specification and IPMI standards grants these features. One of the key functions of the HA is the hot swapping possibility of the CDAQ system elements which is granted by those standards. An Advanced Telecommunications Computers Architecture (ATCA) CDAQ system can be designed implementing the Peripheral Component Interconnect Express (PCIe) standard communication protocol in their fabric lines. Despite the fact that this protocol has its own rules for hot swapping, defined as hot-plug, ATCA and PCIe standards are not entirely compatible in what concerns this aspect.Instituto de Plasmas e Fusão Nuclear (IPFN) has developed and implemented a CDAQ system using the ATCA and PCIe standards.This paper describes the software architecture and functions implemented in the microcontroller of the Intelligent Platform Management Controller (IPMC) of the ATCA boards in order to render those standards compatible in what concerns hot-swap and hot-plug processes as an example of the possibilities that can be achieved taking advantage of the intercommunication features of IPMCs through the Intelligent Platform Management Bus-0 (IPMB-0).

Chip to Chiller Experimental Cooling Failure Analysis of Data Centers: The Interaction Between IT and Facility

Cooling failure in data centers (DCs) is a complex phenomenon due to the many interactions between the cooling infrastructure and the information technology equipment (IT). To fully understand it, a system integration philosophy is vital to the testing and design of experiment. In this paper, a facility-level DC cooling failure experiment is run and analyzed. An airside cooling failure is introduced to the facility during two different cooling set points as well as in open and contained environments. Quantitative instrumentation includes pressure differentials, tile airflow, external contour and discrete air inlet temperature, intelligent platform management interface (IPMI), and cooling system data during failure recovery. Qualitative measurements include infrared imaging and airflow visualization via smoke trace. To our knowledge of current literature, this is the first experimental study in which an actual multi-aisle facility cooling failure is run with real IT (compute, network, and storage) load in the white space. This will establish a link between variations from the facility to the central processing unit (CPU). The results show that using the external IT inlet temperature sensors, the containment configuration shows a longer available uptime (AU) during failure. However, the IPMI data show the opposite. In fact, the available uptime is reduced significantly when the external sensors are compared to internal IT analytics. The response of the IT power, CPU temperature, and fan speed shows higher values during the containment failure. This occurs because of the instantaneous formation of external impedances in the containment during failure, which renders the contained aisle to be less resilient than the open aisle. The tradeoffs between PUE, OPEX, and AU are also explained.

Standardized Solution for Management Controller for MTCA.4

The Micro Telecommunications Computing Architecture (MTCA) standard is a modern platform that is gaining popularity in the area of High Energy Physics (HEP) experiments. The standard provides extensive management, monitoring and diagnostics functionalities. The hardware control and monitoring is based on the Intelligent Platform Management Interface (IPMI), that was initially developed for supervision of complex computers operation. The original IPMI specification was extended to support functions required by the MTCA specification. The Module Management Controller (MMC) is required on each Advanced Mezzanine Card (AMC) installed in MTCA chassis. The Rear Transition Modules (RTMs) have to be equipped with RTM Management Controllers (RMCs) which is required by the MTCA.4 subsidiary specification. The commercially available implementations of MMC and RMC are expensive and do not provide the complete functionality that is required by specific HEP applications. Therefore, many research centers and commercial companies work on their own implementation of AMC and RTM controllers. The available implementations suffer because of lack of common approach and interoperability problems. Since both Lodz University of Technology (TUL) and Deutsches Elektronen-Synchrotron (DESY) have long-term experience in developing ATCA and MTCA hardware, the authors decided to develop a unified solution of management controller fully compliant with AMC and MTCA.4 standards. The MMC v1.00 solution is dedicated for management of AMC and RTM modules. The MMC v1.00 is based on Atmel ATxmega MCUs and can be fully customized by the user or used as a drop-in-module without any modifications. The paper discusses the functionality of the MMC v1.00 solution. The implementation was verified with developed evaluation kits for AMC and RTM cards.

Firmware Upgrade in xTCA Systems

The Advanced Telecommunications Computing Architecture (ATCA) and Micro Telecommunications Computing Architecture (μTCA) standards, collectively known as xTCA, provide a flexible and scalable infrastructure for designing complex control and data acquisition systems. The xTCA standards are becoming more and more popular in physics applications. Programmable devices, such as Field Programmable Gate Arrays (FPGAs), conventional and Digital Signal Processors (DSPs) are present in Advanced Mezzanine Card (AMC) modules and ATCA blades used in the xTCA crates. Those devices typically boot from non-volatile memories available on the modules. This paper deals with an universal framework and set of tools for upgrading firmware in such devices in xTCA systems. The proposed framework uses a fat pipe region interface of backplane for the firmware data transmission and the Intelligent Platform Management Interface (IPMI) standard for PROM memory management and the control of the upgrade procedure. This is the world's first attempt to implement the firmware upgrade in system not using the JTAG Switch Module (JSM).

Vector Modulator Card for MTCA-Based LLRF Control System for Linear Accelerators

Modern low-level radio frequency (LLRF) control systems of linear accelerators are designed to achieve extremely precise field amplitude and phase regulation inside superconducting cavities. One of the crucial components of the feedback loop is a vector modulator used to drive the high-power RF chain supplying the accelerating cavities. The LLRF control systems for the Free Electron Laser in Hamburg (FLASH) and European X-ray Free Electron Laser (XFEL) are based on the emerging Micro-Telecommunications Computing Architecture (MTCA.4) platform offering numerous advantages for high-performance control systems. This paper describes the concept, design, and performance of the first vector modulator (uVM) module dedicated for RF controls compliant with the MTCA.4 specification developed by PCI Industrial Computer Manufacturers Group (PICMG). The uVM module has been built as a double-width, mid-size rear transition module (RTM) that is accessed by an advanced mezzanine card (AMC). The uVM module incorporates digital, analog, and diagnostic subsystems. The digital part is based on Xilinx Spartan 6 family field-programmable gate array (FPGA), with several fast gigalink connections to a control module. The uVM module is equipped with an intelligent platform management interface (IPMI) circuit required by the MTCA.4 standard. The FPGA controls the analog part, which includes fast, high-precision digital-to-analog converters (DACs), In-phase and quadrature modulator chips, programmable attenuators, power amplifiers, and fast RF gates for an external interlock system. The RF chain can be adopted to different carrier frequencies covering a frequency range from 50 MHz to 6 GHz. The design has been carefully optimized for high linearity and low-output signal phase noise. The diagnostic system of the RF chain allows monitoring of input and output power levels and failure detection in the RF circuits. A low-noise and high-performance clocking system makes the uVM a universal device, extending the scope of applications beyond the LLRF control systems. Extensive tests of the board were performed, and the measurement results are presented and discussed in this paper.

The ITER Fast Plant System Controller ATCA prototype Real-Time Software Architecture

IPFN is developing a prototype Fast Plant System Controller (FPSC) based in ATCA embedded technologies dedicated to ITER CODAC data acquisition and control tasks in the sub-millisecond range. The main goal is to demonstrate the usability of the ATCA standard and its enhanced specifications for the high speed, very high density parallel data acquisition needs of the most demanding ITER tokamak plasma Instrumentation and Control (I&C) systems. This effort included the in-house development of a new family of high performance ATCA I/O and timing boards.The standard ITER software system CODAC Core System (CCS) v3.1, with the control based in the EPICS system does not cover yet the real-time requirements fulfilled by this hardware, so a new set of software components was developed for this specific platform, attempting to integrate and leverage the new features in CSS, for example the Multithreaded Application Real Time executor (MARTe) software framework, the new Data Archiving Network (DAN) solution, an ATCA IEEE-1588-2008 timing interface, and the Intelligent Platform Management Interface (IPMI) for system monitoring and remote management.This paper presents the overall software architecture for the ATCA FPSC, as well a discussion on the ITER constrains and design choices and finally a detailed description of the software components already developed.

Resource Consumption Framework for Fault Diagnosis in Cloud

In the heterogeneous parallel and distributed computing environments like cloud there were many related approaches proposed for fault tolerant execution of workflows. Most of the earlier works involved does not depend on failure prediction of the resources that is really hard to achieve with the tracing of historic failure data over years of the desired environment. In this paper, to solve the software fault prediction, unavailability of the resources and monitoring problems we propose a failure prediction model that involves two different methods. In order to predict the failures at the nodes we propose a method using Intelligent Platform Management Interface (IPMI), that monitor the failure at nodes and provide the respective data that is useful for determining likely imminent failures. The other method is to predict the Unavailability of the resources from past behavior that generates some initial results that indicate that nodes are different from one another and their failure is somewhat predictable and monitoring is performed which intimates about the failure.

EPICS device support module as ATCA system manager for the ITER fast plant system controller

This paper presents an Enhanced Physics and Industrial Control System (EPICS) device support module for the International Thermonuclear Experimental Reactor (ITER) Fast Plant System Controller (FPSC) project based in Advanced Telecommunications Computing Architecture (ATCA) specification. The developed EPICS device support module provides an External System Manager (ESM) solution for monitoring and control the ITER FPSC ATCA shelf system and data acquisition boards in order to take proper action and report problems to a control room operator or high level management unit in case of any system failure occurrence. EPICS device support module acts as a Channel Access (CA) server to report problems and publish ATCA system data information to the control room operator, high level management unit or other CA network clients such as Control System Studio Operator Interfaces (CSS OPIs), Best Ever Alarm System Toolkit (BEAST), Best Ever Archive Utility (BEAUTY) or other CA client applications. EPICS device support module communicates with the ATCA Shelf manager (ShM) using HTTP protocol to send and receive commands through POST method in order to get and set system and shelf components properties such as fan speeds measurements, temperatures readings, module status and ATCA boards acquisition and configuration parameters. All system properties, states, commands and parameters are available through the EPICS device support module CA server in EPICS Process Variables (PV) and signals format. ATCA ShM receives the HTTP protocol messages from the EPICS device support module, validates and takes proper action in order to execute and acknowledge the required command operation and respond with an operation success or error return code. Each ATCA board contains inventory data that can be retrieved by the ShM controller to determine if there is enough power available for the payload process, provide manufacturer data information, retrieve board model and serial numbers and check number of available board sensors their types and fields. The ShM can provide this inventory data information to the EPICS device support module if is required. ShM module communicates with the ATCA boards through the Intelligent Platform Management Interface (IPMI) controller via IPMI communication protocol on redundant IPMI bus. This communication protocol is managed by the embedded Intelligent Management Platform Controller (IPMC) in the ATCA board side and by the embedded Shelf Management Controller (ShMC) in the ShM side. Queried inventory data information becomes available in the EPICS device support module upon ShM command response. EPICS device support module can be a valuable and good approach for ATCA ESM aiming the integration in the ITER Control and Data Acquisition (CODAC) core system according the ITER specification.

Real-Time IPMI Protocol Analyzer

The Advanced Telecommunications Computing Architecture (ATCA) is a modern platform, which gains popularity, not only in telecommunication applications, but also in others fields like High Energy Physics (HEP) experiments. Computing systems based on ATCA provide high performance and efficiency and are characterized by significant reliability, availability and serviceability. ATCA offers these features because of an integrated management system realized by the Intelligent Platform Management Interface (IPMI) implemented on dedicated Intelligent Platform Management Controller (IPMC). IPMC is required on each ATCA board to fulfill the ATCA standard and is responsible for many vital procedures performed to support proper operation of ATCA system. It covers, among others, activation and deactivations of modules, monitoring of actual parameters or controlling fans. The commercially available IPMI implementations are expensive and often not suited to demands of specific ATCA applications and available hardware. Thus, many research centers and commercial companies decide to develop their own version of IPMC software. Despite precise IPMI specification of communication requests and responses, these implementations are often incompatible with each other, which leads to incorrect in-system behavior of devices equipped with IPMCs from various vendors. ATCA specifies the I2C protocol as a physical layer of IPMI. There are many devices able to monitor the I2C bus such as logic analyzers or specialized oscilloscopes. However, there is no available equipment capable of debugging IPMI as a higher level protocol. The article compares available methods of IPMI debugging and describes a custom made device prepared to monitor in realtime up to eight IPMI lines and analyze the IPMI protocol. Accessibility of this kind of equipment allows to discover errors and find the reasons of faulty behavior of the IPMC under development, greatly reduce the time to market factor and decrease costs of ATCA system development.