Firmware Upgrade Research Articles

Analysis and Development of Security Framework for IOT Device

Internet of Things (IoT) devices are growing rapidly, making security and privacy crucial. This study analyses and develops an IoT-specific security framework. To address the specific security problems of IoT devices and provide effective techniques and measures to safeguard sensitive data, minimize vulnerabilities, and assure IoT system integrity. The analysis phase identifies and evaluates IoT ecosystem security issues such poor authentication, data encryption, susceptible firmware and software, and lack of standardization. The framework examines these difficulties to understand the security landscape and build effective countermeasures. The analysis informs the development phase, which includes device authentication and access control, data encryption and privacy protection, secure firmware and software upgrades, and standardization and compliance. These steps increase IoT device security by guaranteeing secure communication, data integrity, and protection against unauthorized access and assaults. This Paper Proposed security framework, to assure the security framework's originality and efficacy, it is developed methodically. IoT security literature, best practices, and upcoming technologies are researched extensively. The suggested security architecture is projected to improve IoT device trustworthiness and reliability, promoting their wider usage across domains and generating lightweight cryptography techniques. This analysis-driven and comprehensive methodology addresses security issues to help build secure and resilient IoT ecosystems.

Fast Model Update for IoT Traffic Anomaly Detection With Machine Unlearning

It is often needed to update deep learning-based detection models in traffic anomaly detection systems for the Internet of Things (IoT) because of mislabelled samples or device firmware upgrades. Machine unlearning, a technique that quickly updates the anomaly detection model without re-training the model from scratch, has recently attracted much research attention. We propose a novel machine unlearning method, called ViFLa, which groups training data based on estimated unlearning probability and treats each group as a virtual client in the federated learning framework. Since the virtual clients are physically in the same machine, ViFLa only leverages the concept of data/local models isolation in federated learning without incurring any network communication. ViFLa adopts an attention-based aggregation method called enhanced class distribution weighted sum (ECDWS) to tackle the non-independent and identically distributed (non-IID) data problem caused by the data grouping strategy. It also introduces a new state transition ring mechanism into the statistical query (SQ) learning framework to update the local model of each virtual client quickly. Using real-world IoT traffic data, we showcase the benefit of ViFLa regarding its efficiency and completeness for model updates in the context of IoT traffic anomaly detection.

PMFault: Faulting and Bricking Server CPUs through Management Interfaces

Apart from the actual CPU, modern server motherboards contain other auxiliary components, for example voltage regulators for power management. Those are connected to the CPU and the separate Baseboard Management Controller (BMC) via the I2C-based PMBus. In this paper, using the case study of the widely used Supermicro X11SSL motherboard, we show how remotely exploitable software weaknesses in the BMC (or other processors with PMBus access) can be used to access the PMBus and then perform hardware-based fault injection attacks on the main CPU. The underlying weaknesses include insecure firmware encryption and signing mechanisms, a lack of authentication for the firmware upgrade process and the IPMI KCS control interface, as well as the motherboard design (with the PMBus connected to the BMC and SMBus by default). First, we show that undervolting through the PMBus allows breaking the integrity guarantees of SGX enclaves, bypassing Intel’s countermeasures against previous undervolting attacks like Plundervolt/V0ltPwn. Second, we experimentally show that overvolting outside the specified range has the potential of permanently damaging Intel Xeon CPUs, rendering the server inoperable. We assess the impact of our findings on other server motherboards made by Supermicro and ASRock. Our attacks, dubbed PMFault, can be carried out by a privileged software adversary and do not require physical access to the server motherboard or knowledge of the BMC login credentials. We responsibly disclosed the issues reported in this paper to Supermicro and discuss possible countermeasures at different levels. To the best of our knowledge, the 12th generation of Supermicro motherboards, which was designed before we reported PMFault to Supermicro, is not vulnerable.

Research on Remote Online Firmware Upgrade System for Embedded Devices

<p>To effectively reduce the cost of firmware upgrade and maintenance of embedded devices, according to the requirement of a remote firmware update of embedded devices, this paper proposes a remote online firmware upgrade technology based on ZigBee. Combined with the requirements of remote online firmware upgrade application of tower-mounted amplifier, this paper analyzes and studies the system requirements, ZigBee module selection and design, software design and implementation (communication protocol design of host and slave program, breakpoint continuation function), fault analysis and other aspects. Designed experiments and tested the ZigBee connection success rate under the device startup state and normal operation state, the relationship between transmission firmware size and transmission success rate, system CPU resource consumption, transmission distance, transmission rate, the success rate, data reuse rate and acceleration ratio of breakpoint continuation function, and other performance indicators. The experimental results show that the remote online firmware upgrade technology based on ZigBee for embedded devices can better meet the system’s performance requirements in all aspects and improve the data reuse rate and acceleration ratio in the process of data transmission. The experimental results show that this scheme has the advantages of simplicity, low cost, high reliability, and broad application prospects.</p> <p> </p>

Design of Remote Upgrade System for Data Processing Unit in Marine Engine Room Simulator

With the development of ship intelligence, the frequency of upgrading the marine engine room simulator, which is essential for crew training, has increased. Traditionally, the data processing unit (DPU) of the marine engine room simulator is upgraded by manually downloading the firmware. This makes the hardware maintenance high-cost. In this paper, we first propose a WAN-based firmware upgrade system to enable secure over-the-air upgrades of DPUs and reduce operation and maintenance costs. A distributed hardware structure is given to manage DPU in the simulator via the Internet. We have designed two methods of firmware upgrades, automatic upgrades and remote upgrades. In automatic upgrades, the DPU can download new firmware upgrades from the web server through the router. By designing a series of mechanisms including code rollback, code backup and code confirmation, the In-Application Programming (IAP) technique is realized through the Internet. Firmware upgrades have good fault tolerance mechanisms to ensure that the emulator can still work in the event of an upgrade error. In remote upgrades, we upgrade the DPU firmware through the remote control center. We assessed the performance of the system by measuring the success rate of DPU upgrades, upgrade time and performance after the upgrade. The results show that the DPU upgrade success rate is close to 100% and performance is as good as expected. The results show that the remote firmware upgrade system proposed in this paper is reliable and practical.

Abstract 9977: Concerns Raised Around Adverse Event Reporting Associated with the Subcutaneous Implantable Cardioverter Defibrillator Device

Objective: To review adverse events reports for the subcutaneous implantable cardioverter defibrillator (S-ICD) device and understand challenges in its application. Background: The S-ICD is an increasingly popular alternative to transvenous ICD (TV-ICD) devices since FDA approval in 2012. Although we have the benefit of professional guidelines, there are no high-quality trials showing mortality benefit. There are concerns about higher rates of inappropriate shocks with use of S-ICD, which prompted a ‘Smartpass’ firmware upgrade in 2016. The impact of this upgrade on rates of inappropriate shocks remains ill-defined. Methods: Narrative reports of adverse events associated with the S-ICD reported to the FDA MAUDE database were reviewed for reason for reporting and outcomes. These were compared with the total number of devices found on shareholder reports as of October 2020. Results: Of 35,000 S-ICD devices in the US, 9,349 adverse events and 2,521 inappropriate shocks were reported in MAUDE. There were 108 deaths, 21 of which were due to a failure to convert a ventricular arrhythmia. Rates of inappropriate shocks comprised an increasing proportion of adverse events year over year with no decline following the ‘Smartpass’ upgrade. Conclusion: The multiple instances of death following S-ICD failure are alarming as these fatal events are thought to be rare for TV-ICDs. Additionally, even with the introduction of Smartpass, inappropriate shocks have not declined. These findings, taken in conjunction with the lack of data for mortality benefit of the S-ICD and safety concerns surrounding the recent lead recall, should be considered by clinicians in future decisions about S-ICD use.

An implementation of module management controller for MicroTCA data processing system

On the basis of full compatibility with advanced telecommunications computing architecture (ATCA) standard, micro telecommunications computing architecture (MicroTCA) standard has a evident advantage over ATCA in terms of space, cost and scalability. The MicroTCA standard has become a popular platform in high energy physics. For the advanced mezzanine card (AMC) installed in the MicroTCA chassis, module management controller (MMC) is a prerequisite to fulfill its function. Although there are many MMC solutions, most of their firmware work in polling mode and rarely use the real-time operating system (RTOS). In order to overcome the lack of real-time performance in polling MMC solution, the novel firmware is developed based on FreeRTOS, which is an open source real-time operating system. This firmware implements the basic functions of MMC, such as hot swap, power management, firmware upgrade, and adds the feature of serial command debugging. To verify the difference in real-time performance between the FreeRTOS and the polling solutions, the interrupt reaction time test is carried out on both solutions. The test shows that the average interrupt reaction time of the FreeRTOS solution is about 300 ms shorter than that of the polling method. This result proves that the FreeRTOS-based solution can significantly improve the real-time performance of transaction processing.

Identity of Things: Applying concepts from Self Sovereign Identity to IoT devices

Identity is a crucial property of Internet of Things (IoT) devices. Due to rapid growth and high numbers of similar devices, reliable identification of those devices is a problem. The origin and history of an IoT device is especially important in security relevant environments. Our research addresses this issue by proposing an approach based on blockchain and decentralized identifiers (DID). It is inspired by the concepts of self-sovereign identity (SSI) and bootstrapping of remote secure key infrastructures (BRSKI). Devices are equipped by the manufacturer with an identity stored in a trusted execution environment (TEE) and secured by a blockchain. This identity can be used to trace back the origin of the device. During the bootstrapping process on the customer side, the identity registration of the device is updated in the blockchain. This process is performed by a so-called registrar. Smart contracts prevent unsolicited transfer of ownership and track the history of the device. Besides proof of origin and device security our concept can be used for device inventory and firmware upgrade. A prototype implementation was realized to validate the concept. All six use cases have been implemented and tested using an Ethereum blockchain infrastructure. JSON Web Tokens (JWT) have been used as signed artefacts to transfer information between the stakeholders. This enables an asynchronous communication needed for example in an environment with no direct internet access. Such an infrastructure can be provided by an independent association and can be used by all manufacturers. Depending on the environment, a registration of devices can be optional or mandatory.

Availability Improvements through Data Slicing in PLC Smart Grid Networks.

An electrical power grid, is an interconnected network for delivering electricity from producers to consumers. Electrical grids vary in size from covering a single building through national grids (which cover whole countries) to transnational grids (which can cross continents). As the rollout of smart meters continues worldwide, there are use-cases where common solutions fail and the network availability of certain meters is very low due to poor communication conditions. This paper proposes a data slicing model for large data files which have to travel securely and reliably throughout the Smart Grid. The manuscript addresses improvements for PRIME PLC network availability by using correct data slicing at the application level along a tuned transmission rate in accordance with the noise levels of the power grid. Successful communications, even at low rates, mean that no manual interaction from energy supplier operators is needed reducing the maintenance costs for both the energy companies as well as for the end user. Experiments on a low power electrical grid setup have been performed in order to evaluate availability improvements through the proposed method as well as the feasibility of remote firmware upgrades. The results have shown that the current approach has similar upgrade time results with a manual firmware upgrade performed through an optical probe. Moreover, the results show that the presented remote firmware upgrade method is reliable and practical.

Detecting Behavioral Change of IoT Devices Using Clustering-Based Network Traffic Modeling

The Internet of Things (IoT) is increasingly becoming a major challenge for network administrators to manage connected devices and sensors ranging from smart lights to smoke alarms and security cameras, at scale. IoT devices use an extensive variety of firmware and provide little (or no) access for the management of their operating systems and configurations. Operators of the IoT infrastructure, therefore, need to employ traffic classification models (trained by historical data) to automatically detect their assets on the network and ensure the health of devices against cyber attacks by monitoring their network behavior. On the other hand, IoT manufacturers often automatically perform firmware upgrades from cloud servers to devices that are operational in the field. This can potentially lead to a change of device behavior which makes it difficult for network operators to maintain classification models (incorporating changes without retraining the entire model). In this article, we develop a modular device classification architecture that allows operators to automatically detect IoT devices by their network activity and dynamically accommodate legitimate changes in assets (either addition of new device profile or upgrade of existing profiles). Our contributions are threefold: 1) we identify key traffic attributes that can be obtained from flow-level network telemetry to characterize the behavior of various IoT device types. We develop an unsupervised one-class clustering method for each device to detect their normal network behavior; 2) we tune device-specific clustering models and use them to classify IoT devices from their network traffic in real time. We enhance our classification by developing methods for automatic conflict resolution and noise filtering; and 3) we evaluate the efficacy of our scheme by applying it to traffic traces (benign and attack) from ten real IoT devices and demonstrate its ability to detect behavioral changes with an overall accuracy of more than 94%.

Rate-Latency Optimization for NB-IoT With Adaptive Resource Unit Configuration in Uplink Transmission

Narrowband Internet of Things (NB-IoT) is a cellular IoT communication technology standardized by 3rd Generation Partnership Project (3GPP) for supporting massive machine type communication and its deployment can be realized by a simple firmware upgrade on existing long term evolution (LTE) networks. The NB-IoT requirements in terms of energy efficiency, achievable rates, latency, extended coverage, make the resource allocation, in a limited bandwidth, even a more challenging problem w.r.t. to legacy LTE. The allocation, done with subcarrier (SC) granularity in NB-IoT, should maintain adequate performance for the devices while keeping the power consumption as low as possible. Nevertheless, the optimal solution of the resource allocation problem is typically unfeasible since nonconvex, NP-hard and combinatorial because of the use of binary variables. In this article, after the formulation of the optimization problem, we study the resource allocation approach for NB-IoT networks aiming to analyze the tradeoff between rate and latency. The proposed suboptimal algorithm allocates radio resource (i.e., SCs) and transmission power to the NB-IoT devices for the uplink transmission and the performance is compared in terms of latency, rate, and power. By comparing the proposed allocation to a conventional round robin (RR) and to a brute-force approach, we can observe the advantages of the formulated allocation problem and the limited loss of the suboptimal solution. The proposed algorithm outperforms the RR by a factor 2 in terms of spectral efficiency and, moreover, the study includes techniques that reduce the dropped packets from 29% to 1.6%.

Managing IoT Cyber-Security Using Programmable Telemetry and Machine Learning

Cyber-security risks for Internet of Things (IoT) devices sourced from a diversity of vendors and deployed in large numbers, are growing rapidly. Therefore, management of these devices is becoming increasingly important to network operators. Existing network monitoring technologies perform traffic analysis using specialized acceleration on network switches, or full inspection of packets in software, which can be complex, expensive, inflexible, and unscalable. In this paper, we use SDN paradigm combined with machine learning to leverage the benefits of programmable flow-based telemetry with flexible data-driven models to manage IoT devices based on their network activity. Our contributions are three-fold: (1) We analyze traffic traces of 17 real consumer IoT devices collected in our lab over a six-month period and identify a set of traffic flows (per-device) whose time-series attributes computed at multiple timescales (from a minute to an hour) characterize the network behavior of various IoT device types, and their operating states (i.e., booting, actively interacted with user, or being idle); (2) We develop a multi-stage architecture of inference models that use flow-level attributes to automatically distinguish IoT devices from non-IoTs, classify individual types of IoT devices, and identify their states during normal operations. We train our models and validate their efficacy using real traffic traces; and (3) We quantify the trade-off between performance and cost of our solution, and demonstrate how our monitoring scheme can be used in operation for detecting behavioral changes (firmware upgrade or cyber attacks).

Rejection of Smooth GPS Time Synchronization Attacks via Sparse Techniques

This paper presents a novel time synchronization attack (TSA) model for the Global Positioning System (GPS) based on clock data behavior changes in a higher-order derivative domain. Further, the time synchronization attack rejection and mitigation based on sparse domain (TSARM-S) is presented. TSAs affect stationary GPS receivers in applications where precise timing is required, such as cellular communications, financial transactions, and monitoring of the electric power grid. In the present work, the clock bias and clock drift are monitored at higher-order clock data derivatives where the TSA is seen as a sparse spike-like event. The smoothness of the attack relates to the derivative order where the sparsity is observed. The proposed method jointly estimates a dynamic solution for GPS timing and rejects behavior changes based on such sparse events. An evaluation procedure is presented for two testbeds, namely a commercial receiver and a software-defined radio. Further, the proposed method is evaluated against distinct real-dataset Texas Spoofing Test Battery (TEXBAT) scenarios. Combined synthetic and real-data results show an average RMS clock bias error of 12.08 m for the SDR platform, and 45.74 m for the commercial device. Further, the technique is evaluated against state-of-the-art mitigation techniques and in a spoofing-plus-multipath scenario for robustness. Finally, TSARM-S can be potentially optimized and implemented in commercial devices via a firmware upgrade.

Philips Intellivue NMT module: precision and performance improvements to meet the clinical requirements of neuromuscular block management.

The variability or inaccuracy of acceleromyographic measurements could interfere with the interpretation of the train-of-four (TOF) ratio during neuromuscular block (NMB) recovery. This study evaluated the precision and performance of the Philips Intellivue NMT module (NMT) before (part 1) and after (part 2) several technical upgrades (i.e., firmware upgrade, new cable, and hand adapter) that were recently available. Two cohorts of 30 patients who were scheduled to undergo rhino/septoplasty under general anesthesia were included in the study. TOF ratios were recorded simultaneously every 15s on both hands with the NMT and a TOF-Watch SX installed inside a SL TOF-Tube (TWX). Before rocuronium was administered and once final responses were stabilized, the average of the four successive measurements that determined the baselines and repeatability coefficients were compared using a z test. Simultaneous measurements were recorded at different NMB stages: onset, depth of NMB after intubation, when TWX recovered TOF count 2, TOF ratios 0.5 and 0.9, and when NMT recovered TOF ratio 0.9. The results were compared using a Student t test; p < 0.05 was considered significant. The NMT repeatability coefficients obtained in part 1 were significantly higher than with the TWX, they were significantly lower in part 2. Initially, the NMT significantly overestimated NMB recovery at every stage. Conversely, in the second part of the study, no difference reached statistical significance. With the recent upgrades and the new hand adapter, the NMT provided similar results compared with the TWX, Their implementation should be recommended in clinical practice.

Trigger and timing distributions using the TTC-PON and GBT bridge connection in ALICE for the LHC Run 3 Upgrade

The ALICE experiment at CERN is preparing for a major upgrade for the third phase of data taking run (Run 3), when the high luminosity phase of the Large Hadron Collider (LHC) starts. The increase in the beam luminosity will result in high interaction rate causing the data acquisition rate to exceed 3 TB/sec. In order to acquire data for all the events and to handle the increased data rate, a transition in the readout electronics architecture from the triggered to the trigger-less acquisition mode is required. In this new architecture, a dedicated electronics block called the Common Readout Unit (CRU) is defined to act as a nodal communication point for detector data aggregation and as a distribution point for timing, trigger and control (TTC) information. The ALICE trigger protocol in the upgraded triggerless readout architecture uses two asynchronous fast serial trigger links (FTLs) connections: the TTC-PON and the GBT. We have carried out a study to evaluate the quality of the embedded timing signals forwarded to the connected electronics using the TTC-PON and GBT bridge connection. We have used four performance metrics to characterize the communication bridge: (a) the latency added by the firmware logic, (b) the jitter cleaning effect of the PLL on the timing signal, (c) BER analysis for quantitative measurement of signal quality, and (d) the effect of optical transceivers parameter settings on the signal strength. Reliability study of the bridge connection in maintaining the phase consistency of timing signals is conducted by performing multiple iterations of power on/off cycle, firmware upgrade and reset assertion/de-assertion cycle (PFR cycle). The Intel® development kit having Arria® 10 FPGA is used for developing the prototype of the firmware. The test results are presented and discussed concerning the performance of the TTC-PON and GBT bridge communication chain and its compliance with the ALICE timing requirements.

A challenge-based countermeasure against the spoofed PS-Poll-based DoS attack in IEEE 802.11 networks

802.11 network is an emerging wireless broadband technology used to connect client stations to the network via an access point (AP). This network is particularly vulnerable to denial of service (DoS) attacks targeting the network availability. In this paper, we focus on the spoofed PS-Poll-based DoS attack in 802.11 power save mode (PSM), where the attacker spoofs the polling frame on behalf of the client in order to destruct data buffered at the AP, that are expected to be delivered to asleep client. To cope with this attack, we rely on the prime factorisation to authenticate PS-Poll frame needed to retrieve buffered data. The countermeasure we propose, called authenticated power save poll (APSP), is both detective and preventive one with low communication, computing and storage overheads. It can be integrated in 802.11 standard through firmware upgrade without any required additional hardware. In order to prove the efficiency of the proposed countermeasure and evaluate its performance, extensive experiments are carried out. Simulation results show that the spoofed PS-Poll-based DoS attack is successfully prevented and detected by APSP compared to the basic 802.11 PSM.

Influencing Factors of GNSS Differential Inter-System Bias and Performance Assessment of Tightly Combined GPS, Galileo, and QZSS Relative Positioning for Short Baseline

This paper first investigates the influencing factors of between-receiver Differential Inter-System Bias (DISB) between overlapping frequencies of the Global Positioning System (GPS), Galileo and the Quasi-Zenith Satellite System (QZSS). It was found that the receiver reboot and the type of observations may have an impact on DISBs. The impact of receiver firmware upgrades and the activation of anti-multipath filters are also investigated and some new results are presented. Then a performance evaluation is presented of tightly combined relative positioning for a short baseline with GPS/Galileo/QZSS L1-E1-L1/L5-E5a-L5 observations with the current constellations, in which the recently launched Galileo and QZSS satellites will also be included. It is demonstrated that when DISBs are a priori calibrated and corrected, the tightly combined model can deliver a much higher empirical ambiguity resolution success rate and positioning accuracy with respect to the classical loosely combined model, especially under environments where the observed satellites for each system are limited and only single-frequency observations are available. The ambiguity dilution of precision, bootstrapping success rate, and ratio values are analysed to illustrate the benefits of the tightly combined model as well.

Effects of the Gill-Solent WindMaster-Pro “w-boost” firmware bug on eddy covariance fluxes and some simple recovery strategies

In late 2015 and early 2016, work done by the AmeriFlux Management Project Technical Team (amerilfux.lbl.gov) helped to uncover an issue with Gill WindMaster and WindMaster Pro sonic anemometers used by many researchers for eddy covariance flux measurements. Gill has addressed this issue and has since sent out a notice that the vertical wind speed component (a critical piece of all eddy covariance fluxes) was being erroneously computed and reported. The problem (known as the “w-boost” bug) resulted in positive (upward) wind speeds being under-reported by 16.6% and negative (downward) wind speeds being under-reported by 28.9%. This has the potential to cause similar under estimates in fluxes derived from measurements using these instruments. Additionally, the bug affects corrections for angle of attack as derived by Nakai and Shimoyama, rendering them invalid. While the manufacturer has offered a firmware upgrade for existing instruments that will fix this issue, many existing data sets have been affected by it and are currently in use by the scientific community.To address the issue of affected data, currently in use, we analyzed multi-year and short-term data sets from a variety of ecosystems to assess methods of correcting existing flux data. We found that simple multiplicative correction factors (∼1.18) may be used to remove most of the “w-boost” bias from fluxes in existing data sets that do not include angle of attack corrections.

Modelling the energy consumption for over-the-air software updates in LPWAN networks: SigFox, LoRa and IEEE 802.15.4g

The recent trend towards the use of low-power wide-area-networks (LPWAN) communication technologies in the Internet of Things such as SigFox, Lora and Weightless gives rise to promising applications in smart grids, smart city, smart logistics, etc. where tens of thousands of sensors in a large area are connected to a single gateway. However, to manage such a sheer number of deployed devices, solutions to provide over-the-air firmware updates are required. This paper analyses the feasibility of over-the-air (partial) software updates for three LPWAN technologies (LoRa, SigFox and IEEE-802.15.4g) and discusses the best suited update method for different scenarios: full system updates, application updates and network stack updates.The results indicate that full firmware upgrades consume a substantial amount of energy, especially for the lowest bit-rate LPWAN technologies such as SigFox which drains a single AA battery with 2% when performing a version update. However, technologies with a similar range (i.e. LoRa SF12) require only 0.12%. The trade-off between range and energy (or bit-rate) becomes clear when considering that the least sensitive technology (IEEE-802.15.4g-OFDM) consumes only 0.0001%. Partial updates require significantly less energy for all technologies. Adding a single application uses 6 to 38 times less energy compared to a firmware update, depending on the update method and LPWAN technology. Even partial network stack updates (i.e. MAC) cost 3 to 8 times less energy, making over-the-air updates feasible.

Advanced simulation of IEDs

IEC 61850 is the established standard for substation communication. Since the standard describes different stages of lifecycle of Substation Automation Systems the demands for specification, testing, and especially simulation differ. The approach described in this study can be used in the different stages and fulfil practical requirements. It describes the requirements for simulation and proposes an implementation applicable. If a single Intelligent Electronic Device (IED) and its data model to be simulated in technical investigation phase is sufficient at Factory Acceptance Testing and Site Acceptance Testing multiple IEDs and utilised services as GOOSE and reports will be needed. In addition, the methods described for indicating test mode/behaviour and simulation indication become important. The simulation has to be embedded in testing environment and interact with tools to be used for visualisation of changes in IED's data model. Advanced services in IEC 61850 like setting groups control, switching device control and file transfer have to be added. The requirements will be discussed from a utility's perspective. The method described can be extended for routine testing and maintenance testing, e.g. after firmware upgrades. An outlook develops a vision to be utilised for automation and control testing.