Optical Circuit Switching Network Research Articles

Optical switching will innovate intra data center networks [Invited Tutorial

Reflecting the recent slow-down in Moore’s law and the proliferation of artificial intelligence/machine learning workloads, the performance and energy consumption of networks are becoming barriers in high-performance computing (HPC) and data centers. Optical switches are expected to break these barriers, and indeed their introduction has recently commenced in data centers. This paper discusses how optical switching technologies can innovate future intra data center networks. Hyperscale data centers are much bigger in scale, and network requirements are slightly different from those of HPC. This paper focus on data center networks, since the impact of optical technologies will be more significant in data centers than in HPC. In addition to the scale issue, important metrics to be considered for network design are traffic characteristics and latency, both of which are highlighted in this paper. For hybrid (electrical packet and optical circuit) switching networks, the target latency for the optical circuit switch network (connection setup/teardown time) is shown to be around 10 µs, and the needed technologies are clarified and verified by experiments. The optical switch can simplify the present multi-tier switching network above tier-1 switches into a single tier configuration, which is possible with the development of efficient large port count optical switches. Among the different switching architectures, combining the different dimensions of space and wavelength is shown to be one of the best solutions. Fast switching needs fast device response time. Si photonics devices using Mach–Zehnder interferometers or ring-resonator-based switches and tunable filters are the most promising candidates; they offer cost-effective mass-production and fast operation and so are excellent candidates for the optical switches envisaged. Another critical technology to maximize the benefits of optical switches is a simple and low-latency control mechanism. Different approaches have been suggested as summarized in this work. Among them, harnessing optical switch parallelism is a unique technique that matches recent advances in electrical switch chips. A fast control network is realized by using a fully decentralized and asynchronous control mechanism. A hyperscale data center offers a wide variety of services, and no one system fits all needs. Optimization of parameters is an important task for maximizing the impact of optical switching in different kinds of data centers.

ADMIRE: collaborative data-driven and model-driven intelligent routing engine for traffic grooming in multi-layer X-Haul networks

X-Haul aims to provide a unified transport network for integrating mobile fronthaul/midhaul/backhaul. Its architecture is inherited from the IP-over-WDM paradigm, in which the upper layer is an electrical packet-switched network and the bottom layer is an optical circuit-switched network. Traffic grooming in X-Haul poses a big challenge, as the mobile traffic load has shown. As a multi-layer routing problem, traffic grooming has been well studied through some classic mathematical models, such as the auxiliary graph (AG). Recently, a new approach based on machine learning has been attracting much attention for its network routing decision making. According to previous studies, either of the two approaches has its limitations, but the combination of the two may achieve profit maximization. To this end, we propose and demonstrate, for the first time, to our knowledge, a collaborative data-driven (using machine learning) and model-driven (using experiential knowledge) intelligent routing engine (ADMIRE) for traffic grooming in an X-Haul testbed with a real mobile dataset. The principle of ADMIRE is to exploit the capability of machine learning for an accurate AG model in a dynamic network environment. We compare ADMIRE with a traditional model-driven approach (i.e., the AG), and the evaluation results show that ADMIRE can achieve good performance and a strong generalization ability. In addition, we also verify the influence of data correlation on ADMIRE.

Shufflecast: An Optical, Data-Rate Agnostic, and Low-Power Multicast Architecture for Next-Generation Compute Clusters

An optical circuit-switched network core has the potential to overcome the inherent challenges of a conventional electrical packet-switched core of today’s compute clusters. As optical circuit switches (OCS) directly handle the photon beams without any optical-electrical-optical (O/E/O) conversion and packet processing, OCS-based network cores have the following desirable properties: a) agnostic to data-rate, b) negligible/zero power consumption, c) no need of transceivers, d) negligible forwarding latency, and e) no need for frequent upgrade. Unfortunately, OCS can only provide point-to-point (unicast) circuits. They do not have built-in support for one-to-many (multicast) communication, yet multicast is fundamental to a plethora of data-intensive applications running on compute clusters nowadays. In this paper, we propose Shufflecast, a novel optical network architecture for next-generation compute clusters that can support high-performance multicast satisfying all the properties of an OCS-based network core. Shufflecast leverages small fanout, inexpensive, passive optical splitters to connect the Top-of-rack (ToR) switch ports, ensuring data-rate agnostic, low-power, physical-layer multicast. We thoroughly analyze Shufflecast’s highly scalable data plane, light-weight control plane, and graceful failure handling. Further, we implement a complete prototype of Shufflecast in our testbed and extensively evaluate the network. Shufflecast is more power-efficient than the state-of-the-art multicast mechanisms. Also, Shufflecast is more cost-efficient than a conventional packet-switched network. By adding Shufflecast alongside an OCS-based unicast network, an all-optical network core with the aforementioned desirable properties supporting both unicast and multicast can be realized.

Characterization of Burst-Mode Links for Optical Circuit Switching

The performance of an optically circuit-switched network depends on the transient locking characteristics of the transceiver as a function of the received power before and after a switching event. These characteristics are not specified in current transceiver standards. Here, we present a novel characterization method to assess the transient locking characteristics of transceivers when used in an optical circuit-switched network. The method uses custom hardware code implemented on a high-performance FPGA. This code samples the bit error rate in a manner similar to a sampling oscilloscope. The output is a time-resolved bit error rate measurement on a microsecond time scale. We then use this technique to quantify the transient locking characteristics of commercial datacom transceivers as a function of the nominal received power, the power offset between the channels being switched, and the transition time of the optical switch. These results enable research into optical circuit-switched networks based on commercial datacom transceivers.

A novel bandwidth allocation scheme for OTSS-enabled flex-grid intra-datacenter networks

Optical circuit switching networks have been recognized as a promising solution for inter-datacenter networks. However, for intra-datacenter networks, they may fall short in efficiently provisioning traffic requests due to their relatively coarse-grained channel assignment and special intra-datacenter traffic patterns. Optical time slice switching (OTSS) has been recently proposed as an optical-switching technique that can provide flexible and transparent optical circuits by extending the merit of flex-grid switching to the time domain, thus achieving much finer granularity. As OTSS requires nanosecond speed optical switches which are expensive, it might not be economically viable to make a one-time upgrade for the entire datacenter. Thus, we expect fine-grained OTSS-enabled and coarse-grained flex-grid-enabled optical switching techniques to co-exist in the foreseeable future. In this study, we investigate an OTSS-enabled flex-grid (OTSS-FG) architecture for intra-datacenter networks. For scenarios where traffic flows are given, we develop a Mixed Integer Linear Program to study the optimal bandwidth allocation scheme in an OTSS-FG architecture. When traffic flows are generated in real time, by leveraging machine-learning techniques to detect flow types, we propose a flow-aware bandwidth allocation (FABA) scheme and a dynamic version of FABA, called “D-FABA” scheme. Numerical simulations show that proposed bandwidth allocation scheme can outperform benchmark schemes in terms of average delay and blocking probability.

Online parameter tuning of the flow classification method in the energy-efficient data center network “HOLST”

An online parameter-tuning method for the energy-efficient data center network (DCN) named HOLST (high-speed optical layer 1 switch system for time-slot-switching-based optical data center networks) is proposed. HOLST comprises an optical circuit switching network, optical slot switching network, and electrical packet switching network for the spine layer. It requires flows to be assigned to optimal switches to achieve high power savings. In this study, first we experimentally confirm that the change in the DCN characteristics in the short term of actual data center traffic downgrades the accuracy of flow classification. Subsequently, we propose a procedure for reconfiguring a flow classification function and a method for online parameter tuning for this function. Finally, the accuracy of the flow classification method using the proposed tuning and estimated switching energy consumption in the spine layer are evaluated. The proposed online parameter-tuning function increases the accuracy of flow classification and reduces switching energy consumption relative to the conventional flow classification function.

MCF-SMF Hybrid Low-Latency Circuit-Switched Optical Network for Disaggregated Data Centers

This paper proposes and experimentally evaluates a fully developed novel architecture with purpose built low latency communication protocols for next generation disaggregated data centers (DDCs). In order to accommodate for capacity and latency needs of disaggregated IT elements (i.e., CPU, memory), this architecture makes use of a low latency and high-capacity circuit-switched optical network for interconnecting various end points that are equipped with multi-channel silicon photonic based integrated transceivers. In a move to further decrease the perceived latency between various disaggregated IT elements, this paper proposes a novel network topology that cuts down the latency over the optical network by 34% while enhancing system scalability and channel bonding over multi-core fiber (MCF) switched links to reduce head to tail latency and in turn increase sustained memory bandwidth for disaggregated remote memory. Furthermore, to reduce power consumption and enhance space efficiency, the integration of novel MCF-based transceivers, fibers, and optical switches are proposed and experimentally validated at the physical layer for this topology. It is shown that the integration of MCF-based subsystems in this topology can bring about an improvement in energy efficiency of the optical switching layer, which is above 60%. Finally, the performance of this proposed architecture and topology is evaluated experimentally at the application layer where the perceived memory throughput for accessing remote and local resources is measured and compared using electrical circuit and packet switching. The results also highlight a multi-fold increase in application perceived memory throughput over the proposed DDC topology by utilization and bonding of multiple optical channels to interconnect disaggregated IT elements that can be carried over MCF links.

Enhancement of Extreme Learning Machine for Estimating Blocking Probability of OCS Networks With Fixed-Alternate Routing

In previous work, we proposed a neural network approach to estimate the blocking probability of optical networks with fixed routing. The neural network was implemented by the extreme learning machine (ELM) framework, in which the training inputs were the optical network parameters, and the output was the overall blocking probability. The numerical results showed that the neural-network-based estimation was accurate and thousands of times faster than a computer simulation. In this paper, we apply the neural network approach to optical circuit switching (OCS) networks with fixed-alternate routing and improve the training method by using an enhancement of ELM framework. Unlike the previous ELM framework, the enhancement of ELM framework provides a random-search-based selection phase for the hidden nodes during the training step. As a result, similar performance can be achieved using fewer hidden nodes than the previous ELM framework. The numerical results show that the new enhancement of ELM training algorithm provides more accurate blocking probability estimates while reducing the required number of hidden nodes by a third compared with the previous ELM training algorithm. Furthermore, for some light traffic loading situations, our new training algorithm is hundreds of times more accurate than the existing well-known analytical approximation method.

Scalable Path-Setup Scheme for All-Optical Dynamic Circuit Switched NoCs in Cache Coherent CMPs

Nanophotonics is a promising solution for on-chip interconnection due to its intrinsic low-latency and low-power features, which can be useful for performance and energy in future Chip Multi-Processors (CMPs). This article proposes a novel arbitrated all-optical path-setup scheme for tiled CMPs adopting circuit-switched optical networks. It aims at significantly reducing path-setup latency and overall energy consumption. The proposed arbitrated scheme is able to configure multiple photonic switches simultaneously, instead of sequentially as it is done in state-of-the-art proposals. The proposed fast optical path-setup solution reduces the overhead in each transmission and, most importantly, allows optical circuit-switched networks to effectively serve cache coherence traffic, which is mainly composed of relatively small messages. Specifically, we propose a single-arbiter scheme where the whole topology is managed by a central module (single-arbiter) that takes care of the path-setup procedures. Then, to tackle scalability, we propose a logically clustered architecture (multi-arbiter) in which an arbiter is allocated in each logical core-cluster and an ad hoc distributed reservation protocol coordinates arbiters to manage inter-cluster path reservations. We show that our proposed single-arbiter architecture outperforms a state-of-the-art optical network with sequential path-setup (optical baseline) in the case of 8- and 16-core tiled CMP setups. However, due to serialization issues, the single-arbiter solution is not able to compete with a reference electronic baseline for bigger 32- and 64-core setups even if still performing much better than the optical baseline. Conversely, our multi-arbiter hierarchical solution allows us to improve performance up to almost 20% and 40% for 32- and 64-core setups, respectively, demonstrating a wide applicability of the proposed technique. Energy-wise, the analyzed solutions enable significant savings compared to both the optical baseline with sequential path setup, and to the electronic counterpart. Specifically, results show more than 25% average improvement for the single-arbiter in the 8- and 16-core cases, and more than 40% and 15% savings for the multi-arbiter in the 32- and 64-core cases, respectively.

Transmission of burst bulk data in optical circuit-switched networks with assistive storage

Transmission of burst bulk data in optical circuit-switched networks with assistive storage

Scheduling traffic for maximum switch lifetime in optical data center fabrics

Optical switching is a promising technology to scale the performance of data centers. We address two scenarios: hybrid data centers, in which an optical circuit switching network operates in parallel to an electronic packet switching network that interconnect the servers, and full optical data centers, in which all the traffic is switched in the optical domain. The traffic on the optical network is scheduled proactively by configuring the optical fabric according to a precomputed sequence of configurations. We consider that fast optical circuit switching is implemented through micro-mechanical devices (like MEMS) at each crosspoint, whose lifetime is limited by a maximum number of configuration cycles, due to the experienced mechanical fatigue. We consider the traffic scheduling problem on the optical network and address specifically the problem of minimizing the number of reconfigurations at each crosspoint, in order to maximize the MEMS lifetime, while maximizing the throughput. We propose a family of scheduling algorithms and discuss the achievable tradeoff between lifetime, throughput and computational complexity, throughout a set of extensive simulations and theoretical results.

Time-Shifted Multilayer Graph: A Routing Framework for Bulk Data Transfer in Optical Circuit-Switched Networks With Assistive Storage

Increasing bulk data transfers incur peak-hour bandwidth contention between bulky and short flows as well as between bulky flows. To mitigate this contention, storage is introduced into the forwarding path so that bulk data that are delay tolerant can be temporarily stored and forwarded at a later time. However, the storage introduces an additional complexity into the conventional routing process. What was a spatial resource allocation problem becomes a scheduling problem, in which both bandwidth and storage constraints must be considered and both spatial assignments and temporal arrangements must be performed. In this paper, we propose a routing framework for bulk data transfer in optical circuit-switched networks with assistive storage. This framework is based on a multilayer graph built from a set of snapshots (i.e., layers) of the dynamics in a network. By performing shortest path routing on the multilayer graph, end-to-end paths over time and space are found for requests, thus greatly simplifying the provisioning process. We study how the number of layers used for routing affects the network blocking performance and how traffic characteristics and link capacity affect the number of layers. We find that the majority of requests can be served with only one additional layer. A trade-off between computational complexity and blocking performance can be reached by limiting the number of layers used for routing. In our simulations, the request blocking probability is reduced from 13.5% to 0.9% by limiting routing to 11 layers. Request blocking is avoided when routing is allowed within 29 layers.

Routing and spectrum allocation in multi-ring based data center networks

Recently, we proposed a multi-ring based optical circuit switching (OCS) network following the principle of a Clos network. The network can provide connectivity to a large number of racks which may be distributed across a relatively large geographical space in a data center. However, property of the ring based switch in the central stage of the multi-ring based OCS network introduces a unique routing and spectrum allocation (RSA) problem which is more complex than the routing problem in a classical Clos switching network. In this paper, we extend our work to investigate the RSA problem. For a given set of inter-rack traffic requests, we consider two spectrum allocation schemes, namely fixed spectrum allocation and flexible spectrum allocation. For the fixed case, we show that the RSA problem degenerates into the traditional routing problem of the Clos network. As for the flexible case, property of spectrum division multiplexing technology and bandwidth limitation of the ring based switches should be taken into consideration during allocation of the central module, such that the system throughput can be maximized. This paper presents an integer linear program (ILP) formulation as well as a heuristic algorithm we developed to solve the flexible RSA problem. We evaluate the performance of both the two spectrum allocation schemes under different traffic patterns. Our results demonstrate that, to handle uneven inter-rack traffic pattern in general data center networks, flexible spectrum allocation can lead to an increase of about 120% in system throughput, although its computational complexity is slightly higher than that of the fixed spectrum allocation scheme.

Optical multicast system for data center networks.

We present the design and experimental evaluation of an Optical Multicast System for Data Center Networks, a hardware-software system architecture that uniquely integrates passive optical splitters in a hybrid network architecture for faster and simpler delivery of multicast traffic flows. An application-driven control plane manages the integrated optical and electronic switched traffic routing in the data plane layer. The control plane includes a resource allocation algorithm to optimally assign optical splitters to the flows. The hardware architecture is built on a hybrid network with both Electronic Packet Switching (EPS) and Optical Circuit Switching (OCS) networks to aggregate Top-of-Rack switches. The OCS is also the connectivity substrate of splitters to the optical network. The optical multicast system implementation requires only commodity optical components. We built a prototype and developed a simulation environment to evaluate the performance of the system for bulk multicasting. Experimental and numerical results show simultaneous delivery of multicast flows to all receivers with steady throughput. Compared to IP multicast that is the electronic counterpart, optical multicast performs with less protocol complexity and reduced energy consumption. Compared to peer-to-peer multicast methods, it achieves at minimum an order of magnitude higher throughput for flows under 250 MB with significantly less connection overheads. Furthermore, for delivering 20 TB of data containing only 15% multicast flows, it reduces the total delivery energy consumption by 50% and improves latency by 55% compared to a data center with a sole non-blocking EPS network.

An optical circuit switching network architecture and reconfiguration schemes for datacenter

The paper presents an optical circuit switching network architecture and its control and traffic scheduling schemes. This optical switch network is built on a 32×32 AWG and fast wavelength tunable transmitters and is able to support up to 128 interface ports. The feasibility of the architecture is verified through an experimental study. The traffic scheduling scheme takes into account the transmitter wavelength tuning time so as to avoid frequent connection reconfiguration. The simulation results show that fast connection reconfiguration speed and efficient traffic scheduling scheme enable optical circuit switching network to deliver high transmission bandwidth and low packet delay performance in datacenter network.

Deployment of the Content-Based Switching Network

In this paper, we propose a hybrid network architecture, called Content-based Switching Network (CSN), and its signaling scheme, which addresses the issues inherent to conventional hybrid networks which implement a horizontal separation over the entire network (from edge to edge). We will show how CSN nodes can flexibly choose their switching paradigm (store-and-forward, optical bypass, electrical bypass) during a path establishment. Contents being transferred in one piece from end-to-end, the concept of packet can be eluded in our network, and, in particular, the user is able to avoid complicated transport layer functions, like TCP, if they are not essential. In CSN, very large contents have a special status, since they cannot be store-and-forwarded. We will show how the resource management has been designed in order to deal with such contents. A section is dedicated to deployment and feasibility issues. Simulation results will show that CSN can successfully transfer contents at 1 Gbps and 10 Gbps, the maximum speed being limited by the state-of-the-art device technologies when buffering is required (memory speed), while no major limit is observed in the case of all-optical transfers other than the optical fiber speed. Other results concern the deployment of CSN from an unclean slate approach. They will show how beneficial can be the deployment of CSN from an Optical Circuit Switching network.

An Energy-Efficient Optically Connected Memory Module for Hybrid Packet- and Circuit-Switched Optical Networks

Demands on future data centers and high-performance computing systems will require processor-memory interconnects with greater performance and flexibility than can be provided by existing electronic interconnects. The bandwidth density and bit-rate transparency offered by optical systems are uniquely suited to address these challenges facing memory interconnects. We, thus, investigate a hybrid packet- and circuit-switched optical interconnection network linking microprocessors with their associated main memory, which can simultaneously reduce memory access latency and improve energy efficiency performance. This novel hybrid approach allows low-bandwidth memory control data and small memory transactions to be efficiently transmitted as wavelength-striped optical packets, while long bursts of memory accesses are optically circuit switched. In this study, we experimentally demonstrate an optically connected memory system in which a microprocessor accesses multiple 80-Gb/s memory modules all-optically across a hybrid packet- and circuit- switched optical network. Error-free communication between the microprocessor and main memory is confirmed (bit-error rates less than 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-12</sup> ) with the optical network providing low-memory access latencies. The overall memory system reduces energy consumption by 28%.

A distributed framework for energy-efficient lightpaths in computational grids

Over the past decade, the ever-increasing energy demands of IT infrastructures have posed significant challenges for the research community in terms of reducing their total power consumption and minimizing their environmental impact. Optical communication networks are envisioned to be promising candidates to help preventing this problem affecting the Internet backbone, as well as for distributed applications such as computational grids. In this paper, we propose an adaptive and distributed scheme for the establishment of energy-efficient lightpaths in computational grids. The grid is deployed over an optical circuit-switched backbone network, defining an optical grid network. Each node of the backbone network maintains two different dynamic thresholds values and estimates the changes in network performance by evaluating the moving average of the total wavelength channel occupancy on all its input/output links. The nodes have the ability of reducing the energy consumption by entering into an Energy Saving Mode ESM on the basis of a comparison between their channel occupancy and the thresholds. Furthermore, we extend our framework by allowing the thresholds to be dynamically adapted depending on the network performance in terms of blocking probability. We show that the proposed method achieves considerable energy savings when compared to a normal energy-unaware operational mode and still allows to maintain an acceptable level of network performance in terms of blocking probability and end-to-end delay. Numerical results are obtained with a Java event-driven simulator of two different optical network topologies.

A dynamic and quasi-centralized RWA method for optical fast circuit switching networks employing route pre-prioritization

We propose a new dynamic RWA algorithm for optical fast circuit switching networks, agile optical networks that can accommodate changing broadband demands. The algorithm utilizes pre-computed prioritized route candidates that reflect statistical information such as traffic distribution characteristics to attain better load balancing. The route priority is periodically distributed over a network and RWA for each path connection demand is processed in a distributed manner. Numerical experiments demonstrate that the algorithm matches the performance of a centralized RWA algorithm that uses all the necessary information on a network.

Providing resiliency for optical grids by exploiting relocation: A dimensioning study based on ILP

Grids use a form of distributed computing to tackle complex computational and data processing problems scientists are presented with today. When designing an (optical) network supporting grids, it is essential that it can overcome single network failures, for which several protection schemes have been devised in the past. In this work, we extend the existing Shared Path protection scheme by incorporating the anycast principle typical of grids: a user typically does not care on what specific server this job gets executed and is merely interested in its timely delivery of results. Therefore, in contrast with Classical Shared Path protection (CSP), we will not necessarily provide a backup path between the source and the original destination. Instead, we allow to relocate the job to another server location if we can thus provide a backup path which comprises less wavelengths than the one CSP would suggest. We assess the bandwidth savings enabled by relocation in a quantitative dimensioning case study on an European and an American network topology, exhibiting substantial savings of the number of required wavelengths (in the order of 11–50%, depending on network topology and server locations). We also investigate how relocation affects the computational load on the execution servers. The case study is based on solving a grid network dimensioning problem: we present Integer Linear Programming (ILP) formulations for both the traditional CSP and the new resilience scheme exploiting relocation (SPR). We also outline a strategy to deal with the anycast principle: assuming we are given just the origins and intensity of job arrivals, we derive a static (source, destination)-based demand matrix. The latter is then used as input to solve the network dimensioning ILP for an optical circuit-switched WDM network.