Resurgence of IP over DWDM with 400G QSFP-DD Coherent Optics
by Andrew Lui
First introduced over 20 years ago, IP over DWDM was an exciting topic as DWDM technology began to take off. Talk of IP over DWDM simmered down as several technological and economical headwinds during that time made adoption difficult. However, with the booming demand for IP networks, IP over DWDM may be up for a resurgence in popularity. This time, technology is also at a sweet spot with the advancement of routing equipment and development with coherent optics technology, where it will be easier for service providers to adopt.
A Review of DWDM Technology
Before going into IP over DWDM, I want to give a quick overview of Dense Wavelength Division Multiplexing (DWDM). DWDM is a technology used in fiber optic communications to increase bandwidth by combining multiple wavelengths of light onto a single fiber. I’ve explained this in detail including DWDM’s benefits and applications in a previous article.
DWDM is considered as one of the most popular technologies used in optical transport networks for long distance transmissions. Commonly used in data center connects, it was further developed with the introduction of ROADMs to make it more suitable for telecommunication service providers.
What is IP over DWDM?
IP over DWDM, or IPoDWDM, refers to the integration of IP routers into the optical transport network by plugging the DWDM optics directly into the router, which eliminates the need for transponder equipment. Network operators can increase the capacity of their networks more efficiently while cutting down costs of purchasing transponders. This was the concept; however, it was easier said than done since IPoDWDM did not really take off when it was first proposed. In order to make this concept into reality, several things had to happen.
Traditional DWDM vs IPoDWDM
The challenges faced for widespread IP over DWDM adoption included getting the right form factor, interoperability, and the ethernet routers capability to efficiently implement IP over DWDM architectures. These challenges were met over 20 years ago. Technology has certainly improved by leaps and bounds compared to then and I believe we are at a point now where IP over DWDM will get a second wind for service provider adoption.
IP over DWDM Form Factors
Traditional DWDM optical engines were built onto sleds or modules and equipped into full fledged optical transport platforms several rack units high. These DWDM optics modules were about the size of a paper back book and many times larger than client optics. One of the reasons that such large DWDM optics was needed was to provide adequate room and cooling for the digital signal processor (DSP), and other embedded optical elements.
This meant if routers were equipped with these oversized DWDM optics, they would be sacrificing a lot of capacity, leading to a term called “faceplate tradeoff.” This was unacceptable for most telecom service providers, since this was a step backwards from their goal of maximizing efficiency and scalability.
DWDM optics vs QSFP-DD optics
This brings us to today, where the advancement in silicon technology has led to smaller and more powerful chips. This has enabled transceiver manufacturers such as Infinera, Coherent and Molex to miniaturize the DSP and the transmit-receive optical sub-assemblies (TROSA) and put them into smaller form factors.
This resulted in the development of the Optical Internetworking Forum (OIF) 400ZR pluggable optics, which would fit DWDM optics into 400G form factors (QSFP-DD or OSFP transceivers) to eliminate faceplate tradeoff and sparked a renewed interest in IP over DWDM. This paved the way for the Open ZR+ MSA, which developed a standard for 400G transceivers more suitable for telecom service providers by combining 400ZR with technology from the Open ROADM MSA.
IP over DWDM Interoperability
Traditional DWDM systems were proprietary systems where all components were provided by a single vendor. It is no surprise that past DWDM module designs also had limited standardization of its management interfaces. There are three major areas that need to align when it comes to interoperability for IP over DWDM systems. These are standardized signals, standardized form factors and standardized management.
Standardized signals would allow for data to be easily transferred using transceivers from different vendors while standardized form factors would enable interoperability between transceivers and routers. These are being standardized by organizations such as the OpenZR+ MSA. Having a standardized management interface would allow for widespread adoption across all manufacturers, providing more choices and better prices for modules that implement IP over DWDM.
The standardization of the management interface was accomplished with the Common Management Interface Specification, or CMIS, for short. Provided by the QSFP-DD, OSFP and COBO advisory group, and later with extensions added on by OIF, the CMIS has been one of the key enablers of coherent module interoperability.
Traditional optics vs Standardized optics
With the interoperability and faceplate tradeoff issues solved, the next hurdle to jump would be the routers.
Routers Supporting IP over DWDM
Most router equipment was not designed for IPoDWDM in mind and cannot compensate for the added power and heat generated from IP over DWDM transceivers. This meant even though there were IPoDWDM transceivers, there was not much they could plug into. Many legacy routing equipment manufacturers also offered transponders and DWDM systems, so there was not a lot of motivation for them to develop routers that supported IPoDWDM. Rather, bespoke router coherent cards that would take up whole chassis slots were provided. Service providers were therefore left with only half of the puzzle pieces.
The other half of the puzzle was found when the industry adopted open and disaggregated architectures. Disaggregation separated different components of a traditionally proprietary system into separate parts. Open networking enabled standardized APIs for each component to function as a system, no matter who provided it. This modulated and multi-vendor approach to building routing systems is driving the innovation for IP over DWDM solutions.
Embracing disaggregation and open networking, UfiSpace has introduced several white box routers, which have the 400G QSFP-DD ports that can support coherent 400ZR/ZR+ transceivers. Instead of monolithic routing chassis, we’ve built pizza-box routers that can be interconnected to build routing clusters similar to the ones used by data centers. Service providers will not need to purchase “chassis slots” and instead just plug in the 400ZR/ZR+ transceiver into any of the QSFP-DD ports on the router. For example, UfiSpace’s latest aggregation router, the S9610-36D is a 400G router offers 36 QSFP-DD ports for high powered 400ZR/ZR+ transceivers.
Support for IP over DWDM does not mean just having QSFP-DD interfaces available. It comes down to power and heat dissipation. 400ZR/ZR+ transceivers can push data over very far distances and that requires quite a bit of power. On average 400ZR transceivers require 18W to 20W of power while 400ZR+ transceivers need up to 24W of power. This is double the amount of power compared to 400G-LR4 transceivers. The additional power will inevitably lead to more heat.
In order to support IP over DWDM, routers must be designed with the right power and heat dissipation specs to properly run high power transceivers. This becomes even more apparent when additional features are introduced such as Infinera’s ICE-XR 400G, which includes integrated management features such as in-band remote management and streaming telemetry. The optimized power and cooling design of UfiSpace’s routers have been a major step towards IPoDWDM, even setting a world record of 2400km transmission using Infinera’s ICE-X QSFP-DD coherent pluggable hosted on UfiSpace’s S9700-23D and Corning’s TXF optical fiber.
IPoDWDM Adoption by Service Providers
IP over DWDM has had a resurgence of adoption from service providers and I expect more will be adopted as many more service providers are already conducting proof of concepts. Recent public adoptions of IP over DWDM includes big names like BT, Deutsche Telekom, and Arelion. Colt is one of the recent adopters of IPoDWDM who is more open with their topology and use case. They currently use OpenZR+ transceivers integrated into routers for connections of 450km or less, but will continue to use line systems for longer connections.
UfiSpace continues to collaborate with world renowned 400ZR/ZR+ transceiver vendors, our NOS partners as well as service providers to build solutions that will enable IPoDWDM with our white box routers. For more information about which vendors we have validated, please contact our sales team.