Juniper Express 5 28.8Tbps Network Routing ASIC at HC34

1
HC34 Juniper Express 5 ASIC 2 Line Card PFD With Fabric Interconnect
HC34 Juniper Express 5 ASIC 2 Line Card PFD With Fabric Interconnect

At Hot Chips 34, we got an insight into the new Juniper Express 5. This is a cool modular chiplet approach to building an ASIC for routing and switching devices. With two chiplets, the company can make a number of ASICs for its portfolio.

Note: We are doing this piece live at HC34 during the presentation so please excuse typos.

Juniper Express 5 28.8Tbps Network Routing ASIC at HC34

In the old days, there were routers or switches. There was a question if routers would still exist or if we would just have smart switches. These days, routers can switch and switches can route.

HC34 Juniper Express 5 Routers V Switches
HC34 Juniper Express 5 Routers V Switches

Juniper has a number of ASICs that it makes for routing. The new one here is the Express ASIC for Juniper’s PTX line.

HC34 Juniper Express 5 Juniper Routing ASICs
HC34 Juniper Express 5 Juniper Routing ASICs

There are two main ways that people build big networks. Either 1U-2U boxes and scaling out or scaling up using chassis switches.

HC34 Juniper Express 5 Goals Scale Out And Scale Up
HC34 Juniper Express 5 Goals Scale Out And Scale Up

On the scale-out side, we normally get 32-port Radix for smaller networks but 36-ports is for networks with deeper buffers. 36 pluggable cages is what fits in a standard 1U rack so we get a 28.8Tbps switch as 36x 800Gbps.

HC34 Juniper Express 5 28.8Tbps
HC34 Juniper Express 5 28.8Tbps

Juniper also discussed the scale-up modular chassis.

HC34 Juniper Express 5 Scale Up Modular Chassis
HC34 Juniper Express 5 Scale Up Modular Chassis

When one looks at the scale up and scale out, they can look similar in some ways, but in practice, they are not.

HC34 Juniper Express 5 Pitfalls Of Naive Packet Based Fabric Design
HC34 Juniper Express 5 Pitfalls Of Naive Packet-Based Fabric Design

Juniper Express 5 has a cell-based fabric.

HC34 Juniper Express 5 Dell Based Fabric Design
HC34 Juniper Express 5 Cell Based Fabric Design

Making its new chips is the X-chiplet. This has networking interfaces, packet forwarding, packet buffer (on-die), and XSR interfaces. It is made on TSMC 7nm. Size-wise, this is similar to a modern GPU in terms of TSMC 7nm transistor count.

HC34 Juniper Express 5 X Chiplet
HC34 Juniper Express 5 X Chiplet

The F-chiplet is the fabric and switching chiplet.

HC34 Juniper Express 5 F Chiplet
HC34 Juniper Express 5 F Chiplet

Between the chips there are 112G XSR PAM4 SerDes. One reason for this is that Juniper is looking to the co-packaged optics future.

HC34 Juniper Express 5 Why 112G XSR PAM4
HC34 Juniper Express 5 Why 112G XSR PAM4

32Tbps is RX+TX bandwidth in the below slide. This is a giant chip with two X-chiplets and multiple HBM stacks. The HBM2e gives massive capacity and a lot of memory bandwidth to help the device perform routing functions. Note here that there are no F-chiplets because this is the routing device.

HC34 Juniper Express 5 ASIC 1 28.8T Network Routing Device
HC34 Juniper Express 5 ASIC 1 28.8T Network Routing Device

Here are the features of the big ASIC.

HC34 Juniper Express 5 ASIC 1 Features
HC34 Juniper Express 5 ASIC 1 Features

Here are the packet forwarding pipelines.

HC34 Juniper Express 5 Packet Forwarding
HC34 Juniper Express 5 Packet Forwarding

That big ASIC is not the only one. It can also have a X and a F chiplet configuration called ASIC 2. This also has HBM2e.

HC34 Juniper Express 5 ASIC 2 Line Card PFD With Fabric Interconnect
HC34 Juniper Express 5 ASIC 2 Line Card PFD With Fabric Interconnect

Removing HBM and the X-chips there are ASICs for different size chips based only on one or two F-chiplets.

HC34 Juniper Express 5 ASIC 3 And ASIC 4
HC34 Juniper Express 5 ASIC 3 And ASIC 4

ASIC 5 is a X-chiplet with HBM2e and no F-chiplets.

HC34 Juniper Express 5 ASIC 5
HC34 Juniper Express 5 ASIC 5

ASICs 6 and 7 are X-chiplets without HBM in either 2 or 1 X-chiplet configurations.

HC34 Juniper Express 5 ASIC 6 And ASIC 7
HC34 Juniper Express 5 ASIC 6 And ASIC 7

Here is the co-packaged optics (CPO) being attached using XSR.

HC34 Juniper Express 5 Co Packaged Optics
HC34 Juniper Express 5 Co Packaged Optics

In the end, Juniper gets seven different types of ASICs with only two chiplets and an eighth with the CPO option.

HC34 Juniper Express 5 2 Chiplets Many ASICs
HC34 Juniper Express 5 2 Chiplets Many ASICs

This leads to building a whole portfolio of scale-out and scale-up solutions with just the two chiplets.

Final Words

This is a really interesting design as it is flexible and is different from what many in the industry are doing. Juniper offers some support for things like SONiC and P4 across its portfolio but this is largely going to be innovation for Juniper hardware and software customers unless Juniper decides to put more emphasis on SONiC / P4. During the conference, someone from ByteDance (known widely for TikTok) asked about SONiC support so hopefully, that helps push towards more open networking. Then again, with Juniper, you are getting the hardware, software, and support so its model makes sense.

1 COMMENT

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.