Advertisement


Home AI Inside the Secretive AWS Networking Lab

Inside the Secretive AWS Networking Lab

0

DWDM puts 64 wavelengths on one fiber

AWS builds its own dense wave-division multiplexing modules rather than buying long-haul optics off the shelf. Two generations were on the bench, one at 400G per lane and one at 800G.

STH AWS Networking Lab Tour - DWDM long-haul optical modules on the bench
STH AWS Networking Lab Tour – DWDM long-haul optical modules on the bench

The client side behaves like ordinary DR4 optics talking to switches in the same data center. When a link needs to cover hundreds or thousands of kilometers, the module converts the signal to long-range coherent optics using 16-QAM modulation.

STH AWS Networking Lab Tour - Pluggable optical module detail
STH AWS Networking Lab Tour – Pluggable optical module detail

The newest generation puts up to 64 wavelengths on a single fiber strand at 800G per wavelength. Fiber between data centers is limited and expensive, so the hardware aims to multiply capacity per strand. Across its facilities, AWS runs optical modules from low-end 10G parts through 1.6T and coherent ZR+.

Solely for educational purposes, if you want to learn about ZR+ optics and QAM constellations, we did a piece in Marvell’s lab on their 800G ZR+ optics.

Hollow-core fiber

Some AWS longer-haul routes now use hollow-core fiber, where light travels through an air core instead of glass. The fiber has the same 125 micrometer outer diameter as standard single-mode cable.

STH AWS Networking Lab Tour - Hollow core fiber model with air center channel
STH AWS Networking Lab Tour – Hollow core fiber model with air center channel

A magnified model on the bench showed the open central channel surrounded by fine tube structures that control reflections and guide the light. In the video, we showed how this made the splice points easy to spot, with a sharp reflectivity dip where the hollow-core span begins.

STH AWS Networking Lab Tour - VIAVI tester trace of single-mode and hollow-core fiber
STH AWS Networking Lab Tour – VIAVI tester trace of single-mode and hollow-core fiber

The reason to run it is latency. Light in air arrives roughly 33 percent faster than light in glass, and availability zones operate under tight inter-zone latency budgets. Lower latency on long routes lets AWS place data centers farther apart while staying inside those budgets. Hollow-core is not universal, and single-mode fiber remains standard inside and near data centers. AWS has deployed hollow-core on longer routes since 2024.

64 fibers in one second

The hardware leg of the tour ended at the patch panel. AWS showed MMC connectors, which gang four connectors into a single housing carrying 64 fibers. Testing these links used to mean loopback plugs with dozens of tiny fiber loops that were expensive and tedious to manufacture.

STH AWS Networking Lab Tour - Ganged MMC connector with 64 fibers and Firefly loopback cap
STH AWS Networking Lab Tour – Ganged MMC connector with 64 fibers and Firefly loopback cap

The replacement is a molded part the team calls the Firefly device. It performs a lens-based loopback and doubles as a protective cap during shipping, keeping dust and physical damage away from the connectors. Pinch it off, latch the ganged connector into the panel, and install 64 fibers in about one second.

STH AWS Networking Lab Tour - Fiber plant in the AWS lab racks
STH AWS Networking Lab Tour – Fiber plant in the AWS lab racks

Holding an equivalent LC patch field next to it makes the density difference obvious. LC needs several times the panel space for the same fiber count. Serviceability survives the gang as well, since you can remove each of the four MMC connectors inside the housing individually for maintenance.

A larger goal is deployment speed. If the fiber plant is installed and tested before servers arrive, expensive AI systems and standard compute do not sit idle waiting on cabling.

The thermal chamber

The final stop was a walk-in environmental chamber behind a door marked Chamber 2 Lab, with hearing-protection signage because the chamber is loud while running. AWS takes components from different vendors and manufacturing sites, assembles them end to end inside the chamber, then runs temperature, humidity, thermal cycling, and humidity cycling profiles.

STH AWS Networking Lab Tour - Chamber 2 Lab walk-in thermal chamber entrance
STH AWS Networking Lab Tour – Chamber 2 Lab walk-in thermal chamber entrance

The test scenarios are deliberately harsher than anything expected in the field, so rare environmental events are covered before hardware ships into production.

Final Words

This was an awesome look inside AWS, seeing the kinds of innovations happening not just at traditional networking hardware companies, but inside a hyperscaler. We do an article for every video, but this is probably one, like touring a 100K GPU cluster, where the video will be more interesting for folks.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.