Microsoft announced Project Corsica this week as the culmination of work around its Zipline compression standard. During the Microsoft OCP 2019 Keynote on Denali and Project Zipline, the company showed that its Zipline compression standard is extremely efficient. It also mentioned that it was working with the industry to get not just the software compression side more utilized, but also opening the Verilog RTL implementation for hardware offload of the work.
Microsoft Project Zipline
Microsoft’s rationale for Project Zipline is easy to understand. Zipline provides compression and encryption to both make file sizes smaller as well as keep data at rest encrypted.
For a service like Microsoft Azure, reducing stored data by several percent means millions in cost savings. Likewise, freeing up CPUs from doing that compression and encryption work can have an enormous impact on operating costs. Since OCP Summit 2019, the company has been pushing hard to gain traction from the hardware community. Now, we have a Zipline ASIC.
Project Corsica a New Zipline ASIC
A joint project between Broadcom and Microsoft, Project Corsica takes the project’s logic and turns it into an ASIC capable of handling 100Gbps of encryption/ compression.
The ASIC is found on a relatively simple PCIe x16 card (required at Gen3 speeds for 100Gbps) showing why such an approach can yield huge benefits. The companies claim that the ASIC is 15-25 times faster than doing the same work on a CPU leading to substantially lower latency.
Microsoft opened the work here, putting Zipline on Github. It is recruiting its hardware and CPU vendors such as Intel, AMD, Ampere, Marvell, Mellanox, and Broadcom to support the new standard. For others in the industry, this space is worth watching. If Microsoft can get its hardware partners to provide support, Zipline may become a defacto standard from the cloud to the IoT edge in a few years.