At Hot Chips 2025, Microsoft went into its Azure Hardware Security. The goal is to keep Azure secure since it is a giant cloud service. Microsoft does not just have to protect against external traffic threats, instead it has multi-tenant workloads to keep safe, and some customers want to ensure they are secure from even Azure itself.
Please note, we are covering this live, so please excuse typos.
Microsoft Azure Hardware Security to Help Thwart the World’s 3rd Largest GDP
Having a Hot Chips article with a cheekier title is not something we normally do, but we saw this slide. In terms of world economies, cybercrime now ranks between Germany and China in GDP according to Microsoft’s chart at just over $9T in 2024, and expected to be over $10T in 2025.
Microsoft had a bunch of great stats. They have 70+ Azure regions, 400+ data centers, 275K miles of fiber, 190+ network POPs, and 34,000 dedicated security engineers.
As a result, their systems are designed for security as part of the company’s Secure Future Initiative (SFI.)
Microsoft is also going into its system architecture. Of note, this looks like a newer retention clip Intel thermal solution with each CPU having 12 DIMM slots. Those 12 DIMM slots are either 6-channel 2DPC or they defy industry convention and use different color locking tabs for alternating channels instead of indicating the first or second DIMM in a channel.
Part of this architecture is also the hardware security modules. These can come as PCIe cards or in server and rack mount solutions.
In typcial hardware security designs, there is a central HSM cluster.
Microsoft is moving its hardware security from the centralized model to being integrated into each server.
This is the Azure Integrated HSM which is Microsoft’s security chip. By having a local ASIC, there does not need to be a TLS handshake to a centralized server.
Having an in-house chip means that it needs to make the trade-offs in chip design. Previously, when this was centralized, Microsoft would scale the HSM at a cluster level. Now, it needs to size it for a single server, but also keeping it low power and in a small area.
Microsoft needs not just AES and PKE crypto acceleration, but also the ability to detect things like intrusion and tampering.
Microsoft also went into its confidential computing.
The idea is to protect data in use especially in multi-tenant cloud environments.
Microsoft is now talking about its Caliptra 2.0 open source silicon root of trust.
This is cool. Microsoft is going into the gate count for different features. This is perhaps the most Hot Chips slide we have seen this year.
Here are the verification release gates.
Since this is a custom ASIC for Microsoft, the question might be why open source it? Here are Microsoft’s reasons:
Overall, this is pretty neat.
Final Words
This was a neat look not just at what Microsoft is doing, but also a but about the “why?” and Microsoft’s philosophy around securing its cloud. Microsoft is more open about this than some other cloud providers, so it is a neat look. If nothing else, learning that the cybercrime market is bigger than the AI market at over $10T and that Microsoft has 34,000 dedicated security folks was neat.
I’m a bit curious what they are up to with the HSM placement.
If you want more than a TPM will give you for platform-locked identification and encryption purposes an HSM seems like a potentially reasonable choice; but for much of what you would choose HSMs for the fact that they live behind a network abstraction, and can be scaled separately from number of hosts, is a virtue: you want your VMs and containers to be able to move around as needed while still having whatever level of cert access they are supposed to have; and (especially with HSMs doing more of the cryptographic heavy lifting) the relationship between number of certs or number of cryptographic operations per unit time and number of cores or RAM size can vary all over the place, which seems like it would make node-itegrated HSMs vulnerable to being over or undersized.
I don’t doubt that they have their reasons; but I’d like to know more about how that one works.
While no one knows an actual number – 10T$/yr in cyber crime is an absolute BS.
Who came up with this nonsense?
I’m guessing that the host based HSM solution works into some kind of loose clustering, allowing for VM relocation without completely breaking the encryption.
Admittedly, this seems to defeat a lot of the purpose of an HSM, but I can’t see how you could actually maintain uptime if a VM was tied to a single physical host for it’s entire lifecycle.