iDRACula Vulnerability Impacts Millions of Legacy Dell EMC Servers

14

A Broader Industry Discussion and Concern

Let us be clear, while iDRACula focuses on Dell EMC iDRAC, there is a broader industry concern here. The company had resources mobilized to understand and reproduce what happened on the STH forums within hours. Other vendors do not have this response time.

In order to remotely execute an iDRACula firmware swap, you need to have remote access to the BMC and an existing login. There are still a startling number of IPMI management interfaces directly addressable via a public IP. Further, the default logins are often left in place. Here is a list of the default logins I wrote down in 90 seconds giving myself the task:

  • Dell EMC iDRAC: root/ calvin
  • Supermicro: ADMIN/ ADMIN
  • Cisco UCS: admin / no password
  • Other Cisco: admin/ admin or cisco/ cisco
  • Intel Remote Management Module: admin/ password
  • Lenovo IMM: USERID/ PASSW0RD
  • Gigabyte Server: admin/ password
  • QCT Networking: admin/ no password
  • ASUS ASBM: admin/ admin
  • APC: apc/ apc
  • ASRock Rack: admin/ admin

These were a few top of mind default logins. Exposing interfaces with default logins to the Internet is not a leading security practice, yet it is something done on a regular basis.

Newer Dell iDRAC implementations, along with those like Cisco UCS, prompt users to change default credentials immediately. Those in the industry can probably attest that there are a lot of “root/ calvin” Dell EMC servers with that login active. Likewise, ADMIN/ ADMIN on Supermicro servers and some appliances that use Supermicro as a building block is common and often exposed to the internet.

While many will dismiss the physical access variant, the nature of the vulnerability makes it quite the intriguing vector. We often think about servers operating in a data center. There is a step just before that. A server is rarely fully assembled on-site. Even if it is, motherboards with the BMCs are created elsewhere. That shipping process exposes servers, prior to installation in a data center to various entities in a supply chain. Larger enterprises often order servers not in a box with a tamper-proof seal, but instead as pre-integrated racks ready to wheel into the data center and install. That gives ample touch points where a malicious actor could use this vulnerability to compromise a BMC. Once compromised, a user would not be able to see that this has taken place.

Dell PowerEdge R930 Excellent Packaging
Dell PowerEdge R930 installation in our data center

Imagine if a financial institution could remotely program all 1000 nodes of an exchange or competitive firms servers to reboot at exact times by loading a script to do that before the servers were installed. Also, state actors who can inspect shipments of servers during standard customs screenings would have access to utilize this vulnerability with a reason to delay servers and break security tamper seals.

You may assume that just because this impacts the previous generation servers it is not an issue since the new 14th generation PowerEdge servers are now shipping. Remember, Intel significantly raised prices on its chips with this Xeon Scalable generation, and the previous generation Intel Xeon E5-2600 V4 servers that are susceptible to iDRACula are still shipping in quantity in 2018.

Infrastructure’s largest consumers such as Microsoft and Google are taking management firmware very seriously. They are creating a hardware root of trust chain to validate that the correct software and hardware is in a server. Dell EMC and HPE have been pushing similar solutions. Facebook has been active with the OpenBMC project to push auditable firmware on to BMC devices.

Google Titan Chip Requirements Hot Chips 30
Google Titan Chip Requirements Hot Chips 30

From an industry perspective, securing BMCs needs to be a priority. They are used both by single server shops as well as the largest infrastructure providers. BMCs make servers manageable without full-time data center staff or expensive remote hands. They are what make the digital infrastructure run and provide management for the machines that trillions of dollars worth of global economic activity rely upon.

Final Words

After hearing about this vulnerability, I wanted to give it a memorable name, hence calling it iDRACula. The importance of BMCs, the prevalence of the technology in Dell EMC servers along with other brands, and the potential for vulnerabilities like iDRACula mean that this is is an area of data center technology that needs your attention. Although iDRACula focused on Dell EMC servers, we know of other management solutions that are more vulnerable. Dell EMC has been shipping a new generation of servers with updated hardware to address this type of vulnerability for over a year now. On the other hand, there are large numbers of legacy PowerEdge products still in the field.

We hope that reading this article, STH’s IT professional readers will take a moment to ensure that, at a minimum, their iDRAC (and other IPMI management) firmware is updated, and that iDRAC interfaces are not on public networks and do not have default logins active. Take this article and have a discussion with your teams about iDRACula and how you may have exposed BMCs. The prospect of a malicious actor utilizing these vulnerabilities with great effect is too great to not spend a few minutes reviewing this.

The next step questions for your team today would be:

  • If this can be done on a Dell EMC PowerEdge, generally considered one of if not the best management solutions out there, what about our other servers?
  • Could someone have intercepted the servers and applied a corrupted firmware image in transit. These are exactly the questions that the hyper-scale community like Google and Microsoft see as a big enough threat that they are introducing their own hardware solutions to give their customers peace of mind.

Even casual industry observers recognize the power of remote management. That power makes scale-out automation of servers possible as well as small deployments for startup organizations. The other side of this power equation is that it is a great power that needs to be guarded. We urge our users to take this opportunity and evaluate their risk profile for an iDRACula like vulnerability.

14 COMMENTS

  1. Holy shit. Good job presenting. I’ve sent this to our team as a reminder we need to take this kind of security seriously and not go lax over time. I saw the post in the forums before it was taken down and I’d not put 2 and 2 together.

  2. So if I buy a new PowerEdge R630 today, ship it to our South American hub the person installing it tomorrow could do this to the server before I even see it is online? iDRAC is pre-boot so I’d have no way of knowing.

  3. Aaron: a CVE is in the process of being submitted by dell, yes. While the remote process does involve rolling back to an older firmware, the issue/actual exploit lies in the fact the latest firmware has some holes that when used, allow you to then re-flash to the latest firmware and keep exploits in place undetected

  4. Ok, so by handing a thief the key to your car and/or the key to your garage (because you chose to trust them)…. They can now take the motor out and modify it and put it back (hey that sounds fun)…. Helllllo you gave them the keys!!! They can turn it on or off anytime and have their way with it at will just by utilizing the key!! Or better yet, if I’m the owner and I hold the key I can have MY way with MY server, my choice of handing out the key or leaving the door open is the obvious security vulnerability at large. While a valid security flaw this is just a bit over fluffed. People need to learn more about enterprise best practices to prevent this, such as management vlans blocked from the internet, video surveillance and physical access control… rather than only buy 14th gen servers or learn the hard way.

  5. John, you’re missing a huge part of the server deployment process, the supply chain. You can guard against physical access all you want once you have the equipment in-hand, but it doesn’t matter if anyone who had it previously had bad intentions, especially given as-of-now anything malicious they could potentially load will survive standard firmware re-flashes. Using your analogy, it’s more akin to buying a car that runs unsigned code on the engine control unit, and a pissed off employee at the dealer/tow company/etc loaded in their own modified firmware. The car seems perfectly fine, you keep it secure in the garage and nobody else has the keys, but one day that modified code is going to slam you into a wall.

    Is this likely to happen to rinky-dink IT companies with not much to steal from? Doubtful, the chances are very low. is this likely to happen to large providers, financial institutions, etc? Yes, it already has happened. There is a reason Facebook, Microsoft, OpenBMC etc have spent millions solving the unsigned firmware issue, they have seen the damage it can do. 5 minutes of physical access to hardware to load something that will never be cleaned away is much easier for an entity to achieve in supply chains then you think, just ask any of the victims of previous exploits

  6. This is why you never ignore defense in depth. Physically lock and secure your datacenters. IPMI traffic should be in an isolated vlan with no direct internet or end user access.

LEAVE A REPLY

Please enter your comment!
Please enter your name here