Bloomberg Sources Raise Questions
Over the two articles, Bloomberg cites a number of sources. Let us take a tally:
- Six current and former national security officials
- Three insiders at Apple
- Six former (Supermicro) employees
- One executive of a large web-hosting company
- The person or persons familiar with Amazon’s probe (none from Amazon were cited)
- Joe Grand
- Joe Fitzpatrick
- Perhaps others, but the anonymity makes it difficult to trace.
Assuming that the Apple, Supermicro, and national security officials are all distinct, as is the executive at a large web-hosting company, that is sixteen sources. The Amazon source was called a person familiar with the probe, but we do not know if that is one of the sixteen sources. Joe Grand and Joe Fitzpatrick were cited, but including them would mean we had an overlap of the first sixteen or we actually have eighteen sources.
Let us dissect that list and see what has been said after the article’s publication.
Government Security Officials Cast Doubt
In the original piece, the FBI and the Office of the Director of National Intelligence declined to comment on the story. Since then the FBI and the Department of Homeland Security in the US and the UK NCSC have all commented, suggesting that they had no evidence to support Bloomberg’s claims. Likewise, officials at the NSA have also not seen supporting evidence, nor has the Director of National Intelligence, Dan Coats. Those are fairly big governmental sources. However, there are “six current and former senior national security officials” cited by Bloomberg anonymously supporting the story.
These sources seem to be where Bloomberg is getting its functionality details from. These are the same details that we have shown earlier in this article technically do not align with reality.
Apple Calls Bloomberg Untrue and Calls for Retraction
“Three senior insiders at Apple” are cited in the piece. Although those are anonymous sources, Tim Cook has now called the Bloomberg piece not truthful. This is after a forensic investigation including looking into e-mail records. Comments like these are not made mistakenly. If Mr. Cook made the statement, one can assume it has been thoroughly vetted within Apple.
Apple cited the reporters at Bloomberg as being confused in their initial response. An obvious question is how there could be multiple senior insiders at Apple involved, along presumably with their subordinates who are handling the hardware and security, and a forensic investigation turns up nothing at Apple? This is also not an anonymous PR denial, this is Tim Cook as CEO saying that Bloomberg’s efforts were poor putting together the story and not truthful in their portrayal.
Amazon AWS Response Makes Sense
Amazon Web Services issued a specific response calling Bloomberg’s article “untrue.” In that response, they talk about BMC security flaws that are mitigated through patches or by using the BMCs on a properly configured network. We discussed this proper configuration as it is the default configuration for organizations who run servers. Again, for a high-level overview, we have a basic guide on how this works. If this is a BMC hack, Amazon’s response makes sense as does Apple’s.
@tim_cook is right. Bloomberg story is wrong about Amazon, too. They offered no proof, story kept changing, and showed no interest in our answers unless we could validate their theories. Reporters got played or took liberties. Bloomberg should retract. https://t.co/RZzuUt9fBM
— Andy Jassy (@ajassy) October 22, 2018
Supermicro Responds More Strongly
Supermicro issued a blanket and specific denial. They have gone a step further and sent out a letter from Charles Liang, CEO of Supermicro that is clear and unambiguous:
We are confident that a recent article, alleging a malicious hardware chip was implanted during the manufacturing process of our motherboards, is wrong. From everything we know and have seen, no malicious hardware chip has been implanted during the manufacturing of our motherboards.
We trust you appreciate the difficulty of proving that something did not happen, even though the reporters have produced no affected motherboard or any such malicious hardware chip. As we have said firmly, no one has shown us a motherboard containing any unauthorized hardware chip, we are not aware of any such unauthorized chip, and no government agency has alerted us to the existence of any unauthorized chip. (Source: Supermicro CEO Letter to Customers Dated October 18, 2018.)
Supermicro in the same letter also explains the difficulties we noted with placing additional wires on a motherboard. This type of declaration is far from “no comment.”
OVH (Major Web Host?) Investigates and Responds
OVH is a major web hosting company and the cloud provider that bought VMware’s vCloud Air. The company makes its own servers and utilizes many Supermicro motherboards. OVH has also said that they have investigated and did not find tampering. We do not know if OVH is the large web-hosting company executive source, but it would be the first company we would think of given that description.
This one is a mystery, but if you were targeting 30 companies and a large web host that uses Supermicro is one of them, OVH would be at or near the top of that list.
Named Sources React to Bloomberg’s Reporting
Joe Grand, a cybersecurity expert noted in the article for talking about the implausibility of being able to pull this off. Later, Mr. Grand noted that traffic would be seen if there was indeed a command and control structure, that nobody seems to have found. Joe Grand’s take:
FWIW, my quote in the @business article was given over a year ago as to the difficulty and unlikeliness of finding a properly instrumented hardware implant. I've not seen any proof of this specific alleged attack.
— Joe Grand (@joegrand) October 8, 2018
That does not exactly support Bloomberg’s claims.
Likewise, Joe Fitzpatrick was named in the article. If you hear his interview with Risky Business, he says he is uncomfortable with how Bloomberg reported the story. He also said that the way that Bloomberg presented the hack did not make sense.
In a follow-up piece, Bloomberg brought another, completely different, hack to light citing Yossi Appleboum. I spoke to Mr. Appleboum and he was angry with what happened to Bloomberg’s story. He also noted that he told Bloomberg that this impacted many vendors, not just Supermicro and that it was likely done after manufacturing, while the servers were in-transit to the data center. Here is an excerpt from both showing the juxtaposition of Bloomberg’s portrayal in that second piece and my interview:
During this investigation, I was personally shocked by how starkly Bloomberg’s reporting and Mr. Appleboum contrasted. Read Yossi Appleboum on How Bloomberg is Positioning His Research Against Supermicro and draw your own conclusion.
We will let our readers decide if there is a clear pattern when the only named sources and source companies in the investigation are less than supportive of Bloomberg’s reporting.
Six Anonymous Former Supermicro Employees
We know at least six sources are former Supermicro employees. Here is an excerpt:
“The majority of its workforce in San Jose is Taiwanese or Chinese, and Mandarin is the preferred language, with hanzi filling the whiteboards, according to six former employees.” (Source: Bloomberg)
We are not going to print the rest of the section because it paints the employees in a way trying to tie them to China and Taiwan. This may be part of Bloomberg’s agenda with the piece, but we are not going to publish comments that are trying to incite a feeling about a company due to the ethnicity or cultural backgrounds of its employees.
Since I personally spend time at Supermicro interacting with their teams for reviews, I can say that Mandarin is used. Supermicro, like many hardware companies, has an office and production facility in Taiwan, so it is true having language skills helps. If you are in this industry, you need someone on your team that speaks Mandarin and English. That is now table stakes in the technology industry. That Supermicro Customer Letter cited above is from Charles Liang but is also signed by David Weigand, SVP and Chief Compliance Officer, and Raju Penumatcha, SVP and Chief Compliance Officer. Mr. Weigand and Mr. Penumatcha are not going to hold a conversation in Mandarin first, nor is Kevin Bauer (CFO) or Don Clegg (SVP of Worldwide Sales.)
Located in Silicon Valley, there are a diverse set of ethnicities at Supermicro. If six former employees think that Supermicro runs on Mandarin, I would have difficulty interacting there. Instead, all of my interactions with their product teams, executives, and marketing teams are in English. This type of insinuation on the cultural heritage of folks is completely uncalled for, especially when there are a large number of employees, including in executive ranks, that do not share that cultural heritage.
Here we have six former employees who Bloomberg used for interesting means. They did not confirm the hack. Instead, they were simply used in the article to paint a picture of their former employer in a light that would be unfavorable to Supermicro and supportive of Bloomberg’s propaganda storyline. Having six sources to tell you that a non-English language was spoken in a Silicon Valley company is about as earth-shattering as saying the sun will rise tomorrow. Bloomberg’s story never said its six Supermicro former employees saw any evidence of tampering. That is notable.
We will let our readers weigh the evidence to see if there is a pattern in Bloomberg’s sources. Next we wanted to give our final thoughts and discuss next steps.