Qotom Q20332G9-S10 Power Consumption
The unit came with a 60W 12V power adapter. We have seen these units from Dajing before, and we have a photo of this power adapter along with the system so you can see all of the regulatory markings/ certifications, and what may be missing.
In terms of idle power, we were a little surprised. We saw idle in the 16-18W range. With three low-power internal SSDs, we saw maximum power in the 32-35W range.
There are things in this system, like the ASPEED AST2400, that add to the cost and power consumption. It provides useful features like video output and watchdog functionality but also adds power. On the other hand, the onboard X553 SFP+ networking is lower power than adding an expansion 10GbE card, especially a quad port one.
While the idle power consumption is high, the maximum power consumption is very reasonable. The other aspect to remember here is that we have five 2.5GbE Intel i225 NICs and also built-in 10GbE networking. Power can vary based on cable lengths and what types of SFP+ modules are used and be higher than our range at the maximum.
Still, a key benefit is that the system is fanless and uses an 82C Tcase embedded processor. Still, if this was sitting in a non-air-conditioned building in Scottsdale, Arizona, we would certainly rig some sort of fan over it.
Key Lessons Learned
The overall key lesson learned is simple. This is more of a server and an embedded appliance than many of the other systems we see. Qotom almost certainly has an OEM customer for these boxes. We lose high-end video features, but for a NAS, firewall, or virtual machine/Kubernetes/ container appliance, this has everything.
On the NAS front, however, we were intrigued and covered this in the video. This has more networking than the TrueNAS Mini X+. SSD options are fairly close between them. The CPU is the same. Mini X+ is probably a bit better, but if we could lose a single 3.5″ drive, gain networking, and do it at half the cost, maybe it could be a NAS too. That led us to the external SFF-8087 port.
Again, this is an internal port, but the advantage is that it is a locking connector, making it much safer than a USB connection or most eSATA connections.
We then purchased a QNAP TL-D400S that we reviewed. This is a 4-bay SATA JBOD DAS enclosure.
The QNAP comes with a PCIe card for a NAS or PC that has a SFF-8088 connection and a SFF-8088 cable. We bought a SFF-8087 to SFF-8088 cable (Amazon Affiliate for the one we used.) That let us directly connect the JBOD to the fanless Qotom.
Once we did, the drives were immediately recognized.
Of course, one had to fail, but we replaced it with a 20TB drive that worked. It looks like this particular 18TB drive just had a rough move.
Perhaps the coolest part of this is that one is left with a SFF-8088 to SFF-8088 cable and a small PCIe SATA SFF-8088 controller.
Putting those into another system would mean the JBOD/ DAS could be directly connected to another QNAP NAS or workstation for high-capacity and low-cost sneakernet transfers.
One can build a diskless (save for the boot drive) system with 2x M.2, 1x 2.5″ SATA, and 4x external 3.5″ SATA for just over $700. That is, again, with 4x 10GbE and 5x 2.5GbE networking as well. This is the type of setup that should make TrueNAS Mini X+ buyers at least pause. Likewise, if you are getting a Netgate 8200 Max, this can be very competitive from a hardware spec perspective at ~$1000 less. Given that the Silicom Netgate platform and the TrueNAS Mini X+ have nicer hardware, this is a lot of money to save for similar features.
Final Words
We have been using this system since October. It gets hot, but many similar embedded systems do. It does not have IPMI (but there is watchdog), nor a high-end transcoding GPU. At the same time, we constantly get folks asking for more 2.5GbE ports than 4, SFP+ 10GbE networking, lower top-end power, a way to hook up 3.5″ SATA SSDs, and more. This little system checks every one of those boxes while using a real embedded CPU that many OEMs built their appliances around.
Some will immediately balk at the eight E-core CPU performance of Denverton, or mention that they will not use Intel QuickAssist. Still, for a low-power non-media server, this might be the absolute best package out there.
For those looking for others with these systems, check out the STH Forums thread here.
Where to Buy and Components
We had to buy a lot to make this work. Thank you to the STH YouTube members for giving us the budget to make a piece like this possible via their subscriptions. Here are some affiliate links to what we purchased. There are, of course, other options, but we thought it would be worth giving folks a starting point:
- Qotom via Amazon
- Qotom via AliExpress
- Qotom alternative via AliExpress
- SFF-8087 to SFF-8088 cable via Amazon
- QNAP TLD400S JBOD via Amazon
- Cheap M.2 SSDs via Amazon
- Cheap SATA SSD via Amazon
Note we participate in several affiliate programs including from Amazon Services LLC, so we may earn a small commission if you purchase through these links. We need to pay for buying all of this stuff somehow.
Thanks STH for actually doing real home-lab content. This is a nice system because it’s a real CPU with real NICs and fanless. I still want to see the ms-01 review, but I’m thinking this is a better value. You’ve got the makings of a 3 host Proxmox VE cluster with this for under $1000 and 100W. If each has that DAS you’ve even got enough to do Ceph.
Here is a question, If I have a Xeon-D 1540, is it worth replacing it with this? Currently running it as my firewall, but experiencing odd latency I can’t put my finger on with my knowledge of Opnsense.
I’m curious to know if the C3758R version 4 lane M.2 slots? The R version has 20 PCIE 3 lanes instead of the 16 PCIE 3 lanes on the C3758.
Hoping you guys can publish a review of it configured as a router or firewall. It’s no great shakes in the CPU department, nobody will buy it to compile C++ code…
Some of these Aspeed chips on Board, but no Mgmt-Software on it, very curios!
Maybe you can test some switching and routing performance of this board, would be nice.
It’s got the same CPU as the Netgate 8200 Max that’ll do 18Gbps firewall iperf3 in pfSense
Here’s the speed using the C3758R version in the Netgate 8200 Max specs:
L3 Forwarding
IPERF3 Traffic: 18.60 Gbps
IMIX Traffic: 11.76 Gbps
Firewall (10k ACLs)
IPERF3 Traffic: 18.55 Gbps
IMIX Traffic: 5.1 Gbps
IPsec VPN (AES-GCM-128 w/QAT)
IPERF3 Traffic: 3.24 Gbps
IMIX Traffic: 810 Mbps
That’s why you want QA.
These aren’t new processors. They’re used in so many branded firewalls and other appliances I’m shocked there’s questions on the network perf.
7 year old Atom C3758 = FAIL
Can that ancient 14nm chip route even one 10Gbps connection let alone **four**?
@Patrick – can you confirm if the system actually supports ECC?
although the “barebone” 18w idle is high, and 35w with 3 SSDs suggest some issues… perhaps some bad support in power saving features by the bios? …. unless of course the issue is with drives used – can you perhaps re-test this with some “known-to-be power-saving” drives?
(I’m interested, since I’m in search of possibilities for low idle power home server)
Uh, Carl, your question was literally answered by AldiK in the comment before yours.
It’s a cool system. I’d buy 2 barebones and use them for silent HA cluster.
Stanos I looked at their Supermicro A2SDi-8C+-HLN4F motherboard review with the same chip and it’s really close 19.1W and 32.5W https://www.servethehome.com/supermicro-a2sdi-8c_-hln4f-review/4/
Perhaps that’s what the C3758 uses?
@Patrick did you test the the SFP+ ports with any linux distribution using kernel 6.*, including Proxmox? They work in FreeBSD based systems (*sense).
I have a similar system and the intel x553 SFP+ does not work in Proxmox 8.* reported in Bug 5103 – https://bugzilla.proxmox.com/show_bug.cgi?id=5103
Whether the networking capabilities of an Atom-based firewall appliance are well known or not, I’d prefer that a technical review of such an appliance verify the expected performance and include the results in the report. For that matter, how fast it takes to compile a Linux kernel seems as relevant as using the firewall appliance to warm a cup of coffee. Even so, I found the review and hardware interesting. Thanks!
Would it be possible to introduce a testing methodology for firewalls and networking?
10 Gb ports driven by Intel X553. Last I looked Linux kernel 6.xx did not support the X553. Has this changed? If not this seems to be a rather severe limitation of the unit for any of a variety of new releases. At first glance this looks like a good candidate for an inexpensive experimental Ceph network. But if the 10G ports cannot be used . . .
If that had something like an N305… i’d be all over it.
I was also excited to use these systems as routers. Unfortunately, 4 of them failed over years, after approx 1 years of 24×7 use consistently.
I am still in favor of fanless systems, but am staying away from Qotom specifically as unreliable.
Unless you want to have a spare server fully configured on standby ready to replace a failed system, those are not reliable.
As a former Netgate staffer I found the 8200 price and rack requirement off-putting.
This week as I needed to purchase a 10G firewall for an event kit I considered the 8200 (despite the experiences I had at the company) and I absolutely could not justify the $1300 price tag under any circumstances and purchased this specific firewall on Monday for the kit.
I am looking forward to getting it on my test bench in a week or two.
Netgate needs to up their game – the 8200 price tag is way too high, but then again all their hardware prices are very high.
Does anyone know if this will run VMware?
@Curious: I don’t see a reason it could not run it. It might not be the current release, though… so you should check the VMware HCL against the known parts of this system to be sure.
buckm – the C3758R is x2 bifurcation as well, so you do not get more lanes.
SteveP – Just ran iperf3 between the C3758 and C3758R both running Proxmox VE 8.1.3 and they are passing traffic at 9-10Gbps. See https://forums.servethehome.com/index.php?threads/the-everything-fanless-home-server-firewall-router-and-nas-appliance.42718/post-406041
This would be an interesting router if it had working IPMI. I just can’t bother with a server that does not have IPMI. Not worth the time and effort to hook up a monitor, keyboard, etc. when something goes wrong and I need the console.
This isn’t bad, but as others have pointed out, the lack of iKVM isn’t particularly great.
Furthermore, that 3052 m.2 slot underneath the SSDs is a horrible idea. High-end 5G modules can easily dissipate enough heat to require a heatsink and having to install them underneath the otehr m.2 drives makes that impossible.
Hey, would the USB-c allow a USB-c DAS?
Such as https://www.qnap.com/en/product/tl-d800c
Great box – sad the QTOMO currently does not ship to Germany. Probably some misconfiguration in their Aliexpress store.
@George Salambassi – there is no USB-C on this device
Sadly I cant order this unit in germany
@George Salambassi – oh I see the SS+ on the front now… ehh. Probably?
Patrick – It is good news about Proxmox (8.1.3 maybe typo?). This is using the 6.5 kernel. Ubuntu LTS is still using 6.2, which at least originally had the problem. It looks like the X553 problem must have gotten fixed in the newer kernel. I don’t know if Canonical has backported the fix since I have avoided the X553 for this reason. I appreciate your having done the test.
Can you do a test where you put all 5 2.5g ports into a l2 bridge and then test this as if it was a switch?
I’m sure it can’t do full line rate on all ports, but how much can it do?
If I had this, I’d use it as a firewall and seriously consider just connecting 3 or 4 devices directly to it and not bothering with a separate switch. With 1 wan link, one access point, a 10g nas and 3 wired hosts there would still be one free port.
ECC works?
The motherboard has 2 SATA ports but only one SATA SSD was connected for the article. Is there internal mounting space for a second SATA SSD?
@Tim – I just got mine and, no. Just the single 2.5 screw holes but you could certainly bodge it for SSDs.
@Robert, true, the lack of IPMI or iKVM is not great, but at IPMI cost 5-9W (depending on implementation), but a console port is also not bad, you can do connect it to a RPi Zero W and have remote administration.
@Robert, missed the AST2400, then it is strange that they have not added the IPMI features, maybe a bios bug.
Also wondering if ECC SODIMMs are confirmed working, would be ideal for a proxmox host with ZFS
It also doesn’t work with alpine linux 3.19 no igxb (pretends to be there but no link ever)
ECC so dimms work, but the 10gbps thing is a MESS. I can’t reccomend these if 10gbps is your desire. I tried three different brands of twinax cables sfp+ that work fine with my ciscos, and other (brocade etc) they REFUSE to connect to ‘switches’ which include a zxytel XGS1210-12 AND an aruba 1930 24G Class4 PoE 4SFP/SFP+ 195W Switch JL683A.
The really weird thing is if I ‘cross over’ the sfp+ cable to its two interfaces it comes up and works fine. It just won’t connect to anything else 10gbps that I have with multiple twinax sfp+ cables (including the expensive cisco ones)
So if you want this for 10gbps its a bit of wash. Its kind of sad that the reviewers didn’t do a better job of explaining how tempermental the sfp+ ports are. I read TONS of people having problems with this (the authors claim to not have had any problems) myself included (the problems part).
I even used the ‘same’ version of proxmox and couldn’t really ever get things working. I might try freebsd just to see if its working. Why can’t they tell us what SFP+ modules work with the 2#$$@#%#$^%#$% thing? Or even better yet which twinax cables actually work?
Mostly its a nice PC, just don’t count on 10gbps working unless you are willing to spend tons of time on it. I was sad, since I wanted it for a nas, but its kind of useless as that. It will make a nice vm box, I just will have to live without the 10gbps until someone kindly shows me the right cable.
So if you want the 10gbps modules to talk to themselves, the 10gps works great, otherwise IMHO 10gbps is not a reason to buy this box. I tried MULTIPLE operating systems, and multiple cables with no ability to get this working.
oh and they take an eon to reboot.
I just installed truenas core and it seems the BSD drivers dont’ hate the thing. The linux ones are a disaster so alas all things linux are out unless you are a scholar and a gentleperson.
Its working with my 10gbps switch (the aruba) and the cisco twinax cable. Seems to be a dorky linux driver than. Sadness.
Final update: it seems like twin Ax is a very bad way to use 10G. This thing seems to prefer Intel SFP plus cards. They’re not too expensive on eBay. I could not recommend using TwinaX when you have multi-vendor switches.
Thomas Munn: FS.com sells DACs where they allow you to pick the vendor code for each end.
On an unrelated note, my unit came with a 12V 7.5A PSU from Channel Well Technology. CWT has designed premium ATX PSUs for consumer brands like Corsair in the past, so I’d expect it to be pretty decent, not the type of generic PSUs we often see included with these mini-PCs. It’s also UL rated.
Dear,
what is de max amount of Memmory and storage for the Everything Home Server.
Does the Everything Home Server support ProxMox or VMWare ESXi
We need for small environments a small server solution.
Many thanks and best regards,
Wim Kobben, Erasmus IT Services
Tried multiple 10GTek SFP+ and SFP28 DACs into a Mikrotik CRS310-8G+2S+IN, CRS326-24G-2S+RM and Zyxel XS1930-10 and no dice. I think it’s probably the Linux drivers, but if you can’t run newer Linux Kernels it’s pretty worthless IMO.
Some updates on my continued travails with this box: I was running it in my server closet (which does admittedly get quite hot) and the passive cooling fails massively. The system completely froze and wouldn’t restart until I unplugged it and let it ‘cool down’. Seems OK in ambient nice 23C 70’sF but in anything approaching 95F (about 30 something C) it sadly dies. To answer the other person’s question 64GB max, with ECC. I simply cannot recommend this system based upon my own testing and attempts to make it usable in a reliable fashion. The ‘fix’ is to manually updated the drivers for proxmox (which require using DKMS and all kinds of unpleasant kernel things) and ‘hopefully’ it works. You also need to use the INTEL drivers (from intel’s out of tree distro) since some genius in the ‘in tree’ kernel COMPLETELY removed the autonegotiation function of the driver!!!!
Final say on this: I got all my fancy SFP+’s only to have the unit refuse to boot (aka power on) anymore. I am sending it back to amazon. STAY AWAY.
The C3758R version sold out quick! Luckily, I got my order in. This one’s going to become my new gateway! I don’t think this one will be a heavy lifter, as the atom only has E-cores, but it’ll be nice to offload some light tasks like pihole and maybe Home Assistant.
I suggest enabling the s3 – power up, after power loss
And regarding colsole
I suggest setting it to VT100, 115200 and the same with opnsense.
Then a simple RJ-45 cisco console cable to usb-c will work in the bios.
regards
Svemd
@Svend Gundestrup Did you have to do anything special to get the console port working? I don’t get any output from it with a Cisco console cable using minicom, but it’s possible I’m doing something wrong. :)
@Svend Same question – I’m using `minicom –device /dev/ttyUSB0 -b 115200 -t vt100` and the console is dead preboot out of the box (i only pulled out the drive and that’s it).
I just picked up the C3758 (non-R) and a Noctua 5V USB powered fan to keep it cool. Picked up some large rubber band to keep it attached to the top grill.
I’m currently planning on installing OpenSense on it. I was going to go with Pfsense, but decided at the last min to change course.
I’m going to use all the ports on day one once I set it up. I’m also planning on getting the Minisforum MX-01 to have both of these units next to each other, connected together via the SFP+ ports.
After reading everything here and what i’ve seen written around online, it appears going with an Intel branded SFP+ adapter makes the most sense. Any suggestions beyond that? Any other alternatives?
So I’m not sure what all the handwringing about SFP+ DACs is. I’ve just loaded Proxmox 8.1.4 on my new rackmount unit and I’ve tested it in my company’s lab with a 10Gtek CAB-10SFP-P2M (my go-to cable for mixed-vendor applications). I’ve tested it plugged into our Mikrotik CRS326-24G-2S+ (first pic) and Unifi USW-PRO-48-POE (second pic), iperfing to our TrueNAS scale server (single Intel X722+same DAC, also plugged into the Pro-48). https://imgur.com/a/X90QcUX
It’s perfectly fine. No boot hangups either with the DAC plugged in.
I’m going to redact the above comment for reasons that should be obvious when you look at the iperf results ♂️ More testing required, will be back.
Alright so I’m back. Even after trying a bunch of different options in Proxmox 8.1.4 to enable the X553s (see this thread: https://www.servethehome.com/intel-x553-networking-and-proxmox-ve-8-1-3/), nothing worked. So I nuked it and went back to 7.4X (completely stock) and wouldn’t you know it, it works like it’s supposed to now. https://imgur.com/a/xnySI4N This leaves no question in my mind about the DAC or switches being the problem – the new driver tree for 8.X’s kernel is absolutely the culprit here.
Got mine today. Loaded Proxmox 8.1.4 in UEFI boot (systemd-boot) and updated to kernel 6.5.11-7pve. Can confirm the X553s will NOT link. I do get a syslog note saying: ixgbe 0000:0x:00.0 eno#: detected SFP+: #, but no link/communication.
The QNAP is sold out EVERYWHERE! I called the manufacture and they said their was no lead up time on the box. dude, SO many people are getting this. any alternatives that use that SFF cable?
I absolutely love my Terra master is a bit pricier but it works!
message to everyone who wanted to buy :
i live in china and they had some models are back in stock, receiving mine soon :)
try to msg them on a aliexpress to get yours
version C3758R