ASRock Industrial AI Box-A395 LLM Performance
For this, we thought it might be fun to go from the GPT-OSS-120B that we were using last year when we started to review Strix Halo systems, to Qwen3.6-35B-A3B and Qwen3.8-27B, which are popular today, and also try NVIDIA’s Nemotron open model. The idea was to show both current LLM performance and how far we have come in the past year. One thing to note, however, is that we aren’t using highly customized engines, because doing that across many systems and models when those engines change weekly or daily is rough.
ASRock Industrial AI BOX-A395 GPT-OSS-120B Performance
Starting off with an older 120B MoE model, here is a quick look at some of the performance using different input/output scenarios:
There is nothing too surprising here, as the larger input/ output calls were slower than the smaller ones. In a rough case users were only getting in the mid-teens of T/s, but on smaller calls users could maintain higher tokens/s/user.
In terms of TTFT, the prefill-heavy analysis scenario was slower to generate tokens, as we would expect:
Just taking a look at tokens per user here is what that looks like:
We just thought it was a fun one to run since about a year ago, this was the big model people were talking about. Now, it is being used much less.
ASRock Industrial AI BOX-A395 Qwen3.6-35B-A3B Performance
A model we have been using a lot is Qwen3.6-35B-A3B because it is small enough to fit on 32GB GPUs without issue, and it is still fairly decent for a multimodal model.
Here we got substantially higher T/s and in most cases, except with aggressive concurrency, we got reasonable TTFT figures.
Here is what the tokens/s looks like with a number of users:
Although this is a smaller model, serving users at decent concurrency and performance is always welcome. The 128GB here is not needed as much for fitting the model, but it helps when you have higher concurrency and larger input/output token counts.
ASRock Industrial AI BOX-A395 Qwen3.8-27B Performance
Qwen3.8-27B is another popular one, but unlike the previous two, it is a denser model. Here we are running the Unsloth UD-Q4_K_M model.
Here are the TTFT metrics to go along with those curves:
Here is another look at concurrency and performance:
Overall, running a model of this class is probably better done on a GPU with faster memory, but it is running pretty decently here if you ever had to step up and did not want to use a MoE model.
ASRock Industrial AI BOX-A395 Qwen3.8 Flash Next Performance
On the Qwen3.8 Flash Next, this is a bigger model, and there are better ways to run it, but we thought we would at least show some of the data points we collected:
Here are the TTFT results:
Here are the throughput vs. users results:
Overall, we did not get much here, but this is a model that is roughly 93.7GB on disk even in UD-IQ4_XS so it is getting closer to the limits of this machine. At the same time, you can run it.
ASRock Industrial AI BOX-A395 Nemotron Performance
Something that is out there still is that just because you have an AMD GPU, you cannot run many models. AMD is pretty well supported at this point, and so we even fired up NVIDIA’s Nemotron-3 Nano 30B-A3B on this AMD GPU system.
Since it is a pretty small model, it performed quite well. You can see we were even running the large_repo_refactor test, which is 65536 input and 4096 output tokens, and driving concurrency through that.
Here is a per user look:
Overall, we wanted to show that you can run an NVIDIA open-weight model on an AMD GPU platform. That is good since NVIDIA has been pushing hard on open-weight models.
