Mini PC ASUS Ascent GX10, NVIDIA GB10 Blackwell, 128GB RAM, 2TB SSD

Pro:
- **Runs a 120B reasoning model locally** — genuinely, offline, private, at zero marginal cost. A 4090 can't hold it; a Mac Studio can but has no CUDA.
- **128 GB unified memory + huge context windows** (128k–256k) — the real differentiator. It reads whole files and diffs without truncation, which matters more for real work than raw speed.
- **Power-efficient** versus an equivalent multi-GPU build (tens of watts idle, a few hundred under load, against ~1.8 kW for 4×4090).
- **Desktop form factor**, and the CUDA / llama.cpp stack mostly works on ARM64.
- **An end-to-end local dev pipeline is actually viable** — proven on real, shipped code.
- **Small/MoE models are snappy** (70–86 tok/s) — great for utility and mid-size assistant work.

Contra:
- **Cold model swaps take 1.5–2.6 minutes** (measured 85 s / 156 s), plus a brief full-box UI freeze at the moment of swap. Any workflow that rotates models pays this repeatedly.
- **Only one big model at a time.** On the 128 GB pool you cannot hold two 50–60 GB models; every reasoner↔coder transition is a reload.
- **Runs hot during loading/swapping** — noticeably warm, fans spin up each time a big model loads.
- **CPU pins at 100%** on the inference server, *especially* when compiling a Rust project **while** a model is running — `cargo` and `llama-server` saturate the cores together, and the box has no headroom to build and infer at once.
- **128 GB ceiling** — no frontier-tier models (235B / 671B). gpt-oss-120b is the practical top.
- **ARM64 ecosystem friction** — repeated papercuts: no arm64 Chrome (broke headless rendering until I switched to arm64 Playwright), native-TLS needed vendored OpenSSL, and some tooling still assumes x86.
- **Immature power management** — it froze hard in sleep early on; I had to mask suspend/hibernate entirely to keep it stable.
- **Modest bandwidth (~273 GB/s)** caps decode speed; the 120B at ~24–36 tok/s is usable, not fast.

Altele:
## How I use it

A fully-local AI software-development pipeline — a Discord-driven bot that plans, writes, reviews, and merges real code. Models are served by **llama.cpp** behind **llama-swap**, fronted by a **LiteLLM** OpenAI-compatible proxy. Everything runs offline on the box; no cloud inference, no per-token cost. Over weeks it has shipped real pull requests, including an incremental port of a Node.js service to Rust, stage by stage.

## Performance (measured)

Decode speed, warm, via the litellm endpoint, short prompt:

| Model | Role | Quant | Context | Weights (GGUF) | tok/s (warm) | Cold load |
|---|---|---|---|---|---|---|
| Qwen3-4B-Instruct | utility | Q4_K_XL | 32k | 2.4 GB | **70** | persistent |
| Qwen3-30B-A3B | assistant | Q4_K_XL | 131k | 17 GB | **86** | ~seconds |
| Qwen3-Coder-Next | coder | Q4_K_M | 256k | 46 GB | **42** | **~85 s** |
| gpt-oss-120b | architect/reasoner | MXFP4 | 131k | 61 GB | **24–36** | **~156 s** |

Speed is **memory-bandwidth-bound (~273 GB/s)**, which shows in the pattern: the small and MoE models (3B–4B active) are quick (70–86 tok/s), while the dense-ish 120B reasoner lands at **~24–36 tok/s** — faster than you read, but not fast. A ~2,000-token plan is a minute-plus of generation. The real value here isn't throughput, it's that a **120B model runs at all, locally, with a 128k context window** — you trade tok/s for capability, privacy, and zero marginal cost.

## Memory & model swapping

Because everything shares one 128 GB pool, you can hold **one big model at a time** plus a small persistent utility model. Measured resident usage: ~36 GB with just the utility model, ~63 GB with the coder loaded, **~75 GB with the 120B loaded**. The coder (46 GB) and the architect (61 GB) together would be ~107 GB before KV cache — they cannot coexist, so swapping is mandatory.

**llama-swap** manages this with two groups: a *persistent* tiny model (Qwen3-4B, always resident so per-turn helper calls don't evict the big model) and a *swap* group where the large models load one at a time. It's the only sane way to run several big models on a single pool — but swapping is where the pain lives.

**The swap penalty is the defining limitation.** Loading a cold big model took **85 s for the coder and 156 s (2.6 minutes) for the 120B** in my measurements. The "few-second freeze" you notice is just the UI stall at the moment of the swap; the model isn't actually *usable* for one to two-and-a-half minutes. In a multi-model workflow (plan → code → review, which rotates between the reasoner and the coder), that adds up to real dead time.

## Value

At ~€3,600 it sits right next to the **Mac Studio M3 Ultra (128 GB)**, which has better thermals, near-silent operation, and a more mature platform — **but no CUDA** and a different inference stack. If you specifically want the **CUDA / llama.cpp ecosystem** and a true local 100B+ box at this size and price, the GX10 is one of very few options. If you mostly want fast small models, quiet operation, or polish, the value is weaker.

## Who it's for

- **Yes:** local-AI developers and tinkerers who want big models offline, value 128 GB + CUDA, and will tolerate the rough edges and minute-long swaps.
- **No:** anyone wanting plug-and-play polish, silent/cool operation, frontier-model capacity, or a mature ARM software experience.

## Verdict

**3.5 / 5.** A remarkable capability wrapped in a hot, slow-to-swap, first-generation package. Buy it for what it *uniquely* does — a local 120B reasoner with a 128k window on a desk, fully private — not for comfort or speed. If your workload tolerates the swap penalty and lives within one big model at a time, it's one of a kind. If you need fast iteration across multiple large models, or polish, look at a Mac Studio or a cloud endpoint.

Pro:
Până acum a funcționat așa cum mă așteptam. Nu am întâmpinat probleme semnificative.

Contra:
E super scump, deși e cel mai ieftin din gamă.

4 comentarii

Adrian - Acum 2 luni

salut, pentru ce folosesti aceasta unitate ? Local ai ?

Alexandru Serban Zodian - Acum 2 luni

Da pentru local AI, am vrut sa rulez Mistral 3-5 Medium cu Eagle speculative decoding.

Darius Florea - Acum 2 luni

Salut, cred ca ai castig de cauza mai bun cu modele mai mici, MoE sau sa aiba active params mai putin, am reusit sa rulez la o viteza decenta GPT-OSS 120B, Qwen3.5 122B A12B, sunt cateva repos publice si pentru a rula qwen3.6 27B pe la 40-50 tok/s, gemma4 27B A3B. Am rulat si nemotron 120B dar e cam slabut pe partea de code review, depinde ce vrei sa faci cu ele.

Sunt curios ce setup ai, cum te folosesti de spark

Marian Busoi - Acum 2 saptamani

DeepSeek V4 Flash 0731 e super, dar ideal e sa ai doua Asus in cluster.