On this page 10
This is the PC half of our Mac Studio M5 Ultra guide. That piece is about a machine with one pool of memory. This one is about what happens on a graphics card when the model is bigger than the card. Every number is ours unless we say otherwise: the same model files and prompts on an RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000 Blackwell, a pair of RTX 5070 Tis, a DGX Spark, an H100, an H200, a B200, a Mac Studio M5 Ultra and a MacBook Pro, through our own benchmark harness. Method at the end.
Where the weights live
A language model writes one token at a time, and every token drags the weights it needs out of memory. For a dense model that’s all of them. For a mixture-of-experts model it’s a small, changing subset: DeepSeek V4 Flash stores 284B parameters and uses about 13B per token, picked by a router that decides afresh at every layer, for every token.
The split isn’t arbitrary. llama.cpp’s fit tool keeps on the card everything a token touches every time: attention, the shared layers, the context cache, and the experts of as many layers as will fit. The experts of the remaining layers stay in system RAM, and the CPU does their share of the math. Which experts a token wants changes every token, so that traffic never stops. On a desktop, 2 to 3GB of expert weights cross the bus per token, and the card spends most of each token waiting.
What a spill costs
Here is where each file ended up, then what that cost. The card’s share includes the context cache; “in RAM” is the file size minus what the card held.
Writing speed first. The grey line under each bar says which PC was behind the card and how much of the file sat in its RAM.
Two clusters. Everything behind server memory writes at 29 to 54 tok/s (leave the 850GB joke aside), and everything behind a desktop’s two channels at 11 to 35, with the lightest model on the fastest desktop at the top of the desktop group and the 2014 Xeon at the bottom. For scale, the same gpt-oss-120b file on a card that holds it, an RTX PRO 6000, writes at 212 tok/s; the same DeepSeek V4 Flash file whole on a B200 writes at 64.
Writing is the good news. Here is the wait before the first word on a 16K-token prompt, which is what an agent hands a model every turn, with a few whole-on-the-card runs at the top for scale.
The trick saves the writing. It doesn’t save the reading.
Writing survives because a token only needs its few experts, and 2 to 3GB over two DDR5 channels is about 30 milliseconds. Reading collapses because a prompt is thousands of tokens at once, and every one of them wants its own experts from RAM. A card that holds the model reads a 16K prompt in seconds. Spilled, it takes well over a minute, up to nearly three, and the card sits at a quarter of its power limit, idling: the 5090 drew 130W of its 575, the 600W PRO 6000 150W, the Radeon 65W of 300. The prompt cache still works, though. A follow-up after that 16K first turn started writing within about two seconds on every split, so you pay the long read once per session.
Now the thesis of this article, in one chart. Same gpt-oss-120b file, eight homes:
The 5090 has 1.8TB/s of memory bandwidth and the Radeon 640GB/s, and once the experts live in system RAM they read within 7% of each other. The pair of 5070 Tis, with the same 35GB in RAM as the Radeon, reads 20% slower and writes at 12 tok/s to the 5090’s 35. The card is no longer what you’re waiting on. Two things set the speed instead, and a desktop has little of either:
- Memory bandwidth. Two channels of DDR5 move about 90GB/s. The Radeon’s PC, on DDR4 at about 50GB/s, wrote gpt-oss-120b at 20 tok/s to the 5090 PC’s 35, with the same amount of model in RAM.
- The cores multiplying those experts. Give the PRO 6000’s PC four cores for that instead of eight and DeepSeek V4 Flash drops from 22 to 15 tok/s on the same two channels. The 5070 Ti pair sat behind a six-core Xeon from 2014, and that’s why it wrote a third as fast as the 5090 PC with the same 35GB in RAM.
The GPU has 24,000 cores and 1.8TB/s. It’s watching eight cores and 90GB/s do the work.
Dense models get no trick
A dense 70B model needs all of its weights for every token, so there is no small subset to keep on the card. Spill it and every token walks the whole file over the bus. In a comparison of an RTX 5090 PC against the M5 Ultra, a 70B that didn’t fit in 32GB finished over 20× slower on the PC, and even an M5 Max laptop beat it. MacStories got the same collapse from a 27B that normally fits: a long context pushed 16 of its 66 layers into system RAM, and it wrote at 4.6 tok/s with 64K of context loaded, 1.5 at 256K. If your model is dense and bigger than your card, there is nothing to tune. Buy memory or pick a smaller model.
Server memory behind the card
The H200 rows in the charts are the same spill with a different PC behind the card: a Xeon with eight DDR5 channels and a terabyte of RAM. DeepSeek V4 Flash doesn’t fit in 141GB either, so 17GB of experts went to RAM. It still wrote at 54 tok/s, faster than the Mac Studio holding the whole model (39 on the same llama.cpp), and read a 4K prompt at 347 tok/s, twice the desktop spill. Reading is where the overflow still shows: the Mac read the same prompt at 778. MiMo-V2.6-Flash the same way: 51 tok/s with 29GB in RAM, where the PRO 6000 with 73GB in desktop RAM managed 18. The overflow is smaller on the bigger card, and the channels behind it are four times as wide. Both help, and a small overflow into eight channels costs you reading, not writing.
We also pushed it to the silly end. DeepSeek V4 Pro is 1.6 trillion parameters, an 850GB file at 4-bit. The card held 135GB, the server’s RAM held the rest, and it ran: 4.7 tok/s writing, two minutes to read a 1,000-token prompt. Running, technically. At that split 85% of the model lives on the CPU side of the slot, and the GPU mostly watches.
Predicting a spill on a napkin
Before we built the office box below on paper, we wanted to know how far to trust the paper. The napkin is one division: the host’s memory bandwidth over the weights a token drags through it. Active parameters times the file’s bytes per parameter gives the second number; the memory channels give the first. Here is that division against every split we measured.
| Split | Active weights per token | Host memory | Napkin | Measured | Measured ÷ napkin |
|---|---|---|---|---|---|
| gpt-oss-120b · RTX 5090 · 33 of 63GB in RAM | 2.8GB | 90GB/s | 32 tok/s | 35 | 1.1 |
| gpt-oss-120b · Radeon R9700 · 35GB in RAM | 2.8GB | 50GB/s | 18 | 20 | 1.1 |
| gpt-oss-120b · two 5070 Tis, 2014 Xeon · 35GB in RAM | 2.8GB | ~60GB/s | 21 | 12 | 0.6 |
| Qwen3.5-122B-A10B · RTX 5090 · 46 of 77GB | 6.3GB | 90GB/s | 14 | 22 | 1.5 |
| Qwen3.5-122B-A10B · Radeon R9700 · 49GB | 6.3GB | 50GB/s | 8 | 13 | 1.7 |
| Qwen3.8-Flash-Next · Radeon R9700 · 84 of 111GB | 4GB | 50GB/s | 12.5 | 11 | 0.9 |
| DeepSeek V4 Flash · RTX PRO 6000 · 61 of 155GB | 7GB | 90GB/s | 13 | 22 | 1.7 |
| GLM-5.3-Flash · RTX PRO 6000 · 64 of 157GB | 9GB | 90GB/s | 10 | 15 | 1.5 |
| MiMo-V2.6-Flash · RTX PRO 6000 · 73 of 167GB | 8GB | 90GB/s | 11 | 17.5 | 1.6 |
| MiMo-V2.5 · RTX PRO 6000 · 92 of 185GB | 9GB | 90GB/s | 10 | 13 | 1.3 |
| DeepSeek V4 Flash · H200 · 17 of 155GB | 7GB | 350GB/s | 50 | 54 | 1.1 |
| MiMo-V2.6-Flash · H200 · 29 of 167GB | 8GB | 350GB/s | 44 | 51 | 1.2 |
| MiMo-V2.5 · H200 · 47 of 185GB | 9GB | 350GB/s | 39 | 29 | 0.75 |
Active weights per token = active parameters × (file size ÷ total parameters). Host memory is the paper figure for the channels fitted: two of DDR5 at about 90GB/s, two of DDR4 at about 50, eight of DDR5 at about 350, and four of the 2014 Xeon’s DDR4 at about 60 at best. Flash-Next’s lookup table is left out of its file size, since a token reads a few rows of it, not the table.
Twelve of thirteen splits land within 1.7× of the napkin, and the misses sort themselves. Where the napkin runs low, a good share of the experts never left the card: the PRO 6000 held 50 to 60% of every model it spilled, and those bytes don’t cross the bus, so the measured speed beats a napkin that assumed they did. Where it runs high, the CPU ran out. Six cores from 2014 got half the napkin with the same bytes in RAM that the 5090 PC handled at 1.1×, and the one server miss, MiMo-V2.5, is also the biggest spill behind the H200. The napkin is a bandwidth ceiling that assumes the cores are free. Read it as the ceiling, and expect to sit at 60 to 110% of it depending on how much of the model stayed on the card and how many cores are doing the rest.
Host bandwidth over active bytes per token. It’s one division, and it was within 1.7× on twelve of thirteen splits.
Files that are smaller than they look
Not every file bigger than the card is a spill. Qwen3.8-Flash-Next is a 111GB download that runs whole on a 96GB RTX PRO 6000, and it isn’t a trick. The model is sold as 125B parameters, but it ships another 55B on top, mostly one big n-gram lookup table: 29GB of the file. llama.cpp treats that table like the token embeddings, an input-layer lookup that stays in system RAM on every machine, Mac included, and never goes to the GPU. The 80GB that does the math fits on the card with room for a 64K context.
So the PRO 6000 ran it at 101 tok/s writing and 2,168 tok/s reading at a 4K prompt, and served eight people at once at 198 tok/s between them. The H200 and the B200, with the whole file on the card, wrote it at 87 and 88. (Our Flash-Next runs on the PRO 6000 were on the 600W Workstation Edition; the rest of this article’s PRO 6000 numbers are the 300W Max-Q.) The same file on a 32GB Radeon is a real spill, 84GB in RAM, and the chart shows what that costs: 11 tok/s.
Check the download size, then check what’s in it.
Datacenter cards for one person
The question we kept getting: how far is a desk from the hardware the labs use? So we gave an H200 (141GB) and a B200 (180GB) the same files and prompts. For one person, both are an RTX PRO 6000: on gpt-oss-120b the H200 writes at 183 tok/s, the B200 at 199, the PRO 6000 at 212. The B200 has 8TB/s of memory bandwidth; writing Qwen3.8-27B for one person, llama.cpp moves about 1.4 of them, and the Mac guide has the chart of how much of its bandwidth each card actually turns into tokens. An H100 is slower than a 5090 for one person (55 vs 78 tok/s on Qwen3.8-27B).
Datacenter cards are built for crowds, and that’s where they leave a desk behind: with 64 people on Qwen3.8-27B at once the B200 served 949 tok/s in total, and 1,646 on gpt-oss-120b. The PRO 6000 with eight people on Qwen3.8-27B managed 186. The big cards also idle hot, 92W for the H200 and 173W for the B200, before anyone asks them anything.
If you’re serving a product, rent one of these. If you’re one person with a big model, a $30,000-plus card won’t write your tokens any faster than the $17,000 one.
The office build
Everything above is one box for one person. Put five people on any of them and the Mac Studio flattens on dense models (Qwen3.8-27B: 65 tok/s in total at four users, 73 at eight), the DGX Spark crawls, and the PRO 6000 runs out of memory for the models worth sharing. The build that fixes this isn’t on anyone’s launch slide, because nobody sells it as a product.
Start with the memory, not the GPU. A current AMD EPYC has twelve DDR5 channels. Twelve 64GB DDR5-6400 sticks give you 768GB at 614GB/s (12 channels × 6,400 MT/s × 8 bytes). That’s half a Mac Studio’s bandwidth with three times the memory, and unlike the Mac, you can put a GPU in front of it. One RTX PRO 6000 gives you 96GB at 1.8TB/s for the parts every token touches (attention, shared layers, the context cache), with 768GB behind it for the experts. GLM-5.3-Flash, MiniMax M3, Qwen3-Coder-480B and the 671B DeepSeeks at 4-bit all fit, with room for context.
Buy the memory channels. The cores come with them.
The napkin above says what it does. DeepSeek V4 Flash drags 7GB through memory per token, and 614 ÷ 7 is 88 tok/s. The 671B DeepSeeks, with 37B active, about 20GB a token, come out at 31; GLM-5.3 at 40B active, around 25. Those are ceilings. On this box nearly every expert lives in RAM, so there is no on-card share to beat the napkin with, and the splits that looked most like that in the table (Flash-Next on the Radeon, MiMo-V2.5 behind the H200) ran at 75 to 90% of theirs. Call it 60 to 80 tok/s for DeepSeek V4 Flash, 20 to 25 for the 671B class, with one person asking. Reading, the part that collapsed on our desktops because eight to twelve CPU cores were doing the expert math, gets 128 Zen 5 cores instead. We haven’t built this one yet; those numbers are napkin, calibrated against the table above.
Three things to know before you spec it. The bandwidth comes from the channels, and only parts with enough chiplets can feed all twelve; the 128-core EPYC 9755 does, the low-core-count bargain parts can’t. The cores are not optional either, as the four-versus-eight test and the 2014 Xeon show; the parts that feed twelve channels bring plenty. And know the ceiling: this is a 300B-to-700B box, not a DeepSeek V4 Pro box. That one is 850GB at 4-bit, and on a server with 1TB of RAM and an H200 it ran at 4.7 tok/s. But MoE is what lets a small company put a 300B model in a closet instead of on a cloud bill, behind a GPU with the compute to serve a room full of people.
One pool doesn’t spill
A Mac and a DGX Spark have nothing to split: CPU and GPU share one memory. When a model fits, nothing collapses. A single Spark holds gpt-oss-120b whole and reads it at 1,577 tok/s, eight times the spilled 5090, while writing at 57. When a model doesn’t fit, it doesn’t load, and there is no trick. That’s the whole reason the 256GB Mac Studio exists as a product.
We ran the big half of the spill chart on an M5 Ultra with 256GB, and it holds every one of those files whole, up to MiMo-V2.5’s 185GB, on the same llama.cpp build:
| Same file, 4K prompt | Mac Studio M5 Ultra, whole | RTX PRO 6000 + desktop RAM | H200 NVL + server RAM |
|---|---|---|---|
| DeepSeek V4 Flash, 155GB | 39 tok/s · reads 778 · 16K in 22 s | 22 · 165 · 96 s | 54 · 347 · 46 s |
| MiMo-V2.6-Flash, 167GB | 69 · 1,072 · 16 s | 18 · 124 · 138 s | 51 · 252 · 76 s |
| MiMo-V2.5, 185GB | 66 · 1,038 · 16 s | 13 · 93 · 157 s | 29 · 157 · 135 s |
Against a desktop spill the Mac writes 1.8 to 5 times as fast and gets through a 16K prompt 4 to 10 times sooner. Against the H200 with eight channels behind it, the card wins the writing only on DeepSeek, which barely spills (17GB in RAM); with 29 and 47GB of MiMo in RAM, the Mac pulls ahead. The Mac reads all three faster than either. What the Mac does with the rest of its 256GB, and where it loses to a PC, is the other half of this story.
The little brother shows the shape. Our 16-inch MacBook Pro M5 Pro has 48GB, of which macOS gives the GPU about 37GB, so gpt-oss-120b’s 63GB file doesn’t load at all. What does load writes fine for chat (gpt-oss-20b at 86 tok/s, Qwen3.6-35B-A3B at 74, the dense Qwen3.8-27B at 16) and reads slowly: a 64K-token prompt is a five-minute wait on the 27B, which rules out agent work on big codebases. Models up to about 35GB load. Above that, a laptop with one pool and a desktop with two are in the same boat, and the laptop has no spill to fall back on.
What we’d build
| You | Build |
|---|---|
| One person, models under 32GB, want the fastest box | An RTX 5090 in a PC with 96GB of RAM. On the fifteen models that fit both cards it wrote 5% faster than the PRO 6000 Max-Q and read 20% faster, which is what 575W does to the same silicon held to 300W. The RAM is for the spill, which keeps gpt-oss-120b usable at 35 tok/s. Don’t expect it to read fast. |
| One person, a 16GB card you already own | Add a second one and 64GB of RAM, and gpt-oss-120b runs. Ours did 12 tok/s behind a six-core Xeon from 2014; the same 35GB in RAM behind a current eight-core desktop CPU and DDR5 got 35 on the 5090 PC, and nothing in the napkin involves the cards. |
| One person, models under 90GB, money no object | RTX PRO 6000 in a real PC. The extra money over a 5090, several times the card’s price, buys the 96GB, not speed, and 96GB is the point. |
| One person, models over 96GB | A 256GB Mac Studio, which held DeepSeek V4 Flash and both MiMos whole at 39-69 tok/s in our runs. Or accept 13-22 tok/s on the PRO 6000 with the experts in RAM. The spill is for the occasional big model, not a daily driver. |
| A team of five to twenty | Two DGX Sparks if the models fit in 128GB each. The EPYC box above if they don’t. |
| A product with real traffic | Rent datacenter cards by the hour and stop reading buying guides. |
How we tested
- Engine: llama.cpp release b11146, the prebuilt binaries for each platform (CUDA for NVIDIA, Vulkan for the Radeon, Metal for the Macs). Whole model on the GPU wherever it fits; otherwise the placement llama.cpp’s fit tool chose for a 19K-token context, with the expert tensors of the last layers in system RAM. GLM-5.3-Flash ran on b11429, the first release that knows its architecture.
- Files: the same GGUF on every machine. gpt-oss-120b is the MXFP4 build (63GB on disk), Qwen3.5-122B-A10B Unsloth’s Q4_K_M (77GB), Qwen3.8-Flash-Next Unsloth’s UD-Q4_K_XL (111GB), DeepSeek V4 Flash and V4 Pro Unsloth’s UD-Q4_K_XL (155GB and 850GB), GLM-5.3-Flash Unsloth’s UD-IQ4_XS (157GB), MiMo-V2.5 Unsloth’s MXFP4_MOE (185GB), MiMo-V2.6-Flash the MXFP4 build (167GB), Qwen3.8-27B Unsloth’s UD-Q4_K_M (16.5GB).
- Prompts: a synthetic log-analysis task at 1K, 4K, 16K, 32K and 64K tokens, 256 tokens generated, fresh cache each time. Three runs per size, medians reported. Split runs stop at 16K, because nobody waits longer than that. Crowds are 1, 2, 4 and 8 simultaneous users with a 1K prompt each; the 64-user B200 figures are from llama.cpp’s batched benchmark on the same files.
- The PCs: the RTX 5090 sits behind a Core i9-14900KF with 96GB of DDR5 on two channels; the Radeon AI PRO R9700 behind a Ryzen 9 5900X with 128GB of DDR4, two channels; the RTX PRO 6000 spill runs and Flash-Next ran on the 600W Workstation Edition behind a Core Ultra 9 285K with 256GB of DDR5, two channels; the rest of the PRO 6000 rows are the 300W Max-Q edition. The two GeForce RTX 5070 Tis (16GB each, the model split across both) sit behind a Xeon E5-2620 v3, a six-core part from 2014, with 64GB of DDR4. llama.cpp got one thread per P-core on the Intel desktop hosts (eight), one per core on the Ryzen (twelve), and its default on the old Xeon. The H200 is the 141GB NVL card in a server with a Xeon Platinum 8592+ and 1TB of DDR5 on eight channels. The Mac Studio is the M5 Ultra with 256GB and the 80-core GPU, with 240GB available to the GPU. The MacBook is the 16-inch M5 Pro with 48GB.
- Power: what the GPU driver reports. That’s the card, not the wall.
Sources
- Our own benchmarks (October-November 2026): llama.cpp on an RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000 Blackwell in both editions, two RTX 5070 Tis, a DGX Spark, an H100, an H200, a B200, a Mac Studio M5 Ultra and a MacBook Pro M5 Pro. Method in “How we tested” above.
- Prices (US, October 2026): street-price tracking for the RTX 5090 and the Radeon AI PRO R9700, NVIDIA’s DGX Spark repricing, and our own check of B&H, Newegg and Micro Center listings for the RTX PRO 6000.
- Hardware Canucks: “Apple M5 Ultra vs The Fastest PC” (YouTube). The 70B spill-over test on a 9950X3D + RTX 5090 PC.
- MacStories: M5 Ultra Mac Studio review (Sept 21, 2026). Their RTX 5090 spill numbers.
- Alex Ziskind: “M5 Ultra: Apple wasn’t messing around” (YouTube, Apple loaner). The Mac Studio’s MLX numbers for DeepSeek V4 Flash.
- Model cards: Qwen3.8-Flash-Next (the 55B of extra parameters and the n-gram table), MiMo-V2.6-Flash (309B total, 15B active), MiMo-V2.5, plus OpenAI’s gpt-oss, Qwen’s and DeepSeek’s cards for the active parameter counts in the napkin table.