Google’s own documentation says the 31B version of Gemma 4 needs about 69.9GB of memory at full 16-bit precision and about 17.5GB at 4-bit. That one table explains why so many people buy the wrong machine for local AI. They pick a PC by its processor or its gaming benchmarks, then find the model they wanted does not fit on the graphics card.
A local AI machine follows different rules from a gaming rig. Memory comes first, then memory speed, then power, cooling and software.
You can apply the checks to a self-built tower, a branded workstation or a prebuilt system. Some sellers now group machines by the models they can run. One handy reference is the range of AI workstations at CryptoGPUs, where each listing names the model sizes it targets, from small unified-memory boxes up to dual RTX PRO 6000 towers. Every specification below comes from the maker’s own page, verified on October 25, 2026.
1. Check How Much Memory Your Models Need
Start with the models, not the hardware. Google’s Gemma 4 table shows how quickly needs change with precision. The 12B model needs about 26.7GB at 16-bit, 13.4GB at 8-bit and 6.7GB at 4-bit. The 31B model needs 69.9GB, 34.9GB and 17.5GB.
Two details on that page are easy to miss. First, the figures cover only the model weights; Google notes they leave out supporting software and the context window, which grows as your prompts and answers get longer. Second, the 26B A4B model is built from many experts and activates only about 4 billion parameters per step, yet all 26 billion must sit in memory. Its 4-bit figure is 14.4GB, much closer to a dense 26B model than to a 4B one.
Meta’s numbers tell the same story at a larger scale. Llama 4 Scout has 17 billion active parameters but 109 billion in total, and Meta says it fits on a single NVIDIA H100 GPU only with Int4 quantization. Note the biggest model you intend to use, its precision and a margin for long prompts. That total is your memory target.
2. Compare GPU Memory and Bandwidth, Not Just Speed
After capacity, bandwidth largely decides how quickly a model answers, because the card reads the weights over and over while generating text. NVIDIA’s comparison page lists these GeForce RTX 50 figures:
- RTX 5090: 32GB, 1,792 GB/s (512-bit bus).
- RTX 5080: 16GB, 960 GB/s.
- RTX 5070 Ti: 16GB, 896 GB/s.
- RTX 5070: 12GB, 672 GB/s.
- RTX 5060 Ti: 16GB or 8GB, 448 GB/s.
A 16GB RTX 5060 Ti holds as much as an RTX 5070 Ti, but the pricier card moves data twice as fast. If your target from step one is above 32GB, look at NVIDIA’s RTX PRO Blackwell cards instead. The RTX PRO 4000 has 24GB, the RTX PRO 4500 has 32GB, the RTX PRO 5000 comes with 48GB or 72GB, the RTX PRO 5500 has 84GB, and the RTX PRO 6000 has 96GB. All of them use GDDR7 with ECC, which corrects memory errors during long jobs.
3. Decide Whether a Compact Unified-Memory Box Is Enough
Not every local AI machine needs a full tower. DGX Spark, NVIDIA’s desktop unit built around the GB10 superchip, gives the CPU and GPU 128GB of shared LPDDR5x, and a 64GB version is sold only through partner manufacturers. It is 150 mm square, 50.5 mm tall and weighs 1.2 kg. NVIDIA says the unit can handle inference for models as large as 200 billion parameters, and its ConnectX networking can join up to four units.
The trade-off is speed. NVIDIA lists 273 GB/s of memory bandwidth, a fraction of an RTX 5090, so large models load easily but answer more slowly. A box like this suits people who want to test big models quietly on a desk. A GPU tower suits people who need fast replies for coding, chat or batch jobs all day.
4. Plan for Power and Cooling
AI workloads can keep a graphics card near full load for hours, so power and heat need more headroom than a gaming build. NVIDIA’s required system power figures are a sensible minimum:
- RTX 5090: 1,000W, fed by four 8-pin cables through the included adapter or one 600W PCIe Gen 5 cable.
- RTX 5080: 850W.
- RTX 5070 Ti: 750W.
- RTX 5070: 650W.
Cooling choices matter most in multi-GPU builds. NVIDIA fits the standard RTX PRO 6000 Workstation Edition with a double flow-through cooler and rates it at up to 600W. The RTX PRO 6000 Max-Q Workstation Edition keeps the same 96GB but is rated at 300W, which makes two cards in one case far easier to cool and power. Check that the case has clear front intake, that the power supply has enough native high-power connectors, and that the room can shed the heat.
5. Confirm the Operating System and Driver Stack
Software problems waste more weekends than hardware ones. On Linux, NVIDIA says Blackwell GPUs require its open-source GPU kernel modules, which became the default from the R560 driver branch. NVIDIA does not support its proprietary modules on these platforms, so older guides that install them are out of date for an RTX 50 or RTX PRO Blackwell card.
On Windows, the usual route is WSL 2. NVIDIA’s CUDA on WSL guide says to install only the normal Windows display driver and not to install any Linux GPU driver inside WSL. NVIDIA lists RTX 50 cards with CUDA compute capability 12.0, so check that the release notes of your AI framework mention Blackwell support before you set up a project. DGX Spark avoids most of this by shipping with DGX OS, NVIDIA’s Ubuntu-based Linux.
To avoid the setup entirely, ask for the software at order time. CryptoGPUs, for instance, will preload Ubuntu 24.04 plus CUDA or ROCm and tools such as Ollama and Open WebUI on its in-house builds, or install Windows instead, and covers parts for three years.
A Final Pre-Purchase Checklist
- Largest model and precision written down, with memory to spare for context.
- GPU or unified memory that holds it, with bandwidth that suits how fast you need answers.
- Power supply at or above NVIDIA’s required system power, with the right connectors.
- Case airflow planned for hours of full load, especially with two cards.
- Operating system, driver and framework versions confirmed for Blackwell.
Work through the checks in order and the shortlist usually shrinks to two or three machines. Price, noise and size decide the rest.
