How to Spec an AI Workstation for Local LLMs in Canada

Building an AI workstation for local LLMs is a capacity-planning problem before it is anything else. The usable model size is decided by GPU VRAM first, then system RAM, then storage. Get that ordering wrong and you end up with a fast CPU staring at a model it cannot load.

We see this mistake constantly. A client in Toronto came to us last quarter wanting a top-tier Ryzen chip and 128GB of RAM to run a 70B model, but had budgeted for a 12GB GPU. The CPU was never the bottleneck. Once the model overflows VRAM and starts leaning on system memory, generation speed collapses regardless of how fast the processor is.

If you are spending your first dollar on the CPU instead of the GPU for local inference, you are optimizing the wrong part. That single decision determines which models you can actually run.

VRAM Sets the Model Ceiling

Every major hardware guide converges on roughly the same VRAM tiers for local inference, and they line up with what we see in real builds. The numbers shift with quantization and context length, so treat these as planning ranges rather than exact thresholds.

The real cost is not just the weights. VRAM also has to hold the KV cache and runtime overhead, and both grow with context length. A model that technically "fits" at short context can spill once you push the window out, which is why the 24GB tier keeps coming up as the practical sweet spot.

For most people building a serious single-GPU setup, 24GB is where flexibility lives. It runs 30B - 34B models comfortably at Q4 without heavy offload, and it leaves room for larger context windows. The jump from 12 - 16GB to 24GB is what opens that middle tier without forcing you into slower workarounds.

In Canada, the used market makes this interesting. A used RTX 3090 with 24GB has been floating around the CAD 980 range according to Canada-focused build guides, which is aggressive value for that much memory. The catch is warranty. A used 3090 that ran a mining or training load hard for two years carries real risk, and there is no manufacturer coverage backing it. We source from verified suppliers for exactly this reason when specing an AI workstation build, because a dead GPU on a production machine is a much bigger cost than the savings.

RAM Sets Usability

System RAM is the second lever, and the pattern is consistent. 16GB works for 7B - 8B experimentation. 32GB is the comfortable mainstream baseline for 14B - 32B work. 64GB is where you go for 70B models, CPU offload, or running several tools at once.

Corsair's current guidance mirrors this: 16GB for 7B - 8B, 32GB for 14B - 32B, 64GB targeting 70B at Q4 or Q5. Puget Systems adds a useful rule we follow in practice - keep at least as much RAM as VRAM, and preferably 1.5 to 2 times more. So a 24GB card pairs best with 32 - 64GB of system memory, not 16GB.

Pro Tip: If you plan to use CPU offload to squeeze a 70B model onto a 24GB card, RAM stops being optional headroom and becomes the thing keeping the model running at all. Budget for 64GB minimum in that scenario, and accept that generation speed will drop noticeably compared to a full-GPU fit.

That offload tradeoff is worth being honest about. Yes, a large enough pool of system RAM lets a 70B model run on hardware that cannot hold it entirely in VRAM. But the responsiveness is not the same. Tokens come slower, interaction feels laggier, and for anything conversational that gap is obvious. Offload is a fallback, not a substitute for VRAM.

Storage Sets Workflow Comfort

Storage is the least dramatic of the three, and also the most commonly under-sized. There is no hard model-loading limit tied to disk capacity the way there is with VRAM. One guide cites 50GB free as a bare floor, which is fine for a single downloaded model and nothing else.

In practice, that floor is a trap. Local workflows accumulate multiple quantizations of the same model, toolchains, caches, and often datasets or dev environments living on the same machine. A 1 - 2TB NVMe SSD is the sane workstation choice. We have watched builds hit space pressure within weeks because someone tried to run everything off a 500GB boot drive.

NVMe speed matters for model load times, less so for ongoing token generation once the model is resident in memory. If you are frequently swapping between large models, faster storage is a genuine quality-of-life gain. If you load one model and leave it running, the difference is minor.

Matching the Build to Your Model Class

The single most important thing to settle before buying anything is which tier you actually need. The hardware jumps between them are large and expensive, so guessing wrong costs real money.

1. 7B - 8B experimentation: 8 - 12GB VRAM, 16GB RAM, 1TB SSD. Entry-level and forgiving.
2. 14B - 32B productive use: 16 - 24GB VRAM, 32GB RAM, 1 - 2TB SSD. The most common serious setup.
3. 70B-class local inference: 24GB VRAM minimum with offload, or multi-GPU for a full fit, 64GB RAM, 2TB SSD.

Apple Silicon is a legitimate wildcard here. A Mac Studio M4 Max with 36GB of unified memory runs around CAD 3,360 new and its shared memory pool sidesteps the VRAM ceiling entirely for certain workloads. The tradeoff is CUDA. Most local-LLM tooling is built around NVIDIA's ecosystem, and while support for Apple Silicon has improved, you will hit tooling that assumes CUDA. If your workflow depends on specific frameworks, verify compatibility before committing to a Mac.

None of these tiers are rigid. Quantization format, model family, and context length all shift the actual VRAM you need, and most published figures come from vendor guides rather than controlled lab benchmarks. Treat the ranges as a starting point and expect to tune once you know your exact models.

Frequently Asked Questions

How much VRAM do I need to run a 70B model locally?

Around 35 - 48GB for a full GPU fit, or 24GB with CPU offload. The offload route works but generation speed drops noticeably, so for responsive use you either want a 48GB card or a multi-GPU setup.

Is a used RTX 3090 worth it for local LLMs in Canada?

For the VRAM per dollar, yes - 24GB around CAD 980 is hard to beat. The risk is condition and zero warranty, since many were run hard on mining or training loads, so buy from a reputable seller and factor in that there is no coverage if it fails.

Do I need a fast CPU for local inference?

No, not as a priority. VRAM and RAM decide what runs and how well; the CPU only becomes relevant during CPU offload, and even then RAM capacity matters more than clock speed.

Where most people go sideways is spending on the wrong component in the wrong order. If you are trying to figure out whether a used 24GB card, a newer lower-VRAM GPU, or an Apple Silicon machine fits your specific model class and budget, that decision is easier with someone who has built these systems and knows the current Canadian market. Our team specs and assembles professional workstations around real workloads, not spec-sheet guesses.

Explore OrdinaryAI to see how we configure machines for local LLMs and machine learning, or reach out and we will size a build around the exact models you plan to run.

Explore More at OrdinaryTech

• See how businesses have deployed our systems in our client success stories.
• Talk through your requirements with a free consultation.

Written by Sadip Rahman, Founder & Chief Architect at OrdinaryTech - a Toronto-based custom PC company that has built over 5,000 systems for gamers, creators, and businesses across Canada.