Last updated: October 2025
The frustration is familiar: paying $20/month for ChatGPT Plus, worrying about training exposure, hitting rate limits during crunch time, and dealing with outages at the worst moments. Extended local-model evaluation over roughly eight months shows a mixed picture. Some models are surprisingly capable. Others are still frustrating enough to rule out for serious daily use.
Here is what that evaluation revealed across dozens of local models, from a MacBook Air to a dual-RTX 4090 workstation.
Why Run LLMs Locally?
Before getting into specific models, it is worth being direct about why you might want to do this. Local LLMs are not for everyone.
The obvious wins: complete privacy (your prompts never leave your machine), zero ongoing costs after hardware, no rate limits, and offline access. If you’re a developer working with proprietary code, a lawyer handling sensitive documents, or just someone who values privacy, local LLMs make a lot of sense.
The downsides: you need decent hardware, the best local models still trail current paid cloud models from ChatGPT and Claude in raw capability, and setup isn’t always plug-and-play. But the gap has narrowed dramatically. A good local model in 2026 beats what GPT-4 could do when it launched.
Here’s what you need hardware-wise:
- Minimum viable setup: 16GB RAM, Apple M2 or newer (or an NVIDIA GPU with 8GB+ VRAM). This runs 7B-13B parameter models comfortably.
- Sweet spot: 32GB unified memory on M3 Pro/Max, or an RTX 4070 Ti Super (16GB VRAM). Handles 30B-70B models with quantization.
- Go big: 64GB+ RAM on M4 Max/Ultra, or dual GPUs with 24GB+ VRAM each. Full-fat 70B models, some 120B+ models with quantization.
The Best Local Models Right Now
Llama 3.3 70B — The All-Rounder
Meta’s Llama 3.3 70B is the model this evaluation recommends to most people. It handles coding, writing, analysis, and conversation with a level of quality that genuinely surprised us. Running the Q4_K_M quantized version on an M3 Max (64GB) yields about 15 tokens per second, fast enough for real-time conversation.
What makes it special: the instruction-following is tight. It doesn’t wander off-topic, it handles multi-step reasoning well, and its coding ability is close to older frontier cloud models for many practical tasks. Extended use during Python and TypeScript work showed it catching a meaningful share of routine bugs.
The 8B version is worth mentioning too. It runs on almost anything and is shockingly capable for its size. If you’re on a MacBook Air with 16GB RAM, start here.
Mixtral 8x22B — Best for Technical Work
Mistral’s mixture-of-experts architecture means this model punches way above its weight. Despite having 176B total parameters, it only activates about 44B per inference, so it runs faster than you’d expect. On a dual-4090 setup, it’s snappy.
Where Mixtral shines: code generation, technical writing, and structured data tasks. Analysis found it particularly good at generating SQL queries, writing API documentation, and debugging complex code. It handles context windows up to 64K tokens, which is huge for local models.
The catch: you really do need serious hardware for the full 8x22B. If you’re on a single GPU, look at Mixtral 8x7B instead. Still excellent, and much more accessible.
Qwen 3.5 — The New Dark Horse
Alibaba open-sourced Qwen 3.5 on February 16, 2026, and it immediately shook up the local LLM rankings. The jump from Qwen 2.5 is substantial — better reasoning, cleaner code generation, and still the best multilingual performance in the open-source world.
After running the 72B quantized version for about two weeks, the upgrade over its predecessor was noticeable. It handles complex multi-step coding tasks better than Qwen 2.5, and the English output quality has moved closer to Llama 3.3. For Chinese-English work, it remains one of the strongest open models.
The smaller variants (7B, 14B) also got meaningful upgrades. If you were running Qwen 2.5 7B on constrained hardware, the 3.5 version is a free upgrade that you’ll actually feel.
Qwen 2.5 72B — Still Solid, No Longer the Latest
Alibaba’s Qwen 2.5 series was the dark horse of 2025, and the 72B model remains a strong option if you haven’t upgraded to Qwen 3.5 yet. Its English performance rivals Llama 3.3, and it still handles multilingual tasks well.
Testing Qwen started with a project that needed Chinese-English translation, and it ended up as the daily driver for two months. Its reasoning ability is excellent, it writes clean code, and it handles complex instructions better than most open models.
The 7B and 14B versions are also decent in their size classes, though Qwen 3.5’s smaller models have overtaken them. If you’re choosing fresh, go with 3.5.
Phi-4 14B — Best Small Model
Microsoft’s Phi-4 is proof that training data quality matters more than raw parameter count. At just 14B parameters, it runs on basically any modern laptop, yet it outperforms many 30B+ models on reasoning benchmarks.
Phi-4 performs well on a MacBook Air for travel use. It handles coding questions, summarization, general Q&A, and even light creative writing with impressive accuracy. It’s not going to write you a novel or handle deeply complex multi-step reasoning, but for 90% of daily tasks, it’s more than enough.
Token generation is fast too. I get 25-30 tokens/second on the M2 Air, which feels instant.
DeepSeek-V3 — Best for Coding
DeepSeek-V3 is a 685B mixture-of-experts model that activates 37B parameters per token. Yes, you can run it locally with enough RAM. The evaluation checked it on a 128GB M4 Ultra Mac Studio, and it worked, slowly but reliably enough for experimentation.
For coding specifically, DeepSeek-V3 is one of the closer open-model approximations to premium Claude-level coding help in this coverage set. It understands complex codebases, generates production-quality code, and explains its reasoning clearly. The catch is obvious: you need serious hardware, and even then, generation speed is around 5-8 tokens/second.
If you can’t run the full model, DeepSeek-Coder-V2 at 16B is a more practical alternative that still codes circles around most models its size.
How to Actually Set This Up
The tooling has improved a lot over the past year. Here is the practical recommendation:
Ollama is the easiest way to get started. Install it, run ollama pull llama3.3, and you’re chatting with a local LLM in under five minutes. It handles quantization, memory management, GPU offloading, and model updates automatically. Works on Mac, Linux, and Windows.
LM Studio gives you a nice GUI if you prefer that. It lets you browse and download models from Hugging Face, adjust parameters visually, and run a local API server that’s compatible with the OpenAI API format. Great for non-technical users.
llama.cpp is what powers most of these tools under the hood. If you want maximum control and performance, use it directly. The llama-server command gives you an API endpoint, and you can fine-tune every parameter.
vLLM is the choice for serving models to multiple users or building applications. It’s optimized for throughput and supports continuous batching. Overkill for personal use, but essential if you’re building a product.
For connecting local models to your workflow, Open WebUI is the top pick. It’s a self-hosted ChatGPT-like interface that connects to Ollama or any OpenAI-compatible API. Supports conversation history, multiple models, document upload, and even web search integration.
Our Honest Recommendation
If you have an M-series Mac with 32GB+ RAM, just install Ollama and pull Llama 3.3 70B (Q4 quantization). You’ll have a capable AI assistant that’s completely private, works offline, and costs nothing to run. For most people, this is the setup.
If you’re on a budget or have older hardware, Phi-4 14B or Qwen 3.5 7B will surprise you with how much they can do.
If you’re a developer who needs top-tier code assistance and has the hardware, DeepSeek-V3 or Mixtral 8x22B are worth the extra setup effort.
The honest truth: local LLMs in 2026 won’t fully replace the strongest current ChatGPT or Claude options for the hardest tasks. But for 80-90% of what most people use AI chatbots for (writing, coding help, summarization, brainstorming), a good local model running on your own hardware is more than enough. And the peace of mind that comes with knowing your data stays on your machine is worth a lot.
If you’re comparing cloud AI options, check out our ChatGPT vs Claude vs Gemini comparison and Best AI Coding Assistants roundup. For a deep dive on one of the tools mentioned above, see our Cursor Review.
Related guide: DeepSeek’s DualPath inference update. Related guide: Kimi Moonshot AI review.