Top 10 Local LLMs for Self-Hosting
Want the best API models from the major labs? See Top 10 Cloud & API LLMs.
Local LLMs have matured enough that on-device and on-premises inference is genuinely practical. Open-weight families — Llama, Qwen, Gemma, DeepSeek, and Mistral among them — ship reliable specifications, long context windows, and first-class support in runners like Ollama and LM Studio. This guide compares ten of the most deployable options, weighted toward what matters for self-hosting: license clarity, GGUF availability, context length, parameter count, and the VRAM you need to run them.
At a glance
| Model Family | Primary Size | Context Window | Best Use Case |
|---|---|---|---|
| Llama 3.1 | 8B / 70B | 128K | High-precision RAG |
| Qwen3 | 14B / 32B | 32K+ | Agentic workflows |
| DeepSeek R1 | 7B / 32B | 128K | Logic & programming |
Match the model to your VRAM
Check your GPU’s video RAM before downloading. A model too large for your VRAM spills onto the CPU, and speed drops from conversational to unusable.
| Tier | Example Models | VRAM | Typical Hardware |
|---|---|---|---|
| Entry | Llama 3.2 3B | 4–8 GB | MacBooks, laptops, basic automation |
| Prosumer | Llama 3.1 8B / Qwen3 14B | 12–16 GB | RTX 3060/4070 or better — the sweet spot for local RAG |
| Workstation | Mixtral / Qwen 32B | 24 GB+ | RTX 3090/4090 or Mac Studio — near-frontier logic |
The ten models
1. Meta Llama 3.1-8B
The people’s champion. Its strength is less raw intelligence than its 128K context window: for RAG you can feed it hundreds of pages without it losing the thread. Extremely stable, and it works with every runner (Ollama, LM Studio, vLLM) out of the box.
2. Meta Llama 3.2-1B / 3B
Where 3.1 is for power, 3.2 is for efficiency — edge models. For a lightweight helper on a phone or a low-powered server, the 3B is surprisingly capable at summarization and intent classification.
3. Alibaba Qwen3-14B / 32B
Arguably the most versatile model here. It consistently outperforms Llama at tool-calling — invoking external APIs and databases. For agent workloads that need the model to do things rather than just talk, Qwen is often the better pick.
4. DeepSeek R1 (Distill Versions)
DeepSeek R1 popularized “reasoning” models that think through problems with an explicit Chain-of-Thought. The distilled 7B and 32B versions are excellent at complex logic, math, and coding — reach for these when a local agent needs to debug code or work through calculations from raw data.
5. Google Gemma 2-9B / 27B
Gemma 2 uses sliding-window attention and knowledge distillation from Google’s Gemini models, and it feels more creative than Llama. For marketing copy or product descriptions, Gemma 2-9B often produces more human-sounding prose.
6. Mixtral 8x7B (MoE)
A Mixture-of-Experts model: 47B total parameters but only ~13B active per token. That buys the speed of a small model with the knowledge of a large one. It needs at least 24 GB of VRAM (RTX 3090/4090), making it workstation-class.
7. Microsoft Phi-4-mini-3.8B
Proof that data quality beats data quantity — despite its size, Phi-4-mini beats models twice as large on reasoning benchmarks. Ideal for background tasks like auto-tagging catalog items or sentiment analysis on reviews.
8. Microsoft Phi-4-Reasoning-14B
The larger sibling built for deep reasoning — less chatbot, more logic engine. For workflows that analyze complex legal documents or technical specs, Phi-4-Reasoning is exceptionally reliable.
9. Yi-1.5-9B / 34B
A strong choice for bilingual applications. Yi offers superior performance in Chinese and other East Asian languages compared with Western-centric models.
10. InternLM 2.5-7B / 20B
A researcher favorite, highly optimized for structured data extraction. For turning messy HTML into clean JSON, InternLM’s chat-base variants are among the best at following strict formatting instructions.
Which one to pick
A practical two-tiered approach:
- Development and prototyping: start with Llama 3.1-8B — the most documented, with the widest runner support.
- Production reasoning: once the logic is sound, move to DeepSeek R1 (32B distill) or Qwen3-32B for the extra edge in decision-making and tool use.
