How to Run LLMs Locally with Ollama (Step-by-Step)

Ollama is the easiest way to run large language models on your own hardware. One install, one command to download a model, and you have a local AI assistant with no subscription fee, no rate limits, and no data sent to any external server.

This guide covers installation on Mac, Windows, and Linux, recommended models for different hardware configurations, and how to use Ollama from the command line and from a browser interface.

What you need

Minimum:

• Mac with Apple Silicon (M1 or newer), or Windows/Linux PC with 8GB RAM
• Enough disk space for the model you want to run (7B models: ~4GB, 14B models: ~8GB, 32B models: ~20GB)

Recommended for better performance:

• Mac with 16GB+ unified memory, or
• PC with a dedicated NVIDIA GPU with 8GB+ VRAM

Ollama runs CPU inference on any machine, but a GPU or Apple Silicon chip gives dramatically faster response times.

Step 1: Install Ollama

Mac: Download the installer from ollama.com and drag it to your Applications folder. Or install via Homebrew:

brew install ollama

Windows: Download the installer from ollama.com. It installs as a background service that starts automatically.

Linux:

curl -fsSL https://ollama.com/install.sh | sh

After installation, confirm it works:

ollama --version

Step 2: Download and run a model

Pull and run a model in one command:

ollama run llama4

This downloads Llama 4 Scout (the default, ~7GB) and opens an interactive chat session. Type your question and press Enter.

To exit the chat, type /bye.

Choosing the right model for your hardware

8GB RAM (CPU only):

• llama4 (Llama 4 Scout, quantized): Fast enough for most tasks on Apple Silicon
• phi4-mini: Microsoft's 3.8B model, very fast on CPU
• gemma3:4b: Google's 4B model, good for code tasks

16GB RAM:

• llama4 full quality: Better output than the smaller quantized version
• mistral-small: Strong general-purpose model
• qwen2.5:14b: Excellent for coding and reasoning

24GB+ RAM or 8GB+ GPU:

• qwen2.5:32b: Near-frontier quality for many tasks
• llama4:scout with higher quality settings

GPU with 16GB+ VRAM:

• qwen2.5:72b: Frontier-quality reasoning model
• llama4:maverick: Llama 4's larger, higher-capability variant

To see all available models: visit ollama.com/library or run ollama list after downloading some models.

Step 3: Useful commands

List downloaded models:

ollama list

Run a specific model:

ollama run qwen2.5:14b

Delete a model:

ollama rm llama4

Pull a model without running it:

ollama pull mistral-small

Step 4: Use the Ollama API

Ollama runs a local server on port 11434. You can send requests to it from any application:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama4",
    "prompt": "Explain the difference between TCP and UDP in one paragraph",
    "stream": false
  }'

The API is compatible with OpenAI's API format for chat completions, which means any application that supports OpenAI can be configured to use Ollama instead:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama4",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Step 5: Add a browser interface

The command-line chat is functional but not comfortable for longer conversations. Open WebUI gives you a full ChatGPT-style interface that connects to your local Ollama server.

Install Open WebUI with Docker:

docker run -d -p 3000:8080 \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Open http://localhost:3000 in your browser. Create an account (local only, no external server) and start chatting.

Step 6: Connect to your code editor

To use local Ollama models for code completion in VS Code:

1. Install the Continue extension in VS Code
2. Open Continue settings (⌘/Ctrl + , then search "Continue")
3. Add Ollama as a provider with http://localhost:11434
4. Select your preferred model (qwen2.5:14b works well for code)

Continue will use Ollama for all AI features: completions, chat, and editing.

For Aider (terminal-based coding agent):

aider --model ollama/qwen2.5:14b

Benchmark expectations

Running qwen2.5:14b on a Mac M2 with 16GB memory gives roughly 15-25 tokens per second, which feels responsive for chat. A dedicated NVIDIA RTX 4080 (16GB) runs the same model at 60-80 tokens per second, which is fast.

For perspective: the old GPT-4 API at 10 tokens per second felt slow to most users. At 25 tokens per second locally, the experience is comparable to cloud API response times for most tasks.

When local models are not the right choice

Local Ollama is excellent for privacy-sensitive work, development without internet access, and cost-sensitive high-volume use. It is not the right choice when:

• You need the absolute latest frontier capabilities (Llama 4 Maverick is strong but Claude Opus 4.6 and GPT-5 are stronger on complex reasoning)
• You need multimodal image input on a budget system (image models require more VRAM)
• Your machine is slower than an Apple Silicon Mac and you want fast responses

For most everyday tasks including coding, writing, summarization, and question answering, a well-chosen local model on capable hardware matches cloud model quality in 2026.