Ollama Complete Guide: Install, Models, API and Security

by David Porter

Wednesday, 26 August 2026 at 23:00

thumbnail_ollama-complete-guide-install-

This Ollama guide covers a model runner for Windows, macOS and Linux. Ollama downloads compatible model packages, starts local inference, exposes a command-line interface and automatically serves an HTTP API at http://localhost:11434.

That combination makes Ollama useful for three groups:

Beginners who want a short command for a local chat.
Developers who want to connect an application to a local model.
Teams experimenting with private assistants, document search or local AI agents.

The quick start is:

ollama run qwen3.5:4b

The command downloads the selected model if needed, loads it and opens an interactive chat. Yet a production-worthy setup needs more thought: model size, context length, storage, updates, licensing, API exposure and tool permissions all matter.

This article is part of AI World Today’s Local AI hub. Beginners who have never opened a terminal may prefer to start with how to run AI locally.

What does Ollama do?

Ollama combines several jobs that otherwise require separate setup:

It pulls a model package from a registry.
It stores the model locally.
It selects available CPU or GPU acceleration.
It applies the correct chat template.
It serves chat, generation, embedding and model-management endpoints.
It exposes command-line controls for running and maintaining models.

Ollama can run downloaded models locally, but it also supports cloud-tagged models and optional web services. “Running Ollama” therefore does not prove that every workload is offline. Check the model tag and disable cloud features when a strictly local environment is required.

Ollama is one route among several. LM Studio offers a graphical interface; llama.cpp offers lower-level control. Our local AI versus cloud AI guide explains the architectural choice, while the existing LM Studio tutorial covers that alternative.

Ollama installation

Use the official Ollama download page rather than third-party installers.

Install Ollama on Windows

Ollama runs as a native Windows application and makes its CLI available in PowerShell, Command Prompt and other terminals. Current Windows requirements specify Windows 10 22H2 or newer.

Download and run OllamaSetup.exe.
Open PowerShell.
Check the installation:

ollama -v

The normal installer does not require administrator rights. Ollama’s application and model files occupy different locations, and downloaded models can consume tens or hundreds of gigabytes.

Install Ollama on macOS

Current macOS documentation requires macOS Sonoma 14 or newer. Apple M-series devices receive CPU and GPU acceleration; x86 Macs use CPU execution.

Download the disk image.
Move Ollama into Applications.
Start the app.
Open Terminal and run:

ollama -v

Install Ollama on Linux

The official Linux installation guide publishes:

curl -fsSL https://ollama.com/install.sh | sh

Review remote installation scripts before granting shell access when your security policy requires it. Ollama also documents manual packages for x86-64, ARM64 and AMD ROCm.

Start and inspect the service:

sudo systemctl start ollama sudo systemctl status ollama

For a manual foreground process:

ollama serve

Essential Ollama commands

Ollama’s current CLI reference uses these commands:

Task	Command
Open the interactive menu	ollama
Run or chat with a model	ollama run qwen3.5:4b
Download or update a model	ollama pull qwen3.5:4b
List downloaded models	ollama ls
List loaded models	ollama ps
Stop a loaded model	ollama stop qwen3.5:4b
Remove a downloaded model	ollama rm qwen3.5:4b
Start the Ollama server	ollama serve
Launch a supported integration	ollama launch
Sign in for Ollama cloud functions	ollama signin

ollama pull is useful before reproducing a test because model tags can be updated. For stable business deployments, record the exact package and version information rather than assuming latest will remain unchanged.

Removing a model deletes the local copy. It does not uninstall Ollama itself.

How to choose an Ollama model

The right model is the smallest one that performs the task reliably under your latency and quality requirements. A larger parameter count can improve capability, but it also increases storage, memory use and often response time.

Examples from Ollama’s library on 21 August 2026 include:

Model command	Listed package size	Listed maximum context	Notes
ollama run llama3.2:1b	1.3 GB	128K	Small text model for limited hardware
ollama run qwen3.5:4b	3.4 GB	256K	Compact multimodal model
ollama run deepseek-r1:8b	5.2 GB	128K	Distilled reasoning model
ollama run ministral-3:8b	6.0 GB	256K	Multilingual, vision and tools
ollama run gemma4:12b	7.6 GB	256K	Multimodal workstation step-up
ollama run gpt-oss:20b	14 GB	128K	Open-weight reasoning model
ollama run deepseek-r1:70b	43 GB	128K	Large quantized model for high-memory systems

Package size is not total memory consumption. Runtime also needs a context cache, compute buffers and memory for the operating system. The complete local AI hardware guide explains RAM, VRAM, quantization and model size. The best local AI models guide focuses on capability and license.

Also check what a tag actually represents. deepseek-r1:8b is a distilled 8B model, while deepseek-r1:671b is the full model and a 404 GB package in Ollama’s library. A shared family name does not imply similar hardware or identical behavior. See our DeepSeek model guide and guide to running DeepSeek locally for that family.

Parameter count, active parameters and package size

Three numbers are often mixed together:

Total parameters: all learned values in the model.
Active parameters: values used for a token in a sparse mixture-of-experts model.
Package size: the downloaded representation after its data type and quantization are chosen.

For example, Ollama lists Gemma 4 26B as a 25.2B-total-parameter mixture-of-experts model with about 3.8B active parameters per token. The lower active count can reduce computation. It does not turn the stored 25.2B model into a 3.8B download or guarantee 3.8B-style memory use, because the expert weights still exist.

Quantization stores weights at lower precision. The llama.cpp project supports integer quantization from 1.5 to 8 bits to reduce memory use and accelerate inference on supported hardware. Lower precision can trade some quality for a smaller footprint.

Context length in Ollama

The maximum context shown beside a model is an architectural limit. Ollama’s allocated context is a runtime setting and may be much smaller.

According to Ollama’s context documentation, its current defaults are:

Available VRAM	Default context allocation
Below 24 GiB	4,096 tokens
24–48 GiB	32,768 tokens
48 GiB or more	262,144 tokens

Ollama recommends at least 64,000 tokens for tasks such as web search, agents and coding tools. That is a workload recommendation, not evidence that every computer can allocate it comfortably.

Set a server-wide value when starting Ollama:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Then inspect the result:

ollama ps

Example fields include model name, loaded size, processor split, context and unload time. Increasing context raises memory use. For ordinary chat, a smaller value can be faster and safer. For document work, use retrieval to send relevant passages rather than placing an entire archive into every prompt; our local RAG guide explains that pattern.

Use the Ollama API with curl

Ollama’s native API base URL is:

http://localhost:11434/api

After downloading a model, send a non-streaming generation request:

curl http://localhost:11434/api/generate \ -d '{ "model": "qwen3.5:4b", "prompt": "Explain quantization in three sentences.", "stream": false }'

Use the chat endpoint for messages with roles:

curl http://localhost:11434/api/chat \ -d '{ "model": "qwen3.5:4b", "messages": [ {"role": "system", "content": "Answer in concise plain English."}, {"role": "user", "content": "What is GPU offloading?"} ], "stream": false }'

Omit "stream": false to receive the default streaming response. The official API documentation also covers embeddings and model-management endpoints.

Use Ollama from Python

Ollama maintains an official Python library. Install it in a virtual environment:

python -m pip install ollama

Then call the local model:

from ollama import chat response = chat( model="qwen3.5:4b", messages=[ {"role": "user", "content": "Give me a four-step backup checklist."} ], ) print(response.message.content)

The Ollama service must be running. Download the selected model with ollama pull before calling it from an application.

OpenAI-compatible endpoints

Ollama supports parts of the OpenAI API at:

http://localhost:11434/v1/

That lets some existing applications change their base URL and use a local model. The official compatibility guide shows the Python client with api_key='ollama'. The client library requires a value, but Ollama ignores it for this local endpoint.

That detail is security-critical: a dummy API key in application configuration does not add authentication to the local Ollama server.

Ollama API security: do not expose port 11434 directly

Ollama’s authentication page states that its local API at http://localhost:11434 requires no authentication. By default, Ollama binds to 127.0.0.1:11434, limiting ordinary access to the same computer.

Keep that default unless you are deliberately administering a network service.

Avoid these shortcuts:

Setting OLLAMA_HOST=0.0.0.0:11434 on an untrusted network.
Forwarding router port 11434 to the internet.
Publishing the port from a cloud VM without firewall restrictions.
Assuming an api_key field in an OpenAI client secures Ollama.
Opening broad browser origins without understanding cross-origin access.

If remote access is required, place Ollama behind an authenticated reverse proxy or private VPN, enable TLS, restrict source networks, apply host firewall rules, log access and rate-limit requests. Keep Ollama and the proxy updated. Separate the model server from tools that can modify files or execute code.

An exposed text model can consume memory and compute. An exposed agent endpoint can be worse because model output may trigger actions. Review AI agent security and design explicit trust boundaries using our AI agent architecture guide. Do not give an internet-facing model unrestricted shell, browser, email or filesystem permissions.

Make Ollama local-only

Ollama states that it does not see prompts and responses for locally run models. Cloud models and optional cloud features create a different data path.

To disable cloud functions, add this to ~/.ollama/server.json:

{ "disable_ollama_cloud": true }

Or set:

OLLAMA_NO_CLOUD=1

Restart Ollama. Its logs should report that Ollama cloud is disabled. This removes Ollama cloud models and web search; it does not block every other program or tool on the machine from reaching the internet.

For sensitive workflows, combine local-only mode with device encryption, user access controls, trusted model sources, patching, backups and a review of logs. See is local AI private and safe? and AI World Today’s broader AI Security coverage.

Where Ollama stores models

Ollama documents these default model paths:

Platform	Default model location
macOS	~/.ollama/models
Linux standard installer	/usr/share/ollama/.ollama/models
Windows	C:\Users\%username%\.ollama\models

Set OLLAMA_MODELS to use another drive. On Linux, the Ollama service user needs read and write permissions for that directory.

Do not treat the model directory like an ordinary folder of documents. Ollama manages manifests and blobs, and multiple model tags may share data. Use ollama rm for normal cleanup.

GPU support and offloading

Ollama’s hardware support page lists current backends and requirements:

Nvidia GPUs through CUDA, with supported compute capability and drivers.
Selected AMD GPUs through ROCm.
Apple GPUs through Metal.
Additional Windows and Linux GPUs through Vulkan.

Run ollama ps after loading a model. A CPU/GPU split shows partial offloading. Full GPU placement is generally faster, but a model can still run partly or entirely in system RAM.

For buying decisions, compare our guides to Nvidia AI GPUs, Nvidia CUDA and Nvidia versus AMD for AI.

Ollama and local AI agents

Ollama can provide the model layer for an agent. Its current CLI can launch supported integrations, and its chat API can serve models with tool-calling capabilities.

The model alone is not the complete agent. A working system may also contain:

An orchestrator that decides when to call the model.
Tool definitions and permission checks.
Retrieval or memory storage.
A sandbox for code execution.
Audit logs and human-approval steps.
Timeouts, budgets and stop conditions.

Our AI Agents hub covers that larger system. Use the guide to building an AI agent for an implementation path and open-source AI agent frameworks when choosing an orchestrator.

Local inference reduces one external dependency. It does not remove prompt injection, unsafe tools, excessive permissions or malicious documents.

Common Ollama problems

ollama is not recognized

Restart the terminal after installation. On Windows, confirm the installer added Ollama to the user PATH. On macOS, start the application so it can offer to create the CLI link.

The first response takes a long time

The model may still be downloading or loading into memory. Later calls are faster while the model remains loaded. Use ollama ps to check its state.

Ollama uses the CPU instead of the GPU

Update Ollama and GPU drivers, confirm support in Ollama’s hardware list, then restart. On Linux, a suspend/resume cycle can occasionally prevent Nvidia detection; Ollama documents a driver-module workaround on its GPU page.

The process runs out of memory

Use a smaller model or lower context. The package size does not include every runtime allocation. Close other memory-heavy applications.

The model claims 256K context but Ollama uses 4K

The first figure is the model’s maximum. The second is Ollama’s allocated runtime context based on settings and VRAM.

The API works locally but not from another computer

That is the secure default: Ollama binds to loopback. Do not solve it by exposing the port publicly. Design authenticated private access first.

Ollama checklist

Install from Ollama’s official site.
Start with a model that leaves memory headroom.
Use explicit model tags for repeatable deployments.
Verify CPU/GPU placement and context with ollama ps.
Keep the local API on 127.0.0.1 unless a secured network design requires otherwise.
Disable cloud features if the environment must remain local-only.
Review model licenses and provenance.
Put tool use behind permissions, isolation and human approval.
Update Ollama, models, drivers and the operating system.

Ollama makes local inference easy to start. The durable work begins after the first prompt: selecting the right model, measuring its real memory use and treating the API as a service that needs security boundaries.

byDavid Porter

AI Agents Local AI AI tools

Best Local AI Models in 2026: 7 Models Compared

Local AI Hardware Requirements: RAM, VRAM, GPUs and Model Sizes

Write a comment