You can run local AI on Windows and Mac without training a model or renting a server. Install a model runner, download a model that fits your memory and start a chat. The shortest cross-platform Ollama command is:
ollama run qwen3.5:4b
That command downloads the selected model package on first use, loads it and opens an interactive conversation. A 16 GB computer is a practical starting point for compact models. Eight gigabytes can handle small quantized models; 32 GB creates room for larger models and longer context.
Windows and macOS reach the same result through different hardware paths. A Windows PC often has separate system RAM and dedicated GPU VRAM. Apple Silicon uses unified memory shared by the CPU and GPU. Choosing by the exact model package and available memory is therefore more reliable than comparing a Mac’s memory figure directly with a PC graphics card.
This platform guide builds on how to run AI locally. Visit the Local AI hub for model, privacy, document and agent guides.
Choose Ollama or LM Studio
Both applications run downloaded models on your computer.
| Choice | Best for | Windows | Mac | Interface |
| Ollama | Repeatable commands, APIs and integrations | Native app | Native app | Terminal and app menu |
| LM Studio | Visual model search and desktop chat | x64 and ARM64 | Apple Silicon | Graphical app |
Ollama handles model downloads, hardware acceleration, templates and a local API. Use it when you want commands that are easy to reproduce or plan to connect other software.
LM Studio offers a graphical model browser, chat interface, document attachments and a local API. It is the easier route for someone who wants to avoid a terminal after installation. AI World Today has a dedicated LM Studio tutorial.
The model determines much of the quality; the runner determines how you install, load and connect it. You can test both without downloading the largest available model.
Check your hardware before installation
Record four values:
- Total system RAM or Apple unified memory.
- GPU model and dedicated VRAM on a Windows PC.
- Available SSD space.
- Operating-system version.
Use this starting range for quantized language models:
| Available memory | First model class | Example package |
| 8 GB | 1B–3B | Llama 3.2 1B, 1.3 GB in Ollama |
| 16 GB | 3B–9B | Qwen 3.5 4B, 3.4 GB; Ministral 3 8B, 6.0 GB |
| 32 GB | 12B–27B | Gemma 4 12B, 7.6 GB; Qwen 3.5 27B, 17 GB |
| 64 GB | Larger quantized models | DeepSeek R1 70B, 43 GB |
The listed GB value is the model package, not total runtime memory. Context, compute buffers, images, the operating system and other applications need memory too. Read local AI hardware requirements before buying a machine.
How to run local AI on Windows with Ollama
Step 1: confirm Windows support
Ollama’s current Windows documentation requires Windows 10 22H2 or newer. The application supports Nvidia and selected AMD Radeon GPUs through documented acceleration paths. It can also run on the CPU.
Open Settings → System → About to see installed RAM and Windows version. Open Task Manager → Performance → GPU to see the GPU and dedicated memory.
Step 2: install Ollama
Download OllamaSetup.exe from the official Ollama download page. The standard Windows installer runs in your account and does not require administrator rights.
After installation, open PowerShell and verify:
ollama -v
If PowerShell cannot find the command, close and reopen the terminal. Confirm that Ollama is running in the taskbar.
Step 3: download and run a model
Start with a compact model:
ollama run qwen3.5:4b
For very limited memory:
ollama run llama3.2:1b
For a 16 GB or roomier machine that you want to test with a larger package:
ollama run ministral-3:8b
Wait for the initial download. Then type a practical prompt:
Turn the following meeting notes into a decision list. Do not add facts that are absent from the notes:
[paste test notes]
Leave the chat with /bye.
Step 4: verify GPU use
Run:
ollama ps
The PROCESSOR field reports 100% GPU, 100% CPU or a CPU/GPU split. A split means some layers are in dedicated VRAM and the rest in system RAM. The model can work this way, but full GPU placement is generally faster.
If Ollama falls back to CPU, update Ollama and the graphics driver, then check the exact card in Ollama’s hardware support list. Nvidia, AMD ROCm and Vulkan have different requirements.
Step 5: manage model storage
Windows stores models by default under:
C:\Users\%username%\.ollama\models
Set the user environment variable OLLAMA_MODELS if model files should live on another drive. Quit and restart Ollama after changing it.
Useful commands are:
ollama ls
ollama pull qwen3.5:4b
ollama stop qwen3.5:4b
ollama rm qwen3.5:4b
Use ollama rm for ordinary cleanup so Ollama can manage shared model data correctly.
How to run local AI on a Mac with Ollama
Step 1: identify the Mac
Choose Apple menu → About This Mac. Record the chip, memory and macOS version.
Current Ollama macOS requirements specify macOS Sonoma 14 or newer. Apple M-series Macs receive CPU and GPU support through Metal. Intel Macs use CPU execution.
Apple unified memory is shared by the operating system, CPU and GPU. A 16 GB Mac does not give a model all 16 GB. Close heavy applications and start with a compact model.
Step 2: install Ollama
Download the macOS disk image from Ollama, open it and move the app into Applications. Start Ollama. It will check whether the CLI is available in your path and can offer to create a link.
Open Terminal and verify:
ollama -v
Step 3: run a model
On an 8 GB Mac, begin with:
ollama run llama3.2:1b
On a 16 GB Apple Silicon Mac, try:
ollama run qwen3.5:4b
An 8B model can also fit in many 16 GB configurations, but usable context and free memory depend on the other applications running.
Check placement and context:
ollama ps
Ollama reports Apple acceleration as GPU placement even though the memory pool is unified.
Step 4: find and manage model files
The default model directory is:
~/.ollama/models
The same ollama ls, pull, stop and rm commands work on macOS.
Run local AI with LM Studio on Windows
LM Studio’s current system requirements recommend at least 16 GB RAM and 4 GB dedicated VRAM on Windows. Its x64 version requires AVX2 CPU support; Windows on ARM is also supported.
- Download LM Studio from its official site.
- Install and open the app.
- Open Discover.
- Search for a small instruction-tuned model.
- Choose a quantized download that fits your memory.
- Open Chat and load the model.
- Start with a short test prompt.
LM Studio tells you about model compatibility and memory before loading. A green-looking estimate is still a preflight check; watch real memory use during a representative task.
Run local AI with LM Studio on Mac
LM Studio currently supports Apple Silicon Macs and requires macOS 14 or newer. Its documentation recommends 16 GB or more of RAM, while noting that 8 GB Macs may run smaller models with modest context. Intel Macs are not supported by current LM Studio releases.
- Install the macOS app.
- Search for an appropriate GGUF or MLX model in Discover.
- Download one quantized variant.
- Load it in Chat.
- Keep the context conservative until memory pressure is known.
LM Studio supports llama.cpp models on Windows and Mac and can also use Apple’s MLX format on Apple Silicon.
Is the model really offline?
A downloaded model can perform local inference without an internet connection. That does not make every feature offline.
LM Studio’s offline documentation says local chat, document chat and its local server can operate offline. Model search, downloads, runtime downloads and update checks require connectivity.
Ollama can run downloaded models locally, but it also supports cloud-tagged models and cloud features. Enforce local-only mode with:
OLLAMA_NO_CLOUD=1
Or add this to ~/.ollama/server.json:
{
"disable_ollama_cloud": true
}
Restart Ollama. This disables Ollama cloud models and its web search. Other tools or programs can still make network requests. The guide to whether local AI is private and safe maps the complete data path.
Secure the local API on both platforms
Ollama automatically serves http://localhost:11434. Its local API requires no authentication. The secure default binds it to 127.0.0.1, which normally limits access to the same computer.
Do not expose port 11434 to the internet, bind it to every interface or publish it through a tunnel without an authenticated reverse proxy or private VPN, TLS, firewall restrictions and monitoring. An API key entered into an OpenAI-compatible client is ignored by the local Ollama endpoint; it does not secure the server.
The risk grows when a model can call tools. A local AI agent may read files, browse or run code. Apply the boundaries in AI agent security before granting those permissions.
Protect locally stored prompts and documents
Local processing moves responsibility to the device owner.
- Use a strong login password and automatic screen lock.
- Enable Windows Device Encryption or BitLocker where available.
- Enable FileVault on a Mac and store the recovery key safely.
- Install operating-system and application updates.
- Keep sensitive model and document directories out of broad network shares.
- Use a separate standard user account for daily work when practical.
- Back up encrypted data and test recovery.
Microsoft describes BitLocker as full-volume encryption that addresses data exposure from lost or decommissioned devices. Apple describes FileVault as built-in encryption for data at rest.
Encryption protects a powered-off or locked device. It does not stop malware or another logged-in user from reading accessible files.
Add documents or local agents later
Get ordinary chat working before adding a wider stack.
For private document questions, the local RAG guide covers parsing, embeddings, retrieval and citations. For action-taking systems, use the AI Agents hub and understand AI agent architecture.
Both workloads need more memory than a one-turn chat. Document indexes, embedding models, browsers and tool processes consume RAM and storage around the main language model.
Windows and Mac troubleshooting
The model loads but the computer becomes unresponsive
Select a smaller package, lower context and close memory-heavy applications. A weight file that fits can still leave too little memory for context and the operating system.
Generation is slow
Use ollama ps to check CPU/GPU placement. Update drivers on Windows. On Mac, confirm the app is native and the model is not too large for unified memory.
The command is missing after installation
Restart the terminal. On Windows, confirm Ollama is running. On Mac, start the app so it can add the CLI link.
The advertised context is 128K or 256K, but the runner uses less
The advertised figure is the model’s maximum. Ollama currently chooses a runtime default based on VRAM, beginning at 4K below 24 GiB. Raising context increases memory use.
The model answers confidently but incorrectly
Local models hallucinate. Use a model suited to the task, request sources when source text is available and verify important claims. A bigger model reduces some errors but does not remove the need for checks.
FAQ
Is Windows or Mac better for local AI?
Windows offers a wide choice of upgradeable Nvidia and AMD hardware. Apple Silicon offers efficient Metal acceleration and unified memory in a simpler package. The best choice depends on model size, software compatibility, portability and budget.
Can a normal laptop run local AI?
Yes. Small quantized 1B–9B models run on many recent laptops. Speed, fan noise, battery use and context capacity vary.
Can I run local AI on an Intel Mac?
Ollama supports CPU-only execution on x86 Macs that meet its macOS requirement. LM Studio’s current Mac application supports Apple Silicon rather than Intel.
Do I need an Nvidia GPU on Windows?
No. Ollama can run on the CPU and supports documented AMD and Vulkan paths. Nvidia CUDA has broad support, which often makes a compatible Nvidia card the easiest discrete-GPU route.
Which model should I install first?
Start with Qwen 3.5 4B on a 16 GB system or Llama 3.2 1B on a constrained machine. Then use the best local AI models guide to match a model to language, coding, vision or reasoning tasks.
Can I run ChatGPT itself on Windows or Mac?
ChatGPT is a hosted OpenAI product. You can run open-weight models locally and use them through a similar chat interface, but that does not create a local copy of the ChatGPT service. See can you run ChatGPT locally?.
Final checklist
- Confirm operating-system and memory requirements.
- Install Ollama or LM Studio from the official site.
- Choose a model package well below available memory.
- Test with a real, non-sensitive task.
- Verify processor placement and context.
- Keep local APIs on loopback unless secured remote access is deliberately engineered.
- Enable disk encryption and protect recovery keys.
- Add documents, APIs and tools one component at a time.
Windows and Mac can both run capable local AI. The decisive variables are the exact model package, available memory and whether the complete workflow stays inside the security boundary you intended.
byDavid Porter
Popular news
Anthropic Researchers Warn Advanced AI Models Could Trigger Human Extinction
Senator Josh Hawley Launches Massive Investigation Into OpenAI After Critical Hugging Face Breach
Andy Burnham Faces Urgent Demands to Block Artificial Superintelligence in Manchester
UK Computer Science Graduates Face a Massive Job Market Shift Driven by AI
Anthropic Researchers Issue Urgent Warning Regarding Elon Musk and Global AI Safety Risks
OpenAI Board Member Paul Christiano Warns AI Control Could Be Lost Forever
Latest comments
Loading