Ollama
The least interesting part of the local setup, which is exactly why it works.
Tested 2026-11-08
Ollama runs local models and serves them on a local port, so anything that speaks the OpenAI-style API can use them. Mine listens on 127.0.0.1:11434 and both my local models are pulled through it.
The good part is that it stays out of the way. Pull a model, make a tag with the settings you want baked in, then forget the runtime exists. The less good part is that tuning is all environment variables and model file edits, so you find the settings that work by reading release notes rather than by clicking anything.
Pros
- One command to pull and run a model
- Custom tags let you bake in context size and thinking settings
- Serves a local API, so any client can use the same models
- Free, open source, and no account
Cons
- Tuning context and KV cache is environment variables, not a settings screen
- `ollama ps` is the only easy way to see whether the model actually fit on the GPU
- No built-in cost or speed reporting
What I tested
| Test | Result |
|---|---|
| Long document summary | N/A for the runtime. |
| Coding task | N/A. Served both local models used in the tests above without a failure. |
| Hard reasoning | N/A. |
| Image understanding | TODO: test a vision model through it. |
| Cost for a week of real use | $0. It is the reason the local models cost nothing per token. |
Bottom line: Boring in the right way. Install it, pull a model that fits your card, and stop thinking about the runtime.
Links
What this site is
I test AI tools myself — cloud models, local models, and the apps that manage them. Reviews only appear once I’ve run the same prompts through each tool on my own machine or account. No sponsorships, no affiliate links, no ads. More about how this works.
One email a week
What I tested this week, what it cost to run, and the one thing worth your time. No spam, unsubscribe in one click.