Every Ollama install runs a local HTTP server on port 11434, and that server is the real interface to the models. The ollama run command is a thin client on top of it. Once you know the two main endpoints, the streaming format, and the options object, you can wire a local model into any application. This guide covers the Ollama API with working examples for generation, chat, streaming, structured output, tool calling, and Python.
There is also an OpenAI-compatible route, so existing code that talks to OpenAI can point at Ollama with a base-URL change. Both paths are covered below.
How the Ollama API works
The Ollama API is a plain REST API served at http://localhost:11434. You send JSON with POST, and by default you get a stream of newline-delimited JSON objects back. No API key is required for local access. The full endpoint list is in the official API reference.
The core endpoints are:
| Endpoint | Method | Purpose |
|---|---|---|
/api/generate |
POST | Single-prompt text completion |
/api/chat |
POST | Multi-turn chat with message history and tools |
/api/embed |
POST | Generate embeddings |
/api/tags |
GET | List installed models |
/api/ps |
GET | List models currently loaded in memory |
/api/pull |
POST | Download a model |
/api/show |
POST | Show a model’s parameters and template |
A first request against a model that is not loaded pays a one-time load cost. After that, Ollama keeps the model in memory for 5 minutes by default, controlled by the keep_alive field.
Check that the server is up with a quick call:
curl http://localhost:11434/api/versionGenerating text: /api/generate and /api/chat
Use /api/generate for a single prompt with no conversation history. It is the right fit for classification, extraction, summarization, and other one-shot tasks.
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1",
"prompt": "Summarize in one sentence: Ollama serves a local HTTP API on port 11434.",
"stream": false
}'Use /api/chat when you need turn-by-turn context or tool calling. You pass a messages array with role values of system, user, assistant, or tool, and you send the whole history back on each request.
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1",
"messages": [
{"role": "system", "content": "You answer in one short sentence."},
{"role": "user", "content": "What is the KV cache?"}
],
"stream": false
}'For most application code, /api/chat is the better default even for single questions, because the message format matches how every other chat API works.
Streaming and response metrics
By default stream is true, and Ollama returns one JSON object per chunk as tokens are produced. Each chunk carries a partial response (or message), and the final chunk has "done": true plus timing data.
{"model":"llama3.1","created_at":"...","response":" scattering","done":false}
{"model":"llama3.1","created_at":"...","response":"","done":true,
"total_duration":4883583458,"load_duration":1067152416,
"prompt_eval_count":26,"eval_count":298,"eval_duration":3789981000}All durations are in nanoseconds. To get tokens per second, divide eval_count by eval_duration and multiply by 1,000,000,000:
tokens/sec = eval_count / eval_duration * 1e9Set "stream": false to get a single response object instead. That is simpler for short outputs, but the client waits for the full generation with no partial output.
Setting model options
The options object tunes sampling and context for a single request. It works on /api/generate, /api/chat, and /api/embed.
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1",
"messages": [{"role": "user", "content": "Name three container runtimes."}],
"stream": false,
"options": {
"temperature": 0.2,
"num_ctx": 8192,
"num_predict": 200,
"seed": 42,
"stop": ["\n\n"]
}
}'The options you will use most:
temperature— lower is more deterministic. Use0to0.3for extraction and code.num_ctx— context window in tokens for this request. Larger values use more memory.num_predict— cap on generated tokens.seed— fixed seed plustemperature: 0gives repeatable output.stop— an array of strings that end generation.
⚠️ Note: num_ctx set above what your hardware can hold forces a partial CPU offload and slows generation. Confirm placement with ollama ps after a large-context request. The Ollama hardware requirements guide covers how context size maps to memory.
Structured JSON output from the Ollama API
The format field constrains the model’s output. Set it to "json" for any valid JSON, or pass a JSON schema object to force a specific shape.
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1",
"messages": [{"role": "user", "content": "List two Linux distros with release years. Respond in JSON."}],
"stream": false,
"format": {
"type": "object",
"properties": {
"distros": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {"type": "string"},
"year": {"type": "integer"}
},
"required": ["name", "year"]
}
}
},
"required": ["distros"]
}
}'Two rules make this reliable. Keep the word “JSON” in the prompt, since the docs call for it explicitly. Also set temperature low so the model does not wander outside the schema.
Tool calling with the Ollama API
The /api/chat endpoint supports function calling through a tools array. You describe each function with a JSON Schema, and the model replies with a tool_calls entry instead of text when it decides to use one.
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1",
"messages": [{"role": "user", "content": "What is the weather in Toronto?"}],
"stream": false,
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}]
}'Your code runs the function, then sends the result back as a message with "role": "tool" and the same tool_name. The model uses that result to write its final answer. Tool support depends on the model, so check the model card for a “tools” capability before relying on it.
Calling the Ollama API from Python
The official ollama Python package wraps the REST API. Install it with pip install ollama.
from ollama import chat
response = chat(model='llama3.1', messages=[
{'role': 'user', 'content': 'Why is the sky blue?'},
])
print(response.message.content)Streaming is a loop over the response:
from ollama import chat
stream = chat(
model='llama3.1',
messages=[{'role': 'user', 'content': 'Explain the KV cache in two sentences.'}],
stream=True,
)
for chunk in stream:
print(chunk['message']['content'], end='', flush=True)To reach an Ollama server on another host, pass a Client with the address:
from ollama import Client
client = Client(host='http://192.168.1.50:11434')
response = client.chat(model='llama3.1', messages=[
{'role': 'user', 'content': 'ping'},
])There is an AsyncClient with the same methods for asyncio code, plus embed(), list(), ps(), and pull() helpers.
The OpenAI-compatible endpoint
Ollama also serves an OpenAI-style API at http://localhost:11434/v1. It supports /v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/models. This lets you reuse OpenAI SDK code without rewriting it.
from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')
response = client.chat.completions.create(
model='llama3.1',
messages=[{'role': 'user', 'content': 'Hello'}],
)
print(response.choices[0].message.content)The api_key value is required by the SDK but ignored by Ollama for local use. The native /api endpoints expose more of Ollama’s features, such as the full options object, so use /v1 mainly for drop-in compatibility. If you are still choosing a runtime, the Ollama vs LM Studio comparison covers which one fits a server workload.
Securing the Ollama API for remote access
The Ollama API has no built-in authentication. Anyone who can reach port 11434 can use your models and pull or delete them. That is fine on localhost, but exposing it to a network needs a front door.
For remote access, keep Ollama bound to 127.0.0.1 and put a reverse proxy in front with authentication and TLS:
server {
listen 443 ssl;
server_name ollama.example.com;
location / {
proxy_pass http://127.0.0.1:11434;
proxy_set_header Host localhost:11434;
auth_basic "Ollama";
auth_basic_user_file /etc/nginx/.htpasswd;
}
}⚠️ Note: setting OLLAMA_HOST=0.0.0.0 to allow remote clients without a proxy puts an unauthenticated model server on the open network. Only do this inside a private network or behind a firewall that restricts the source IPs. The Ollama on a DigitalOcean droplet guide walks through the firewall setup, and the Ollama Cloud pricing guide covers the hosted option with real API keys.
Frequently Asked Questions
Q: What port does the Ollama API use?
A: Port 11434 on 127.0.0.1 by default. Change the bind address with the OLLAMA_HOST environment variable, for example OLLAMA_HOST=0.0.0.0:11434.
Q: Does the Ollama API need an API key?
A: No, not for local use. The API is open to anything that can reach the port. Ollama’s hosted cloud models use a key, and self-hosted remote access should sit behind a reverse proxy that adds authentication.
Q: What is the difference between /api/generate and /api/chat?
A: /api/generate takes a single prompt string for one-shot tasks. /api/chat takes a messages array with roles, keeps conversation structure, and supports tool calling. Use /api/chat for application code.
Q: How do I get JSON output from the Ollama API?
A: Set the format field to "json" for any valid JSON, or to a JSON schema object to enforce a structure. Keep the word “JSON” in your prompt and use a low temperature.
Q: Can I use the OpenAI Python SDK with Ollama?
A: Yes. Point the client base_url at http://localhost:11434/v1 and pass any non-empty api_key. The /v1 endpoint covers chat, completions, and embeddings.
Quick Summary:
– The Ollama API is a keyless REST API on http://localhost:11434; /api/generate is single-prompt, /api/chat is multi-turn with tools.
– Responses stream as newline-delimited JSON by default; the final chunk carries nanosecond timings for a tokens-per-second calculation.
– The options object sets temperature, num_ctx, num_predict, seed, and stop per request.
– format accepts "json" or a JSON schema for structured output; tools on /api/chat enables function calling.
– The ollama Python package and the OpenAI-compatible /v1 endpoint both wrap the same server.
– There is no authentication — never expose port 11434 to a network without a reverse proxy and TLS.
Try the structured-output example against a model you already have, then swap the schema for one your app actually needs.