Skip to main content

Ollama

What is Ollama?​

Ollama runs open LLMs locally and exposes an OpenAI-compatible HTTP API, giving PHP apps local inference with no external calls or per-token cost. Point Prism, Neuron AI, LLPhant, or openai-php at it, and pair it with pgvector for fully local RAG.

Start Ollama​

docker compose up -d ollama

Stop Ollama​

docker compose stop ollama

This stops the container without deleting downloaded models (kept in the ollama Docker volume). To remove the container: docker compose rm -f ollama.

Configuration​

All settings live in ollama/defaults.env and can be overridden by adding the same line to your own .env:

VariableDefaultWhat it does
OLLAMA_VERSIONlatestImage tag from the ollama/ollama Docker Hub image.
OLLAMA_HOST_PORT11434Host-side port Ollama is published on (container port 11434).

Models are stored in the ollama named Docker volume at /root/.ollama, not under DATA_PATH_HOST, so they persist across container restarts but aren't visible directly on your host filesystem.

Pull and use a model​

docker compose exec ollama ollama pull llama3.2

The API is at http://localhost:11434 from your host, or http://ollama:11434 from other containers. It's OpenAI-compatible under http://ollama:11434/v1, so most OpenAI PHP SDKs work by just pointing the base URL there.

Common issues​

  • First request after docker compose up is slow or fails. No model is pulled by default, run ollama pull <model> first; the pull itself can take a while depending on model size and your connection.
  • Out of disk space. Models are large (several GB each) and accumulate in the ollama volume. Remove unused ones with docker compose exec ollama ollama rm <model>.
  • CPU-only inference is slow. Ollama uses GPU acceleration when available; without GPU passthrough configured in your Docker setup, larger models will be noticeably slower.
  • App can't connect but the container is running. Use the container name ollama, not localhost, from inside another container.

Need a unified gateway across multiple LLM providers instead? See LiteLLM. Need an OpenAI-compatible server with more model formats? See LocalAI. New to Laradock? Start with Getting Started.