Skip to main content

LLM Bindings

Dagger supports a wide range of popular Large Language Models (LLMs), including those from OpenAI, Anthropic and Google. Dagger can access these models either through their respective cloud-based APIs or using a local provider like Ollama.

Dagger uses the system's standard environment variables to route LLM requests. Dagger will look for these variables in your environment, or in a .env file in the current directory (.env files in parent directories are not yet supported).

tip

Secrets in .env files can use Dagger's built-in secrets support. For example:

# OpenAI API key stored in 1Password
OPENAI_API_KEY="op://Private/OpenAI API Key/password"

Anthropic​

  • ANTHROPIC_API_KEY: required
  • ANTHROPIC_MODEL: optional, defaults to "claude-3-5-sonnet-latest". See other model name strings

OpenAI​

  • OPENAI_API_KEY: required
  • OPENAI_MODEL: optional, defaults to "gpt-4o". See other model name strings
  • Optional:
    • OPENAI_BASE_URL: for alternative OpenAI-compatible endpoints like Azure or local models
    • OPENAI_AZURE_VERSION: for use with Azure's API to OpenAI.

Google Gemini​

  • GEMINI_API_KEY: required
  • GEMINI_MODEL: optional, defaults to "gemini-2.0-flash". See other model name strings

Azure OpenAI Service​

  • OPENAI_BASE_URL required, for example "https://your-azure-openai-resource.cognitiveservices.azure.com"
  • OPENAI_API_KEY: required, Azure OpenAI deployment API key
  • OPENAI_MODEL: optional, for example, "gpt-4o". See other model name strings
  • OPENAI_AZURE_VERSION: optional, for example, "2024-12-01-preview"

Ollama​

  1. Install Ollama from ollama.ai.

  2. Start the Ollama server with host binding:

    OLLAMA_HOST="0.0.0.0:11434" ollama serve
    note

    Ollama has a default context length of 2048. To change this, also set the OLLAMA_CONTEXT_LENGTH environment variable.

  3. Pull models to your local Ollama service:

    ollama pull MODEL-NAME

    For example, a good model for working with code is qwen2.5-coder. This model is available in various sizes. For example, to pull qwen2.5-coder:14b, which can fit in roughly 16 GB of memory, use the following command:

    ollama pull qwen2.5-coder:14b

    Any model from Ollama or any other model distributor that supports tools can be used with Dagger. Review this list of Ollama models supporting tools to find a suitable model.

  4. Obtain the host IP address:

    On macOS / Linux (modern distributions):

    ifconfig | grep "inet " | grep -v 127.0.0.1

    On Linux (older distributions):

    ip addr | grep "inet " | grep -v 127.0.0.1
    note

    This step is needed because Dagger's LLM core type runs inside the Dagger Engine and needs to reach the Ollama service running on the host. Although we are exploring the implementation of automatic tunneling, the current approach is to use the host's actual IP address (instead of localhost) to allow Dagger to communicate with Ollama.

  5. Configure the following environment variables. Replace YOUR-IP with the IP address from the previous step and MODEL-NAME with the default model to use (this can be changed at runtime). The /v1/ route refers to Ollama's OpenAI compatible routes.

    OPENAI_BASE_URL=http://YOUR-IP:11434/v1/
    OPENAI_MODEL=MODEL-NAME

    For example, if your IP is 192.168.64.1 and your preferred model is qwen2.5-coder:14b:

    OPENAI_BASE_URL=http://192.168.64.1:11434/v1/
    OPENAI_MODEL=qwen2.5-coder:14b
    warning

    The trailing / in the route URL is mandatory.