Skip to main content

Cookbook

The Dagger Cookbook provides practical, real-world examples for common development workflows. Each recipe shows you how to solve specific problems using Dagger's features and APIs.

These recipes demonstrate patterns you can adapt for your own projects. They cover everything from basic container operations to advanced CI/CD workflows and AI agent integrations. Use your browser's find function (Ctrl+F or Cmd+F) to quickly locate specific topics.

Builds​

Build and compile applications in reproducible container environments. This section contains practical examples for building artifacts (files and directories) with Dagger.

Perform a multi-stage build​

The following Dagger Function performs a multi-stage build.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Build and publish Docker container
func (m *MyModule) Build(
	ctx context.Context,
	// source code location
	// can be local directory or remote Git repository
	src *dagger.Directory,
) (string, error) {
	// build app
	builder := dag.Container().
		From("golang:latest").
		WithDirectory("/src", src).
		WithWorkdir("/src").
		WithEnvVariable("CGO_ENABLED", "0").
		WithExec([]string{"go", "build", "-o", "myapp"})

	// publish binary on alpine base
	prodImage := dag.Container().
		From("alpine").
		WithFile("/bin/myapp", builder.File("/src/myapp")).
		WithEntrypoint([]string{"/bin/myapp"})

	// publish to ttl.sh registry
	addr, err := prodImage.Publish(ctx, "ttl.sh/myapp:latest")
	if err != nil {
		return "", err
	}

	return addr, nil
}

Example​

Perform a multi-stage build of the source code in the golang/example/hello repository and publish the resulting image:

dagger -c 'build https://github.com/golang/example#master:hello'

Perform a matrix build​

The following Dagger Function performs a matrix build.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
	"fmt"
)

type MyModule struct{}

// Build and return directory of go binaries
func (m *MyModule) Build(
	ctx context.Context,
	// Source code location
	src *dagger.Directory,
) *dagger.Directory {
	// define build matrix
	gooses := []string{"linux", "darwin"}
	goarches := []string{"amd64", "arm64"}

	// create empty directory to put build artifacts
	outputs := dag.Directory()

	golang := dag.Container().
		From("golang:latest").
		WithDirectory("/src", src).
		WithWorkdir("/src")

	for _, goos := range gooses {
		for _, goarch := range goarches {
			// create directory for each OS and architecture
			path := fmt.Sprintf("build/%s/%s/", goos, goarch)

			// build artifact
			build := golang.
				WithEnvVariable("GOOS", goos).
				WithEnvVariable("GOARCH", goarch).
				WithExec([]string{"go", "build", "-o", path})

			// add build to outputs
			outputs = outputs.WithDirectory(path, build.Directory(path))
		}
	}

	// return build directory
	return outputs
}

Example​

Perform a matrix build of the source code in the golang/example/hello repository and export build directory with go binaries for different operating systems and architectures.

dagger -c 'build https://github.com/golang/example#master:hello | export /tmp/matrix-builds'

Inspect the contents of the exported directory with tree /tmp/matrix-builds. The output should look like this:

/tmp/matrix-builds
└── build
    ├── darwin
    │   ├── amd64
    │   │   └── hello
    │   └── arm64
    │       └── hello
    └── linux
        ├── amd64
        │   └── hello
        └── arm64
            └── hello

8 directories, 4 files

Build multi-arch image​

The following Dagger Function builds a single image for different CPU architectures using native emulation.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Build and publish multi-platform image
func (m *MyModule) Build(
	ctx context.Context,
	// Source code location
	// can be local directory or remote Git repository
	src *dagger.Directory,
) (string, error) {
	// platforms to build for and push in a multi-platform image
	var platforms = []dagger.Platform{
		"linux/amd64", // a.k.a. x86_64
		"linux/arm64", // a.k.a. aarch64
		"linux/s390x", // a.k.a. IBM S/390
	}

	// container registry for the multi-platform image
	const imageRepo = "ttl.sh/myapp:latest"

	platformVariants := make([]*dagger.Container, 0, len(platforms))
	for _, platform := range platforms {
		// pull golang image for this platform
		ctr := dag.Container(dagger.ContainerOpts{Platform: platform}).
			From("golang:1.20-alpine").
			// mount source code
			WithDirectory("/src", src).
			// mount empty dir where built binary will live
			WithDirectory("/output", dag.Directory()).
			// ensure binary will be statically linked and thus executable
			// in the final image
			WithEnvVariable("CGO_ENABLED", "0").
			// build binary and put result at mounted output directory
			WithWorkdir("/src").
			WithExec([]string{"go", "build", "-o", "/output/hello"})

		// select output directory
		outputDir := ctr.Directory("/output")

		// wrap the output directory in the new empty container marked
		// with the same platform
		binaryCtr := dag.Container(dagger.ContainerOpts{Platform: platform}).
			WithRootfs(outputDir)

		platformVariants = append(platformVariants, binaryCtr)
	}

	// publish to registry
	imageDigest, err := dag.Container().
		Publish(ctx, imageRepo, dagger.ContainerPublishOpts{
			PlatformVariants: platformVariants,
		})

	if err != nil {
		return "", err
	}

	// return build directory
	return imageDigest, nil
}

Example​

Build and publish a multi-platform image:

dagger -c 'build https://github.com/golang/example#master:hello'

Build multi-arch image with cross-compliation​

The following Dagger Function builds a single image for different CPU architectures using cross-compilation.

info

This Dagger Function uses the containerd utility module. To run it locally install the module first with dagger install github.com/levlaz/daggerverse/containerd@v0.1.2

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Build and publish multi-platform image
func (m *MyModule) Build(
	ctx context.Context,
	// Source code location
	// can be local directory or remote Git repository
	src *dagger.Directory,
) (string, error) {
	// platforms to build for and push in a multi-platform image
	var platforms = []dagger.Platform{
		"linux/amd64", // a.k.a. x86_64
		"linux/arm64", // a.k.a. aarch64
		"linux/s390x", // a.k.a. IBM S/390
	}

	// container registry for the multi-platform image
	const imageRepo = "ttl.sh/myapp:latest"

	platformVariants := make([]*dagger.Container, 0, len(platforms))
	for _, platform := range platforms {
		// parse architecture using containerd utility module
		platformArch, err := dag.Containerd().ArchitectureOf(ctx, platform)

		if err != nil {
			return "", err
		}

		// pull golang image for the *host* platform, this is done by
		// not specifying the a platform. The default is the host platform.
		ctr := dag.Container().
			From("golang:1.21-alpine").
			// mount source code
			WithDirectory("/src", src).
			// mount empty dir where built binary will live
			WithDirectory("/output", dag.Directory()).
			// ensure binary will be statically linked and thus executable
			// in the final image
			WithEnvVariable("CGO_ENABLED", "0").
			// configure go compiler to use cross-compilation targeting the
			// desired platform
			WithEnvVariable("GOOS", "linux").
			WithEnvVariable("GOARCH", platformArch).
			// build binary and put result at mounted output directory
			WithWorkdir("/src").
			WithExec([]string{"go", "build", "-o", "/output/hello"})

		// select output directory
		outputDir := ctr.Directory("/output")

		// wrap the output directory in the new empty container marked
		// with the same platform
		binaryCtr := dag.Container(dagger.ContainerOpts{Platform: platform}).
			WithRootfs(outputDir).
			WithEntrypoint([]string{"/hello"})

		platformVariants = append(platformVariants, binaryCtr)
	}

	// publish to registry
	imageDigest, err := dag.Container().
		Publish(ctx, imageRepo, dagger.ContainerPublishOpts{
			PlatformVariants: platformVariants,
		})

	if err != nil {
		return "", err
	}

	// return build directory
	return imageDigest, nil
}

Example​

Build and publish a multi-platform image with cross compliation:

dagger -c 'build https://github.com/golang/example#master:hello'

Build image from Dockerfile​

The following Dagger Function builds an image from a Dockerfile.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Build and publish image from existing Dockerfile
func (m *MyModule) Build(
	ctx context.Context,
	// location of directory containing Dockerfile
	src *dagger.Directory,
) (string, error) {
	ref, err := src.
		DockerBuild(). // build from Dockerfile
		Publish(ctx, "ttl.sh/hello-dagger")

	if err != nil {
		return "", err
	}

	return ref, nil
}

Example​

Build and publish an image from an existing Dockerfile

dagger -c 'build https://github.com/dockersamples/python-flask-redis'

Build image from Dockerfile using different build context​

The following function builds an image from a Dockerfile with a build context that is different than the current working directory.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Build and publish image from Dockerfile using a build context directory
// in a different location than the current working directory
func (m *MyModule) Build(
	ctx context.Context,
	// location of source directory
	src *dagger.Directory,
	// location of Dockerfile
	dockerfile *dagger.File,
) (string, error) {

	// get build context with dockerfile added
	workspace := dag.Container().
		WithDirectory("/src", src).
		WithWorkdir("/src").
		WithFile("/src/custom.Dockerfile", dockerfile).
		Directory("/src")

	// build using Dockerfile and publish to registry
	ref, err := workspace.DockerBuild(dagger.DirectoryDockerBuildOpts{
		Dockerfile: "custom.Dockerfile",
	}).Publish(ctx, "ttl.sh/hello-dagger")

	if err != nil {
		return "", err
	}

	return ref, nil
}

Example​

Build an image from the source code in https://github.com/dockersamples/python-flask-redis using the Dockerfile from a different build context, at https://github.com/vimagick/dockerfiles#master:registry-cli/Dockerfile:

dagger -c 'build https://github.com/dockersamples/python-flask-redis https://github.com/vimagick/dockerfiles#master:registry-cli/Dockerfile'

Cache application dependencies​

The following Dagger Function uses a cache volume for application dependencies. This enables Dagger to reuse the contents of the cache across Dagger Function runs and reduce execution time.

package main

import "dagger/my-module/internal/dagger"

type MyModule struct{}

// Build an application using cached dependencies
func (m *MyModule) Build(
	// Source code location
	source *dagger.Directory,
) *dagger.Container {
	return dag.Container().
		From("golang:1.21").
		WithDirectory("/src", source).
		WithWorkdir("/src").
		WithMountedCache("/go/pkg/mod", dag.CacheVolume("go-mod-121")).
		WithEnvVariable("GOMODCACHE", "/go/pkg/mod").
		WithMountedCache("/go/build-cache", dag.CacheVolume("go-build-121")).
		WithEnvVariable("GOCACHE", "/go/build-cache").
		WithExec([]string{"go", "build"})
}

Example​

Build an application using cached dependencies:

dagger -c 'build .'

Execute functions concurrently​

The following Dagger Function demonstrates how to use native-language concurrency features (errgroups in Go, task groups in Python), and promises in TypeScript to execute other Dagger Functions concurrently. If any of the concurrently-running functions fails, the remaining ones will be immediately cancelled.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"

	"golang.org/x/sync/errgroup"
)

// Constructor
func New(
	source *dagger.Directory,
) *MyModule {
	return &MyModule{
		Source: source,
	}
}

type MyModule struct {
	Source *dagger.Directory
}

// Return the result of running unit tests
func (m *MyModule) Test(ctx context.Context) (string, error) {
	return m.BuildEnv().
		WithExec([]string{"npm", "run", "test:unit", "run"}).
		Stdout(ctx)
}

// Return the result of running the linter
func (m *MyModule) Lint(ctx context.Context) (string, error) {
	return m.BuildEnv().
		WithExec([]string{"npm", "run", "lint"}).
		Stdout(ctx)
}

// Return the result of running the type-checker
func (m *MyModule) Typecheck(ctx context.Context) (string, error) {
	return m.BuildEnv().
		WithExec([]string{"npm", "run", "type-check"}).
		Stdout(ctx)
}

// Run linter, type-checker, unit tests concurrently
func (m *MyModule) RunAllTests(ctx context.Context) error {
	// Create error group
	eg, gctx := errgroup.WithContext(ctx)

	// Run linter
	eg.Go(func() error {
		_, err := m.Lint(gctx)
		return err
	})

	// Run type-checker
	eg.Go(func() error {
		_, err := m.Typecheck(gctx)
		return err
	})

	// Run unit tests
	eg.Go(func() error {
		_, err := m.Test(gctx)
		return err
	})

	// Wait for all tests to complete
	// If any test fails, the error will be returned
	return eg.Wait()
}

// Build a ready-to-use development environment
func (m *MyModule) BuildEnv() *dagger.Container {
	nodeCache := dag.CacheVolume("node")
	return dag.Container().
		From("node:21-slim").
		WithDirectory("/src", m.Source).
		WithMountedCache("/root/.npm", nodeCache).
		WithWorkdir("/src").
		WithExec([]string{"npm", "install"})
}

Example​

Execute a Dagger Function which performs different types of tests by executing other Dagger Functions concurrently.

dagger -c 'my-module $(host | directory .) | run-all-tests'

Persist service state across runs​

The following Dagger Function uses a cache volume to persist a Redis service's data across Dagger Function runs.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Create Redis service and client
func (m *MyModule) Redis(ctx context.Context) *dagger.Container {
	redisSrv := dag.Container().
		From("redis").
		WithExposedPort(6379).
		WithMountedCache("/data", dag.CacheVolume("my-redis")).
		WithWorkdir("/data").
		AsService(dagger.ContainerAsServiceOpts{UseEntrypoint: true})

	redisCLI := dag.Container().
		From("redis").
		WithServiceBinding("redis-srv", redisSrv).
		WithEntrypoint([]string{"redis-cli", "-h", "redis-srv"})

	return redisCLI
}

var execOpts = dagger.ContainerWithExecOpts{
	UseEntrypoint: true,
}

// Set key and value in Redis service
func (m *MyModule) Set(
	ctx context.Context,
	// The cache key to set
	key string,
	// The cache value to set
	value string,
) (string, error) {
	return m.Redis(ctx).
		WithExec([]string{"set", key, value}, execOpts).
		WithExec([]string{"save"}, execOpts).
		Stdout(ctx)
}

// Get value from Redis service
func (m *MyModule) Get(
	ctx context.Context,
	// The cache key to get
	key string,
) (string, error) {
	return m.Redis(ctx).
		WithExec([]string{"get", key}, execOpts).
		Stdout(ctx)
}

Example​

  • Save data to a Redis service which uses a cache volume to persist a key named foo with value `123:

    dagger -c 'set foo 123'

  • Retrieve the value of the key foo after recreating the service state from the cache volume:

    dagger -c 'get foo'

Add OCI annotations to image​

The following Dagger Function adds OpenContainer Initiative (OCI) annotations to an image.

package main

import (
	"context"
	"fmt"
	"math"
	"math/rand/v2"
)

type MyModule struct{}

// Build and publish image with OCI annotations
func (m *MyModule) Build(ctx context.Context) (string, error) {
	address, err := dag.Container().
		From("alpine:latest").
		WithExec([]string{"apk", "add", "git"}).
		WithWorkdir("/src").
		WithExec([]string{"git", "clone", "https://github.com/dagger/dagger", "."}).
		WithAnnotation("org.opencontainers.image.authors", "John Doe").
		WithAnnotation("org.opencontainers.image.title", "Dagger source image viewer").
		Publish(ctx, fmt.Sprintf("ttl.sh/custom-image-%.0f", math.Floor(rand.Float64()*10000000))) //#nosec
	if err != nil {
		return "", err
	}
	return address, nil
}

Example​

Build and publish an image with OCI annotations:

dagger -c build

Add OCI labels to image​

The following Dagger Function adds OpenContainer Initiative (OCI) labels to an image.

package main

import (
	"context"
	"time"
)

type MyModule struct{}

// Build and publish image with OCI labels
func (m *MyModule) Build(
	ctx context.Context,
) (string, error) {
	ref, err := dag.Container().
		From("alpine").
		WithLabel("org.opencontainers.image.title", "my-alpine").
		WithLabel("org.opencontainers.image.version", "1.0").
		WithLabel("org.opencontainers.image.created", time.Now().String()).
		WithLabel("org.opencontainers.image.source", "https://github.com/alpinelinux/docker-alpine").
		WithLabel("org.opencontainers.image.licenses", "MIT").
		Publish(ctx, "ttl.sh/my-alpine")

	if err != nil {
		return "", err
	}

	return ref, nil
}

Example​

Build and publish an image with OCI labels:

dagger -c build

Invalidate cache​

The following function demonstrates how to invalidate the Dagger layer cache and force execution of subsequent workflow steps, by introducing a volatile time variable at a specific point in the Dagger workflow.

note
  • This is a temporary workaround until cache invalidation support is officially added to Dagger.
  • Changes in mounted cache volumes or secrets do not invalidate the Dagger layer cache.
package main

import (
	"context"
	"time"
)

type MyModule struct{}

// Run a build with cache invalidation
func (m *MyModule) Build(
	ctx context.Context,
) (string, error) {
	output, err := dag.Container().
		From("alpine").
		// comment out the line below to see the cached date output
		WithEnvVariable("CACHEBUSTER", time.Now().String()).
		WithExec([]string{"date"}).
		Stdout(ctx)

	if err != nil {
		return "", err
	}

	return output, nil
}

Example​

Print the date and time, invalidating the cache on each run. However, if the CACHEBUSTER environment variable is removed, the same value (the date and time on the first run) is printed on every run.

dagger -c build

Container Images​

Work with container images and registries. Dagger allows you to build, publish, and export container images, also known as just-in-time artifacts, as part of your Dagger Functions.

Publish a container image to a private registry​

The following Dagger Function publishes a just-in-time container image to a private registry.

package main

import (
	"context"
	"fmt"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Publish a container image to a private registry
func (m *MyModule) Publish(
	ctx context.Context,
	// Registry address
	registry string,
	// Registry username
	username string,
	// Registry password
	password *dagger.Secret,
) (string, error) {
	return dag.Container().
		From("nginx:1.23-alpine").
		WithNewFile(
			"/usr/share/nginx/html/index.html",
			"Hello from Dagger!",
			dagger.ContainerWithNewFileOpts{Permissions: 0o400},
		).
		WithRegistryAuth(registry, username, password).
		Publish(ctx, fmt.Sprintf("%s/%s/my-nginx", registry, username))
}

Examples​

Publish a just-in-time container image to Docker Hub, using the account username user and the password set in the PASSWORD environment variable:

dagger -c 'publish docker.io user env://PASSWORD'

Publish a just-in-time container image to GitHub Container Registry, using the account username user and the GitHub personal access token set in the PASSWORD environment variable:

dagger -c 'publish ghcr.io user env://PASSWORD'

Publish a container image to a private registry with multiple tags​

The following Dagger Function tags a just-in-time container image multiple times and publishes it to a private registry.

package main

import (
	"context"
	"fmt"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Tag a container image multiple times and publish it to a private registry
func (m *MyModule) Publish(
	ctx context.Context,
	// Registry address
	registry string,
	// Registry username
	username string,
	// Registry password
	password *dagger.Secret,
) ([]string, error) {
	tags := [4]string{"latest", "1.0-alpine", "1.0", "1.0.0"}
	addr := []string{}
	ctr := dag.Container().
		From("nginx:1.23-alpine").
		WithNewFile(
			"/usr/share/nginx/html/index.html",
			"Hello from Dagger!",
			dagger.ContainerWithNewFileOpts{Permissions: 0o400},
		).
		WithRegistryAuth(registry, username, password)

	for _, tag := range tags {
		a, err := ctr.Publish(ctx, fmt.Sprintf("%s/%s/my-nginx:%s", registry, username, tag))
		if err != nil {
			return addr, err
		}
		addr = append(addr, a)
	}
	return addr, nil
}

Examples​

Tag and publish a just-in-time container image to Docker Hub, using the account username user and the password set in the PASSWORD environment variable:

dagger -c 'publish docker.io user env://PASSWORD'

Tag and publish a just-in-time container image to GitHub Container Registry, using the account username user and the GitHub personal access token set in the PASSWORD environment variable:

dagger -c 'publish ghcr.io user env://PASSWORD'

Export a container image to the host​

The following Dagger Function returns a just-in-time container. This can be exported to the host as an OCI tarball.

package main

import "dagger/my-module/internal/dagger"

type MyModule struct{}

// Return a container
func (m *MyModule) Base() *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		WithExec([]string{"mkdir", "/src"}).
		WithExec([]string{"touch", "/src/foo", "/src/bar"})
}

Examples​

Load the container image returned by the Dagger Function into Docker:

dagger -c 'base | export-image myimage'

Load the container image returned by the Dagger Function as a tarball to the host fileysystem:

dagger -c 'base | export /home/admin/mycontainer.tgz'

Set environment variables in a container​

The following Dagger Function demonstrates how to set a single environment variable in a container.

package main

import "context"

type MyModule struct{}

// Set a single environment variable in a container
func (m *MyModule) SetEnvVar(ctx context.Context) (string, error) {
	return dag.Container().
		From("alpine").
		WithEnvVariable("ENV_VAR", "VALUE").
		WithExec([]string{"env"}).
		Stdout(ctx)
}

The following Dagger Function demonstrates how to set multiple environment variables in a container.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

type EnvVar struct {
	Name  string
	Value string
}

// Set environment variables in a container
func (m *MyModule) SetEnvVars(ctx context.Context) (string, error) {
	return dag.Container().
		From("alpine").
		With(EnvVariables([]*EnvVar{
			{"ENV_VAR_1", "VALUE 1"},
			{"ENV_VAR_2", "VALUE 2"},
			{"ENV_VAR_3", "VALUE 3"},
		})).
		WithExec([]string{"env"}).
		Stdout(ctx)
}

func EnvVariables(envs []*EnvVar) dagger.WithContainerFunc {
	return func(c *dagger.Container) *dagger.Container {
		for _, e := range envs {
			c = c.WithEnvVariable(e.Name, e.Value)
		}
		return c
	}
}

Example​

Set a single environment variable in a container:

dagger -c set-env-var

Set multiple environment variables in a container:

dagger -c set-env-vars

Filesystems​

Manage files, directories, and artifacts across your workflows. This section contains practical examples for working with files and directories in Dagger.

Clone a remote Git repository into a container​

The following Dagger Function accepts a Git repository URL and a Git reference. It copies the repository at the specified reference to the /src path in a container and returns the modified container.

note

For examples of SSH-based cloning, including private or public Git repositories with an SSH reference format, select the SSH tabs below. This approach requires explicitly forwarding the host SSH_AUTH_SOCK to the Dagger Function. Learn more about this in Dagger's sandboxed runtime model.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

// Demonstrates cloning a Git repository over HTTP(S).
//
// For SSH usage, see the SSH snippet (CloneWithSsh).
type MyModule struct{}

func (m *MyModule) Clone(ctx context.Context, repository string, ref string) *dagger.Container {
	d := dag.Git(repository).Ref(ref).Tree()

	return dag.Container().
		From("alpine:latest").
		WithDirectory("/src", d).
		WithWorkdir("/src")
}

Examples

Clone the public dagger/dagger GitHub repository to /src in the container:

dagger -c 'clone https://github.com/dagger/dagger 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5'

Clone the public dagger/dagger GitHub repository at reference 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5 to /src in the container and open an interactive terminal to inspect the container filesystem:

dagger <<EOF
clone https://github.com/dagger/dagger 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5 |
  terminal
EOF

Clone over SSH with socket forwarding:

dagger <<EOF
clone-with-ssh git@github.com:dagger/dagger.git 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5 $SSH_AUTH_SOCK |
  terminal
EOF

Mount or copy a directory or remote repository to a container​

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or from a remote Git repository. It mounts the specified directory to the /src path in a container and returns the modified container.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a mounted directory
func (m *MyModule) MountDirectory(
	ctx context.Context,
	// Source directory
	source *dagger.Directory,
) *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		WithMountedDirectory("/src", source)
}

An alternative option is to copy the target directory in the container. The difference between these two approaches is that mounts only take effect within your workflow invocation; they are not copied to, or included, in the final image. In addition, any changes to mounted files and/or directories will only be reflected in the target directory and not in the mount sources.

tip

Besides helping with the final image size, mounts are more performant and resource-efficient. The rule of thumb should be to always use mounts where possible.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a specified directory
func (m *MyModule) CopyDirectory(
	ctx context.Context,
	// Source directory
	source *dagger.Directory,
) *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		WithDirectory("/src", source)
}

Examples

  • Mount the /myapp host directory to /src in the container and return the modified container:

    dagger -c 'mount-directory ./myapp/'

  • Mount the public dagger/dagger GitHub repository to /src in the container and return the modified container:

    dagger -c 'mount-directory https://github.com/dagger/dagger#main'

  • Mount the private user/foo GitHub repository to /src in the container and return the modified container:

    dagger -c 'mount-directory ssh://git@github.com/user/foo#main'

  • Mount the public dagger/dagger GitHub repository to /src in the container and list the contents of the directory:

    dagger -c 'mount-directory https://github.com/dagger/dagger#main | directory /src | entries'

  • Copy the /myapp host directory to /src in the container and return the modified container:

    dagger -c 'copy-directory ./myapp/'

  • Copy the public dagger/dagger GitHub repository to /src in the container and return the modified container:

    dagger -c 'copy-directory https://github.com/dagger/dagger#main'

  • Copy the private user/foo GitHub repository to /src in the container and return the modified container:

    dagger -c 'copy-directory ssh://git@github.com/user/foo#main'

  • Copy the public dagger/dagger GitHub repository to /src in the container and list the contents of the directory:

    dagger -c 'copy-directory https://github.com/dagger/dagger#main | directory /src | entries'

Modify a copied directory or remote repository in a container​

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or from a remote Git repository. It copies the specified directory to the /src path in a container, adds a file to it, and returns the modified container.

warning

When a host directory or file is copied or mounted to a container's filesystem, modifications made to it in the container do not automatically transfer back to the host.

Data flows only one way between Dagger operations, because they are connected in a DAG. To transfer modifications back to the local host, you must explicitly export the directory or file back to the host filesystem.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a specified directory and an additional file
func (m *MyModule) CopyAndModifyDirectory(
	ctx context.Context,
	// Source directory
	source *dagger.Directory,
) *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		WithDirectory("/src", source).
		WithExec([]string{"/bin/sh", "-c", `echo foo > /src/foo`})
}

Examples

Copy the /myapp host directory to /src in the container, add a file to it, and return the modified container:

dagger -c 'copy-and-modify-directory ./myapp/'

Copy the public dagger/dagger GitHub repository to /src in the container, add a file to it, and return the modified container:

dagger -c 'copy-and-modify-directory github.com/dagger/dagger#main'

Copy the private user/foo GitHub repository to /src in the container, add a file to it, and return the modified container:

dagger -c 'copy-and-modify-directory ssh://git@github.com/user/foo#main'

Copy the public dagger/dagger GitHub repository to /src in the container, add a file to it, and list the contents of the directory:

dagger -c 'copy-and-modify-directory https://github.com/dagger/dagger#main | directory /src | entries'

Copy a subset of a directory or remote repository to a container using filters specified at run-time​

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or a remote Git repository. It copies the specified directory to the /src path in a container, using a filter pattern specified at call-time to exclude specified sub-directories and files, and returns the modified container.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

note

This is an example of post-call filtering with directory filters.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a filtered directory
func (m *MyModule) CopyDirectoryWithExclusions(
	ctx context.Context,
	// Source directory
	source *dagger.Directory,
	// Exclusion pattern
	// +optional
	exclude []string,
) *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		WithDirectory("/src", source, dagger.ContainerWithDirectoryOpts{Exclude: exclude})
}

Examples

Copy the current host directory to /src in the container, excluding all sub-directories and files starting with dagger, and return the modified container:

dagger -c 'copy-directory-with-exclusions . --exclude=dagger*'

Copy the public dagger/dagger GitHub repository to /src in the container, excluding all Markdown files, and list the contents of the directory:

dagger <<EOF
copy-directory-with-exclusions https://github.com/dagger/dagger#main --exclude=*.md |
  directory /src |
  entries
EOF

Copy the private user/foo GitHub repository to /src in the container, excluding all Markdown files, and list the contents of the directory:

dagger <<EOF
copy-directory-with-exclusions ssh://git@github.com/user/foo#main --exclude=*.md |
  directory /src |
  entries
EOF

Copy a subset of a directory or remote repository to a container using pre-defined filters​

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or a remote Git repository. It copies the specified directory to the /src path in a container, using pre-defined filter patterns to exclude specified sub-directories and files, and returns the modified container.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

note

This is an example of pre-call filtering with directory filters.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a filtered directory
func (m *MyModule) CopyDirectoryWithExclusions(
	ctx context.Context,
	// +ignore=["*", "!**/*.md"]
	source *dagger.Directory,
) (*dagger.Container, error) {
	return dag.
		Container().
		From("alpine:latest").
		WithDirectory("/src", source).
		Sync(ctx)
}

Examples

Copy the specified host directory to /src in the container, excluding everything except Markdown files, and return the modified container:

dagger -c 'copy-directory-with-exclusions ../docs'

Copy the public dagger/dagger GitHub repository to /src in the container, excluding everything except Markdown files, and list the contents of the /src directory in the container:

dagger -c 'copy-directory-with-exclusions https://github.com/dagger/dagger#main | directory /src | entries'

Mount or copy a local or remote file to a container​

The following Dagger Function accepts a File argument, which could reference either a file from the local filesystem or from a remote Git repository. It mounts the specified file to a container in the /src/ directory and returns the modified container.

package main

import (
	"context"
	"fmt"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a mounted file
func (m *MyModule) MountFile(
	ctx context.Context,
	// Source file
	f *dagger.File,
) *dagger.Container {
	name, _ := f.Name(ctx)
	return dag.Container().
		From("alpine:latest").
		WithMountedFile(fmt.Sprintf("/src/%s", name), f)
}

An alternative option is to copy the target file to the container. The difference between these two approaches is that mounts only take effect within your workflow invocation; they are not copied to, or included, in the final image. In addition, any changes to mounted files and/or directories will only be reflected in the target directory and not in the mount sources.

tip

Besides helping with the final image size, mounts are more performant and resource-efficient. The rule of thumb should be to always use mounts where possible.

package main

import (
	"context"
	"fmt"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a specified file
func (m *MyModule) CopyFile(
	ctx context.Context,
	// Source file
	f *dagger.File,
) *dagger.Container {
	name, _ := f.Name(ctx)
	return dag.Container().
		From("alpine:latest").
		WithFile(fmt.Sprintf("/src/%s", name), f)
}

Examples

Mount the /home/admin/archives.zip file on the host to the /src directory in the container and return the modified container:

dagger -c 'mount-file /home/admin/archives.zip'

Mount the README.md file from the public dagger/dagger GitHub repository to the /src directory in the container:

dagger -c 'mount-file https://github.com/dagger/dagger.git#main:README.md'

Mount the README.md file from the public dagger/dagger GitHub repository to the /src directory in the container and display its contents:

dagger <<EOF
mount-file https://github.com/dagger/dagger.git#main:README.md |
  file /src/README.md |
  contents
EOF

Copy the /home/admin/archives.zip file on the host to the /src directory in the container and return the modified container:

dagger -c 'copy-file /home/admin/archives.zip'

Copy the /home/admin/archives.zip file on the host to the /src directory in the container and list the contents of the directory:

dagger -c 'copy-file /home/admin/archives.zip | directory /src | entries'

Copy the README.md file from the public dagger/dagger GitHub repository to the /src directory in the container:

dagger -c 'copy-file https://github.com/dagger/dagger.git#main:README.md'

Copy the README.md file from the public dagger/dagger GitHub repository to the /src directory in the container and display its contents:

dagger <<EOF
copy-file https://github.com/dagger/dagger.git#main:README.md |
  file /src/README.md |
  contents
EOF

Copy a file to the Dagger module runtime container for custom processing​

The following Dagger Function accepts a File argument and copies the specified file to the Dagger module runtime container. This makes it possible to add one or more files to the runtime container and manipulate them using custom logic.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
	"fmt"
	"os"
)

type MyModule struct{}

// Copy a file to the Dagger module runtime container for custom processing
func (m *MyModule) CopyFile(ctx context.Context, source *dagger.File) {
	source.Export(ctx, "foo.txt")
	// your custom logic here
	// for example, read and print the file in the Dagger Engine container
	fmt.Println(os.ReadFile("foo.txt"))
}

Examples

Copy the data.json host file to the runtime container and process it:

dagger -c 'copy-file ../data.json'

Export a directory or file to the host​

The following Dagger Functions return a just-in-time directory and file. These outputs can be exported to the host with dagger call ... export ....

note

When a host directory or file is copied or mounted to a container's filesystem, modifications made to it in the container do not automatically transfer back to the host. Data flows only one way between Dagger operations, because they are connected in a DAG. To transfer modifications back to the local host, you must explicitly export the directory or file back to the host filesystem.

package main

import "dagger/my-module/internal/dagger"

type MyModule struct{}

// Return a directory
func (m *MyModule) GetDir() *dagger.Directory {
	return m.Base().
		Directory("/src")
}

// Return a file
func (m *MyModule) GetFile() *dagger.File {
	return m.Base().
		File("/src/foo")
}

// Return a base container
func (m *MyModule) Base() *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		WithExec([]string{"mkdir", "/src"}).
		WithExec([]string{"touch", "/src/foo", "/src/bar"})
}

Examples​

  • Export the directory returned by the Dagger Function to the /home/admin/export path on the host:

    dagger -c 'get-dir | export /home/admin/export'

  • Export the file returned by the Dagger Function to the /home/admin/myfile path on the host:

    dagger -c 'get-file | export /home/admin/myfile'

Request a file over HTTP/HTTPS and save it in a container​

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) ReadFileHttp(
	ctx context.Context,
	url string,
) *dagger.Container {
	file := dag.HTTP(url)
	return dag.Container().
		From("alpine:latest").
		WithFile("/src/myfile", file)
}

Examples

Request the README.md file from the public dagger/dagger GitHub repository over HTTPS, save it as /src/myfile in the container, and return the container:

dagger -c 'read-file-http https://raw.githubusercontent.com/dagger/dagger/refs/heads/main/README.md'

Request the README.md file from the public dagger/dagger GitHub repository over HTTPS, save it as /src/myfile in the container, and display its contents:

dagger <<EOF
read-file-http https://raw.githubusercontent.com/dagger/dagger/refs/heads/main/README.md |
  file /src/myfile |
  contents
EOF

Set a module-wide default path​

The following Dagger module uses a constructor to set a module-wide Directory argument and point it to a default path on the host. This eliminates the need to pass a common Directory argument individually to each Dagger Function in the module. If required, the default path can be overridden by specifying a different Directory value at call time.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct {
	Source *dagger.Directory
}

func New(
	// +defaultPath="."
	source *dagger.Directory,
) *MyModule {
	return &MyModule{
		Source: source,
	}
}

func (m *MyModule) Foo(ctx context.Context) ([]string, error) {
	return dag.Container().
		From("alpine:latest").
		WithMountedDirectory("/app", m.Source).
		Directory("/app").
		Entries(ctx)
}

Examples

Call the Dagger Function without arguments. The default path (the current directory .) is used:

dagger -c foo

Call the Dagger Function with a constructor argument. The specified directory (/src/myapp) is used:

dagger -c 'my-module --source $(host | directory /src/myapp) | foo'

Services​

Integrate external services into your workflows (databases, caches, APIs). This section contains practical examples for working with services in Dagger.

Bind and use services in Dagger Functions​

The first Dagger Function below creates and returns an HTTP service. This service is bound and used from a different Dagger Function, via a service binding using an alias like www.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Start and return an HTTP service
func (m *MyModule) HttpService() *dagger.Service {
	return dag.Container().
		From("python").
		WithWorkdir("/srv").
		WithNewFile("index.html", "Hello, world!").
		WithExposedPort(8080).
		AsService(dagger.ContainerAsServiceOpts{Args: []string{"python", "-m", "http.server", "8080"}})
}

// Send a request to an HTTP service and return the response
func (m *MyModule) Get(ctx context.Context) (string, error) {
	return dag.Container().
		From("alpine").
		WithServiceBinding("www", m.HttpService()).
		WithExec([]string{"wget", "-O-", "http://www:8080"}).
		Stdout(ctx)
}

Example​

Send a request from one Dagger Function to a bound HTTP service instantiated by a different Dagger Function:

dagger -c get

Expose services in Dagger Functions to the host​

The Dagger Function below creates and returns an HTTP service. This service can be used from the host.

package main

import (
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Start and return an HTTP service
func (m *MyModule) HttpService() *dagger.Service {
	return dag.Container().
		From("python").
		WithWorkdir("/srv").
		WithNewFile("index.html", "Hello, world!").
		WithExposedPort(8080).
		AsService(dagger.ContainerAsServiceOpts{Args: []string{"python", "-m", "http.server", "8080"}})
}

Examples​

  • Expose the HTTP service instantiated by a Dagger Function to the host on the default port:
dagger -c 'http-service | up'

Access the service from the host:

curl localhost:8080
  • Expose the HTTP service instantiated by a Dagger Function to the host on a different host port:
dagger -c 'http-service | up --ports 9000:8080'

Access the service from the host:

curl localhost:9000

Expose host services to Dagger Functions​

The following Dagger Function accepts a Service running on the host, binds it using an alias, and creates a client to access it via the service binding. This example uses a MariaDB database service running on host port 3306, aliased as db in the Dagger Function.

note

This implies that a service is already listening on a port on the host, out-of-band of Dagger.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Send a query to a MariaDB service and return the response
func (m *MyModule) UserList(
	ctx context.Context,
	// Host service
	svc *dagger.Service,
) (string, error) {
	return dag.Container().
		From("mariadb:10.11.2").
		WithServiceBinding("db", svc).
		WithExec([]string{"/usr/bin/mysql", "--user=root", "--password=secret", "--host=db", "-e", "SELECT Host, User FROM mysql.user"}).
		Stdout(ctx)
}

Example​

Send a query to the database service listening on host port 3306 and return the result as a string:

dagger -c 'user-list tcp://localhost:3306'

Use service endpoints​

The following Dagger Function starts a service manually, then retrieves its endpoint and sends a request. This example uses a NGINX HTTP service running on host port 80.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Get(ctx context.Context) (string, error) {
	// Start NGINX service
	service := dag.Container().From("nginx").WithExposedPort(80).AsService()
	service, err := service.Start(ctx)
	if err != nil {
		return "", err
	}

	// Wait for service endpoint
	endpoint, err := service.Endpoint(ctx, dagger.ServiceEndpointOpts{Scheme: "http", Port: 80})
	if err != nil {
		return "", err
	}

	// Send HTTP request to service endpoint
	return dag.HTTP(endpoint).Contents(ctx)
}

Example​

Send a query to the HTTP service listening on host port 80 and return the result as a string:

dagger -c get

Create a transient service for unit tests​

The following Dagger Function creates a service and binds it to an application container for unit testing. In this example, the application being tested is Drupal. Drupal includes a large number of unit tests, including tests which depend on an active database service. This database service is created on-the-fly by the Dagger Function.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Run unit tests against a database service
func (m *MyModule) Test(ctx context.Context) (string, error) {
	mariadb := dag.Container().
		From("mariadb:10.11.2").
		WithEnvVariable("MARIADB_USER", "user").
		WithEnvVariable("MARIADB_PASSWORD", "password").
		WithEnvVariable("MARIADB_DATABASE", "drupal").
		WithEnvVariable("MARIADB_ROOT_PASSWORD", "root").
		WithExposedPort(3306).
		AsService(dagger.ContainerAsServiceOpts{UseEntrypoint: true})

	// get Drupal base image
	// install additional dependencies
	drupal := dag.Container().
		From("drupal:10.0.7-php8.2-fpm").
		WithExec([]string{"composer", "require", "drupal/core-dev", "--dev", "--update-with-all-dependencies"})

	// add service binding for MariaDB
	// run kernel tests using PHPUnit
	return drupal.
		WithServiceBinding("db", mariadb).
		WithEnvVariable("SIMPLETEST_DB", "mysql://user:password@db/drupal").
		WithEnvVariable("SYMFONY_DEPRECATIONS_HELPER", "disabled").
		WithWorkdir("/opt/drupal/web/core").
		WithExec([]string{"../../vendor/bin/phpunit", "-v", "--group", "KernelTests"}).
		Stdout(ctx)
}

Example​

Run Drupal's unit tests, instantiating a database service during the process:

dagger -c test

Start and stop services​

The following Dagger Function demonstrates how to control a service's lifecycle by explicitly starting and stopping a service. This example uses a Redis service.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Explicitly start and stop a Redis service
func (m *MyModule) RedisService(ctx context.Context) (string, error) {
	redisSrv := dag.Container().
		From("redis").
		WithExposedPort(6379).
		AsService(dagger.ContainerAsServiceOpts{UseEntrypoint: true})

	// start Redis ahead of time so it stays up for the duration of the test
	redisSrv, err := redisSrv.Start(ctx)
	if err != nil {
		return "", err
	}

	// stop the service when done
	defer redisSrv.Stop(ctx)

	// create Redis client container
	redisCLI := dag.Container().
		From("redis").
		WithServiceBinding("redis-srv", redisSrv)

	args := []string{"redis-cli", "-h", "redis-srv"}

	// set value
	setter, err := redisCLI.
		WithExec(append(args, "set", "foo", "abc")).
		Stdout(ctx)
	if err != nil {
		return "", err
	}

	// get value
	getter, err := redisCLI.
		WithExec(append(args, "get", "foo")).
		Stdout(ctx)
	if err != nil {
		return "", err
	}

	return setter + getter, nil
}

Example​

Start and stop a Redis service:

dagger -c redis-service

Create interdependent services​

The following Dagger Function runs two services, service A and service B, that depend on each other. The services are set up with custom hostnames, svca and svcb, allowing each service to communicate with the other by hostname.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Run two services which are dependent on each other
func (m *MyModule) Services(ctx context.Context) (*dagger.Service, error) {

	svcA := dag.Container().From("nginx").
		WithExposedPort(80).
		AsService(dagger.ContainerAsServiceOpts{Args: []string{"sh", "-c", `nginx & while true; do curl svcb:80 && sleep 1; done`}}).
		WithHostname("svca")

	_, err := svcA.Start(ctx)
	if err != nil {
		return nil, err
	}

	svcB := dag.Container().From("nginx").
		WithExposedPort(80).
		AsService(dagger.ContainerAsServiceOpts{Args: []string{"sh", "-c", `nginx & while true; do curl svca:80 && sleep 1; done`}}).
		WithHostname("svcb")

	svcB, err = svcB.Start(ctx)
	if err != nil {
		return nil, err
	}

	return svcB, nil
}

Example​

Start two inter-dependent services:

dagger -c 'services | up --ports 8080:80'

Secrets​

Securely handle sensitive data in your workflows. This section contains practical examples for working with secrets in Dagger.

Use secret variables​

The following Dagger Function accepts a GitHub personal access token as a Secret, and uses the Secret to authorize a request to the GitHub API. The secret may be sourced from the host (via an environment variable, host file, or host command) or from an external secrets manager (1Password or Vault):

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Query the GitHub API
func (m *MyModule) GithubApi(
	ctx context.Context,
	// GitHub API token
	token *dagger.Secret,
) (string, error) {
	return dag.Container().
		From("alpine:3.17").
		WithSecretVariable("GITHUB_API_TOKEN", token).
		WithExec([]string{"apk", "add", "curl"}).
		WithExec([]string{"sh", "-c", `curl "https://api.github.com/repos/dagger/dagger/issues" --header "Accept: application/vnd.github+json" --header "Authorization: Bearer $GITHUB_API_TOKEN"`}).
		Stdout(ctx)
}

Using Environment Variables​

You can use a secret sourced from an environment variable by running the following command:

dagger -c 'github-api env://GITHUB_API_TOKEN'

Passing Files​

You can also pass files to secrets by following the example below:

dagger -c 'github-api file://./github.txt'

Secrets from command output​

Secrets also support capturing data from running commands.

dagger -c 'github-api cmd://"gh auth token"'

1Password​

Use a secret from 1Password:

note

If using a 1Password service account, ensure that the OP_SERVICE_ACCOUNT_TOKEN environment variable is set.

export OP_SERVICE_ACCOUNT_TOKEN="mytoken"
dagger -c 'github-api op://infra/github/credential'

Hashicorp Vault​

You can retrieve secrets from Hashicorp Vault.

note

Ensure that the VAULT_ADDR and either the VAULT_TOKEN or VAULT_APPROLE_ROLE_ID (for Vault AppRole authentication) environment variables are set.

export VAULT_ADDR="https://127.0.0.1:8200"
export VAULT_TOKEN="gue55me7"
export VAULT_APPROLE_ROLE_ID="roleid-xxx-yyy-zzz"
dagger -c 'github-api vault://credentials.github'

Mount files as secrets​

The following Dagger Function accepts a GitHub hosts configuration file as a Secret, and mounts the file as a Secret to a container to authorize a request to GitHub.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Query the GitHub API
func (m *MyModule) GithubAuth(
	ctx context.Context,
	// GitHub Hosts configuration file
	ghCreds *dagger.Secret,
) (string, error) {
	return dag.Container().
		From("alpine:3.17").
		WithExec([]string{"apk", "add", "github-cli"}).
		WithMountedSecret("/root/.config/gh/hosts.yml", ghCreds).
		WithWorkdir("/root").
		WithExec([]string{"gh", "auth", "status"}).
		Stdout(ctx)
}

Mounting files​

To mount a file as a secret, you can use the following:

dagger -c 'github-auth file://$HOME/.config/gh/hosts.yml'

Use secret in Dockerfile build​

The following code listing demonstrates how to inject a secret into a Dockerfile build. The secret is automatically mounted in the build container at /run/secrets/SECRET-ID.

package main

import (
	"context"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Build a Container from a Dockerfile
func (m *MyModule) Build(
	ctx context.Context,
	// The source code to build
	source *dagger.Directory,
	// The secret to use in the Dockerfile
	secret *dagger.Secret,
) (*dagger.Container, error) {
	// Ensure the Dagger secret's name matches what the Dockerfile
	// expects as the id for the secret mount.
	secretVal, err := secret.Plaintext(ctx)
	if err != nil {
		return nil, err
	}
	buildSecret := dag.SetSecret("gh-secret", secretVal)

	return source.
		DockerBuild(dagger.DirectoryDockerBuildOpts{
			Secrets: []*dagger.Secret{buildSecret},
		}), nil
}

The sample Dockerfile below demonstrates the process of mounting the secret using a secret filesystem mount type and using it in the Dockerfile build process:

FROM alpine:3.17
RUN apk add curl
RUN --mount=type=secret,id=gh-secret \
    curl "https://api.github.com/repos/dagger/dagger/issues" \
        --header "Accept: application/vnd.github+json" \
        --header "Authorization: Bearer $(cat /run/secrets/gh-secret)"

Dockerfile build with mounted secret​

Build from a Dockerfile with a mounted secret from the host environment:

dagger -c 'build . env://GITHUB_API_TOKEN'

LLMs and Environments​

Build AI-powered workflows and agents with LLM integration. This section contains practical examples for working with LLMs and environments in Dagger.

Use string inputs and outputs​

The following Dagger Function creates an environment with a string input and a string output. The input is a question, and the output represents the answer. The environment is given to an LLM with a prompt describing how it should complete its work.

package main

import (
	"context"
)

type MyModule struct{}

func (m *MyModule) Agent(
	ctx context.Context,
	question string,
) (string, error) {
	environment := dag.Env().
		WithStringInput("question", question, "the question").
		WithStringOutput("answer", "the answer to the question")

	work := dag.LLM().
		WithEnv(environment).
		WithPrompt(`
			You are an assistant that helps with complex questions.
			You will receive a question and you need to provide a detailed answer.
			Make sure to use the provided context and environment variables.
			Your answer should be clear and concise.
			Your question is: $question
			`)

	return work.
		Env().
		Output("answer").
		AsString(ctx)
}

Example​

Ask a question:

dagger -c 'agent "what is an ion"'

Use container inputs and outputs​

The following Dagger Function creates an environment with a container input and a container output. The input is a base alpine container, and the output is the same container, updated with additional libraries and tools. The environment is given to an LLM with a prompt describing how it should update the container.

package main

import (
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Agent() *dagger.Container {
	base := dag.Container().From("alpine:latest")
	environment := dag.Env().
		WithContainerInput("base", base, "a base container to use").
		WithContainerOutput("result", "the updated container")

	work := dag.LLM().
		WithEnv(environment).
		WithPrompt(`
			You are a software engineer with deep knowledge of Web application development.
			You have access to a container.
			Install the necessary tools and libraries to create a
			complete development environment for Web applications.
			Once complete, return the updated container.
			`)

	return work.
		Env().
		Output("result").
		AsContainer()
}

Example​

Have the agent update the container and open an interactive terminal to inspect the container filesystem:

dagger -c 'agent | terminal'

Use directory inputs and outputs​

The following Dagger Function creates an environment with a directory input and a directory output. The input is a directory containing various files, and the output is a different directory containing translations of the original files. The environment is given to an LLM with a prompt describing how it should complete its work.

package main

import (
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Agent() *dagger.Directory {
	dir := dag.Git("github.com/dagger/dagger").Branch("main").Tree()
	environment := dag.Env().
		WithDirectoryInput("source", dir, "the source directory to use").
		WithDirectoryOutput("result", "the updated directory")

	work := dag.LLM().
		WithEnv(environment).
		WithPrompt(`
			You have access to a directory containing various files.
			Translate only the README file in the directory to French and Spanish.
			Ensure that the translations are accurate and maintain the original formatting.
			Do not modify any other files in the directory.
			Create a sub-directory named 'translations' to store the translated files.
			For French, add an 'fr' suffix to the translated file name.
			For Spanish, add an 'es' suffix to the translated file name.
			Do not create any other new files or directories.
			Do not delete any files or directories.
			Do not investigate any sub-directories.
			Once complete, return the 'translations' directory.
			`)

	return work.
		Env().
		Output("result").
		AsDirectory()
}

Example​

Have the agent create the translations and export the directory of translated files to the host:

dagger -c 'agent | export /tmp/out'

Combine inputs and outputs of multiple types​

The following Dagger Function creates an environment with two inputs (a container and a directory) and one output (a file). The input container is a base golang container, the input directory is a Git repository containing Golang source code, and the output is the built Go executable. The environment is given to an LLM with a prompt describing how it should work.

package main

import (
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Agent() *dagger.File {
	dir := dag.Git("github.com/golang/example").Branch("master").Tree().Directory("/hello")
	builder := dag.Container().From("golang:latest")

	environment := dag.Env().
		WithContainerInput("container", builder, "a Golang container").
		WithDirectoryInput("directory", dir, "a directory with source code").
		WithFileOutput("file", "the built Go executable")

	work := dag.LLM().
		WithEnv(environment).
		WithPrompt(`
			You have access to a Golang container.
			You also have access to a directory containing Go source code.
			Mount the directory into the container and build the Go application.
			Once complete, return only the built binary.
			`)

	return work.
		Env().
		Output("file").
		AsFile()
}

Example​

Have the agent build the Go executable and export it to the host:

dagger -c 'agent | export /tmp/out/myfile'

Build a Golang coding agent​

The following Dagger Function creates a Golang coding agent. The input is a programming assignment and the output is a Go executable satisfying the assignment. The environment consists of a base golang container, which is given to an LLM with a prompt describing how it should perform the work.

package main

import (
	"dagger/coding-agent/internal/dagger"
)

type CodingAgent struct{}

// Write a Go program
func (m *CodingAgent) GoProgram(
	// The programming assignment, e.g. "write me a curl clone"
	assignment string,
) *dagger.Container {
	environment := dag.Env().
		WithStringInput("assignment", assignment, "the assignment to complete").
		WithContainerInput("builder",
			dag.Container().From("golang").WithWorkdir("/app"),
			"a container to use for building Go code").
		WithContainerOutput("completed", "the completed assignment in the Golang container")

	work := dag.LLM().
		WithEnv(environment).
		WithPrompt(`
			You are an expert Go programmer with an assignment to create a Go program
			Create files in the default directory in $builder
			Always build the code to make sure it is valid
			Do not stop until your assignment is completed and the code builds
			Your assignment is: $assignment
			`)

	return work.
		Env().
		Output("completed").
		AsContainer()
}

Example​

Have the agent update the container and open an interactive terminal to inspect the container filesystem:

dagger -c 'agent | terminal'

Errors and Debugging​

Handle errors gracefully and build resilient workflows. This section contains practical examples for handling and debugging errors in Dagger workflows.

Terminate gracefully​

The following Dagger Function demonstrates how to handle errors in a workflow.

package main

import (
	"context"
	"errors"
	"fmt"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Generate an error
func (m *MyModule) Test(ctx context.Context) (string, error) {
	out, err := dag.
		Container().
		From("alpine").
		// ERROR: cat: read error: Is a directory
		WithExec([]string{"cat", "/"}).
		Stdout(ctx)
	var e *dagger.ExecError
	if errors.As(err, &e) {
		return fmt.Sprintf("Test pipeline failure: %s", e.Stderr), nil
	} else if err != nil {
		return "", err
	}
	return out, nil
}

Example​

Execute a Dagger Function which creates a container and runs a command in it. If the command fails, the error is captured and the Dagger Function is gracefully terminated with a custom error message.

dagger -c test

Continue using a container after command execution fails​

The following Dagger Function demonstrates how to continue using a container after a command executed within it fails. A common use case for this is to export a report that a test suite tool generates.

note

The caveat with this approach is that forcing a zero exit code on a failure caches the failure. This may not be desired depending on the use case.

package main

import (
	"context"
	"fmt"

	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

var script = `#!/bin/sh
echo "Test Suite"
echo "=========="
echo "Test 1: PASS" | tee -a report.txt
echo "Test 2: FAIL" | tee -a report.txt
echo "Test 3: PASS" | tee -a report.txt
exit 1
`

type TestResult struct {
	Report   *dagger.File
	ExitCode int
}

// Handle errors
func (m *MyModule) Test(ctx context.Context) (*TestResult, error) {
	ctr, err := dag.
		Container().
		From("alpine").
		// add script with execution permission to simulate a testing tool
		WithNewFile("/run-tests", script, dagger.ContainerWithNewFileOpts{Permissions: 0o750}).
		// run-tests but allow any return code
		WithExec([]string{"/run-tests"}, dagger.ContainerWithExecOpts{Expect: dagger.ReturnTypeAny}).
		// the result of `sync` is the container, which allows continued chaining
		Sync(ctx)
	if err != nil {
		// unexpected error, could be network failure.
		return nil, fmt.Errorf("run tests: %w", err)
	}
	// save report for inspection.
	report := ctr.File("report.txt")

	// use the saved exit code to determine if the tests passed.
	exitCode, err := ctr.ExitCode(ctx)
	if err != nil {
		// exit code not found
		return nil, fmt.Errorf("get exit code: %w", err)
	}

	// Return custom type
	return &TestResult{
		Report:   report,
		ExitCode: exitCode,
	}, nil
}

Example​

Continue executing a Dagger Function even after a command within it fails. The Dagger Function returns a custom TestResult object containing a test report and the exit code of the failed command.

Obtain the exit code:

dagger -c 'test | exit-code'

Obtain the report:

dagger -c 'test | report | contents'

Debug workflows with the interactive terminal​

Dagger provides two features that can help greatly when trying to debug a workflow - opening an interactive terminal session at the failure point, or at explicit breakpoints throughout your workflow code. All context is available at the point of failure. Multiple terminals are supported in the same Dagger Function; they will open in sequence.

The following Dagger Function opens an interactive terminal session at different stages in a Dagger workflow to debug a container build.

package main

import (
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Container() *dagger.Container {
	return dag.Container().
		From("alpine:latest").
		Terminal().
		WithExec([]string{"sh", "-c", "echo hello world > /foo && cat /foo"}).
		Terminal()
}

Example​

Execute a Dagger Function to build a container, and open an interactive terminal at two different points in the build process. The interactive terminal enables you to inspect the container filesystem and environment "live", during the build process.

dagger -c container

Inspect directories and files​

The following Dagger Function clones Dagger's GitHub repository and opens an interactive terminal session to inspect it. Under the hood, this creates a new container (defaults to alpine) and starts a shell, mounting the repository directory inside.

package main

import (
  "context"
)

type MyModule struct{}

func (m *MyModule) SimpleDirectory(ctx context.Context) (string, error) {
	return dag.
		Git("https://github.com/dagger/dagger.git").
		Head().
		Tree().
		Terminal().
		File("README.md").
		Contents(ctx)
}

The container created to mount the directory can be customized using additional options. The following Dagger Function revised the previous example to demonstrate this, using an ubuntu container image and bash shell instead of the defaults.

package main

import (
	"context"
	"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) AdvancedDirectory(ctx context.Context) (string, error) {
	return dag.
		Git("https://github.com/dagger/dagger.git").
		Head().
		Tree().
		Terminal(dagger.DirectoryTerminalOpts{
			Container: dag.Container().From("ubuntu"),
			Cmd:       []string{"/bin/bash"},
		}).
		File("README.md").
		Contents(ctx)
}

Example​

  • Execute a Dagger Function to clone Dagger's GitHub repository and open a terminal session in the repository directory:

    dagger -c simple-directory

  • Execute another Dagger Function that does the same as the previous one, except using an ubuntu container image as base and initializing the terminal with the bash shell:

    dagger -c advanced-directory

Create custom spans​

Dagger represents operations performed by a Dagger Function as OpenTelemetry spans. Spans are typically used to separate tasks that are running in parallel, with each branch waiting for completion.

It is possible to instrument custom OpenTelemetry spans inside any Dagger Function. This allows you to define logical boundaries within complex workflows, measure execution time, and track nested operations with greater granularity. These custom spans appear in the Dagger TUI and Traces.

The following Dagger Function demonstrates this by emitting custom spans for various tasks.

warning

The approach described below is experimental and may be deprecated in favor of a new OpenTelemetry span API. Contribute to the ongoing discussion of this topic on GitHub.

package main

import (
	"context"

	"dagger/my-module/internal/telemetry"

	"golang.org/x/sync/errgroup"
)

type MyModule struct{}

func (m *MyModule) Foo(ctx context.Context) error {
	// clone the source code repository
	source := dag.
		Git("https://github.com/dagger/hello-dagger").
		Branch("main").Tree()

	// list versions to test against
	versions := []string{"20", "22", "23"}

	// define errorgroup
	eg := new(errgroup.Group)

	// run tests concurrently
	// emit a span for each
	for _, version := range versions {
		eg.Go(func() (rerr error) {
			ctx, span := Tracer().Start(ctx, "running unit tests with Node "+version)
			defer telemetry.End(span, func() error { return rerr })
			_, err := dag.Container().
				From("node:"+version).
				WithDirectory("/src", source).
				WithWorkdir("/src").
				WithExec([]string{"npm", "install"}).
				WithExec([]string{"npm", "run", "test:unit", "run"}).
				Sync(ctx)
			return err
		})
	}

	return eg.Wait()
}
warning

When using spans to group and measure Dagger API function calls, ensure that the function calls are not lazily evaluated. The duration of the corresponding spans in this case will always be zero.

Example​

Execute a Dagger Function to run unit tests on the dagger/hello-dagger source code repository with different versions of Node.js, emitting a custom span for each version tested:

dagger call foo

Need Something Else?​

Can't find what you're looking for? Check out:

Have a recipe idea? Contribute to the cookbook!