Skip to main content

Use Secrets in Dagger

Introduction

Dagger allows you to utilize confidential information, such as passwords, API keys, SSH keys and so on, when running your Dagger pipelines, without exposing those secrets in plaintext logs, writing them into the filesystem of containers you're building, or inserting them into cache.

This tutorial teaches you the basics of using secrets in Dagger.

Requirements

This tutorial assumes that:

  • You have a Go, Python or Node.js development environment. If not, install Go, Python or Node.js.
  • You have a Dagger SDK installed for one of the above languages. If not, follow the installation instructions for the Dagger Go, Python or Node.js SDK.
  • You have Docker installed and running on the host system. If not, install Docker.

Create and use a secret

The Dagger API provides the following queries and fields for working with secrets:

  • The setSecret query creates a new secret from a plaintext value.
  • A Container's withMountedSecret() field returns the container with the secret mounted at the named filesystem path.
  • A Container's withSecretVariable() field returns the container with the secret stored in the named container environment variable.

Once a secret is loaded into Dagger, it can be used in a Dagger pipeline as either a variable or a mounted file. Some Dagger SDK methods additionally accept secrets as native objects.

Let's start with a simple example of setting a secret in a Dagger pipeline and using it in a container.

The following code listing creates a Dagger secret for a GitHub personal access token and then uses the token to authorize a request to the GitHub API. To use this listing, replace the TOKEN placeholder with your personal GitHub access token.

package main

import (
"context"
"fmt"
"os"

"dagger.io/dagger"
)

func main() {
// initialize Dagger client
ctx := context.Background()
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()

// set secret
secret := client.SetSecret("ghApiToken", "TOKEN")

// use secret in container environment
out, err := client.
Container().
From("alpine:3.17").
WithSecretVariable("GITHUB_API_TOKEN", secret).
WithExec([]string{"apk", "add", "curl"}).
WithExec([]string{"sh", "-c", `curl "https://api.github.com/repos/dagger/dagger/issues" --header "Accept: application/vnd.github+json" --header "Authorization: Bearer $GITHUB_API_TOKEN"`}).
Stdout(ctx)
if err != nil {
panic(err)
}

// print result
fmt.Println(out)
}

In this code listing:

  • The client's SetSecret() method accepts two parameters - a unique name for the secret, and the secret value - and returns a new Secret object
  • The WithSecretVariable() method also accepts two parameters - an environment variable name and a Secret object. It returns a container with the secret value assigned to the specified environment variable.
  • The environment variable can then be used in subsequent container operations - for example, in a command-line curl request, as shown above.

Use secrets from the host environment

Most of the time, it's neither practical nor secure to define plaintext secrets directly in your pipeline. That's why Dagger lets you read secrets from the host, either from host environment variables or from the host filesystem, or from external providers.

Here's a revision of the previous example, where the secret is read from a host environment variable. To use this listing, create a host environment variable named GH_SECRET and assign it the value of your GitHub personal access token.

package main

import (
"context"
"fmt"
"os"

"dagger.io/dagger"
)

func main() {
// initialize Dagger client
ctx := context.Background()
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()

// read secret from host variable
secret := client.SetSecret("gh-secret", os.Getenv("GH_SECRET"))

// use secret in container environment
out, err := client.
Container().
From("alpine:3.17").
WithSecretVariable("GITHUB_API_TOKEN", secret).
WithExec([]string{"apk", "add", "curl"}).
WithExec([]string{"sh", "-c", `curl "https://api.github.com/repos/dagger/dagger/issues" --header "Accept: application/vnd.github+json" --header "Authorization: Bearer $GITHUB_API_TOKEN"`}).
Stdout(ctx)
if err != nil {
panic(err)
}

fmt.Println(out)
}

This code listing assumes the existence of a host environment variable named GH_SECRET containing the secret value. It performs the following operations:

  • It reads the value of the host environment variable using the os.Getenv() function.
  • It creates a Dagger secret with that value using the SetSecret() method.
  • It uses the WithSecretVariable() method to return a container with the secret value assigned to an environment variable in the container.
  • It uses the secret in subsequent container operations, as explained previously.

Use secrets from the host filesystem

Dagger also lets you mount secrets as files within a container's filesystem. This is useful for tools that look for secrets in a specific filesystem location - for example, GPG keyrings, SSH keys or file-based authentication tokens.

As an example, consider the GitHub CLI, which reads its authentication token from a file stored in the user's home directory at ~/.config/gh/hosts.yml. The following code listing demonstrates how to mount this file at a specific location in a container as a secret, so that the GitHub CLI is able to find it.

package main

import (
"context"
"fmt"
"os"

"dagger.io/dagger"
)

func main() {
// initialize Dagger client
ctx := context.Background()
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()

// read file
config, err := os.ReadFile("/home/USER/.config/gh/hosts.yml")
if err != nil {
panic(err)
}

// set secret to file contents
secret := client.SetSecret("ghConfig", string(config))

// mount secret as file in container
out, err := client.
Container().
From("alpine:3.17").
WithExec([]string{"apk", "add", "github-cli"}).
WithMountedSecret("/root/.config/gh/hosts.yml", secret).
WithWorkdir("/root").
WithExec([]string{"gh", "auth", "status"}).
Stdout(ctx)
if err != nil {
panic(err)
}

fmt.Println(out)
}

This code listing assumes the existence of a GitHub CLI configuration file containing an authentication token at /home/USER/.config/gh/hosts.yml. It performs the following operations:

  • It reads the host file's contents into a string and loads the string into a secret with the Dagger client's SetSecret() method. This method returns a new Secret object.
  • It uses the WithMountedSecret() method to return a container with the secret mounted as a file at the given location.
  • The GitHub CLI reads this file as needed to perform requested operations. For example, executing the gh auth status command in the Dagger pipeline after mounting the secret returns a message indicating that the user is logged-in, testifying to the success of the secret mount.

Use secrets from an external secret manager

It's also possible to read secrets into Dagger from external secret managers. The following code listing provides an example of using a secret from Google Cloud Secret Manager in a Dagger pipeline.

package main

import (
"context"
"fmt"
"os"

secretmanager "cloud.google.com/go/secretmanager/apiv1"
"cloud.google.com/go/secretmanager/apiv1/secretmanagerpb"
"dagger.io/dagger"
)

func main() {
ctx := context.Background()

// get secret from Google Cloud Secret Manager
secretPlaintext, err := gcpGetSecretPlaintext(ctx, "PROJECT-ID", "SECRET-ID")
if err != nil {
panic(err)
}

// initialize Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()

// load secret into Dagger
secret := client.SetSecret("ghApiToken", string(secretPlaintext))

// use secret in container environment
out, err := client.
Container().
From("alpine:3.17").
WithSecretVariable("GITHUB_API_TOKEN", secret).
WithExec([]string{"apk", "add", "curl"}).
WithExec([]string{"sh", "-c", `curl "https://api.github.com/repos/dagger/dagger/issues" --header "Accept: application/vnd.github+json" --header "Authorization: Bearer $GITHUB_API_TOKEN"`}).
Stdout(ctx)
if err != nil {
panic(err)
}

// print result
fmt.Println(out)
}

func gcpGetSecretPlaintext(ctx context.Context, projectID, secretID string) (string, error) {
secretUri := fmt.Sprintf("projects/%s/secrets/%s/versions/latest", projectID, secretID)

// initialize Google Cloud API client
gcpClient, err := secretmanager.NewClient(ctx)
if err != nil {
panic(err)
}
defer gcpClient.Close()

// retrieve secret
secReq := &secretmanagerpb.AccessSecretVersionRequest{
Name: secretUri,
}

res, err := gcpClient.AccessSecretVersion(ctx, secReq)
if err != nil {
panic(err)
}

secretPlaintext := res.Payload.Data

return string(secretPlaintext), nil
}

This code listing requires the user to replace the PROJECT-ID and SECRET-ID placeholders with corresponding Google Cloud project and secret identifiers. It performs the following operations:

  • It imports the Dagger and Google Cloud Secret Manager client libraries.
  • It uses the Google Cloud Secret Manager client to access and read the specified secret's payload.
  • It creates a Dagger secret with that payload using the SetSecret() method.
  • It uses the WithSecretVariable() method to return a container with the secret value assigned to an environment variable in the container.
  • It uses the secret to make an authenticated request to the GitHub API, as explained previously.

Use secrets with Dagger SDK methods

Secrets can also be used natively as inputs to some Dagger SDK methods. Here's an example, which demonstrates logging in to Docker Hub from a Dagger pipeline and publishing a new image. To use this listing, replace the DOCKER-HUB-USERNAME and DOCKER-HUB-PASSWORD placeholders with your Docker Hub username and password respectively.

package main

import (
"context"
"fmt"
"os"

"dagger.io/dagger"
)

func main() {
// initialize Dagger client
ctx := context.Background()
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()

// set secret as string value
secret := client.SetSecret("password", "DOCKER-HUB-PASSWORD")

// create container
c := client.Container(dagger.ContainerOpts{Platform: "linux/amd64"}).
From("nginx:1.23-alpine").
WithNewFile("/usr/share/nginx/html/index.html", dagger.ContainerWithNewFileOpts{
Contents: "Hello from Dagger!",
Permissions: 0o400,
})

// use secret for registry authentication
addr, err := c.
WithRegistryAuth("docker.io", "DOCKER-HUB-USERNAME", secret).
Publish(ctx, "DOCKER-HUB-USERNAME/my-nginx")
if err != nil {
panic(err)
}

// print result
fmt.Println("Published at:", addr)
}

In this code listing:

  • The client's SetSecret() method returns a new Secret representing the Docker Hub password.
  • The WithRegistryAuth() method accepts three parameters - the Docker Hub registry address, the username and the password (as a Secret) - and returns a container pre-authenticated for the registry.
  • The Publish() method publishes the container to Docker Hub.

Understand how Dagger secures secrets

Dagger automatically scrubs secrets from its various logs and output streams. This ensures that sensitive data does not leak - for example, in the event of a crash. This applies to secrets stored in both environment variables and file mounts.

The following example demonstrates this feature:

package main

import (
"context"
"fmt"
"os"

"dagger.io/dagger"
)

func main() {
ctx := context.Background()

// create Dagger client
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
if err != nil {
panic(err)
}
defer client.Close()

secretEnv := client.SetSecret("my-secret-var", "secret value here")
secretFile := client.SetSecret("my-secret-file", "secret file content here")

// dump secrets to console
out, err := client.Container().
From("alpine:3.17").
WithSecretVariable("MY_SECRET_VAR", secretEnv).
WithMountedSecret("/my_secret_file", secretFile).
WithExec([]string{"sh", "-c", `echo -e "secret env data: $MY_SECRET_VAR || secret file data: "; cat /my_secret_file`}).
Stdout(ctx)
if err != nil {
panic(err)
}

fmt.Println(out)
}

This listing creates dummy secrets on the host (as an environment variable and a file), loads them into Dagger and then attempts to print them to the console. However, Dagger automatically scrubs the sensitive data before printing it, as shown in the output below:

secret env data: *** || secret file data:
***

::danger Any secret that is to be read from the container environment should always be loaded using withSecretVariable(). If withEnvVariable() is used instead, the value of the environment variable may leak via the build history of the container. Using withSecretVariable() guarantees that the secret will not leak in the container build history or image layers. :::

Conclusion

This tutorial walked you through the basics of using secrets in Dagger. It explained the various API methods available to work with secrets and provided examples of using secrets as environment variables, file mounts and native objects.

Use the API Key Concepts page and the Go, Node.js and Python SDK References to learn more about Dagger.