Skip to main content

Get Started with the Dagger Go SDK

Introduction​

This tutorial teaches you the basics of using Dagger in Go. You will learn how to:

  • Install the Go SDK
  • Create a Go CI tool that builds a Go application for multiple architectures and Go versions using the Go SDK

Requirements​

This tutorial assumes that:

note

This tutorial creates a CI tool to build a Go application for multiple architectures. If you don't have a Go application already, clone an existing Go project. A good example is the Go example projects repository, which you can clone as below:

git clone https://go.googlesource.com/example
cd example/hello

The code samples in this tutorial are based on the above Go project. If using a different project, adjust the code samples accordingly.

Step 1: Add the Dagger Go SDK to the project​

note

The Dagger Go SDK requires Go 1.20 or later.

From your existing Go module, install the Dagger Go SDK using the commands below:

go get dagger.io/dagger@latest

Step 2: Create a Go module for the tool​

The next step is to create a new Go module and sub-directory for the tool.

mkdir multibuild && cd multibuild
go mod init multibuild

Step 3: Create a Dagger client in Go​

note

If you would prefer to use the final main.go file right away, it can be found in Step 5

Create a new file in the multibuild directory named main.go and add the following code to it.

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	if err := build(context.Background()); err != nil {
		fmt.Println(err)
	}
}

func build(ctx context.Context) error {
	fmt.Println("Building with Dagger")

	// initialize Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		return err
	}
	defer client.Close()

	return nil
}

This Go CI tool stub imports the Dagger SDK and defines two functions: main(), which provides an interface for the user to pass in an argument to the tool and build(), which defines the pipeline operations.

The build() function creates a Dagger client with dagger.Connect(). This client provides an interface for executing commands against the Dagger engine. This function is sparse to begin with; it will be improved in subsequent steps.

Try the Go CI tool by executing the commands below from the project directory:

dagger run go run multibuild/main.go

The dagger run command executes the specified command in a Dagger session and displays live progress. Once complete, the Go CI tool outputs the string below, although it isn't actually building anything yet.

Building with Dagger

Step 4: Create a single-build pipeline​

Now that the basic structure of the Go CI tool is defined and functional, the next step is to flesh out its build() function to actually build the Go application.

Replace the multibuild/main.go file from the previous step with the version below (highlighted lines indicate changes):

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	if err := build(context.Background()); err != nil {
		fmt.Println(err)
	}
}

func build(ctx context.Context) error {
	fmt.Println("Building with Dagger")

	// initialize Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		return err
	}
	defer client.Close()

	// get reference to the local project
	src := client.Host().Directory(".")

	// get `golang` image
	golang := client.Container().From("golang:latest")

	// mount cloned repository into `golang` image
	golang = golang.WithDirectory("/src", src).WithWorkdir("/src")

	// define the application build command
	path := "build/"
	golang = golang.WithExec([]string{"go", "build", "-o", path})

	// get reference to build output directory in container
	output := golang.Directory(path)

	// write contents of container build/ directory to the host
	_, err = output.Export(ctx, path)
	if err != nil {
		return err
	}

	return nil
}

The revised build() function is the main workhorse here, so let's step through it in detail.

  • It begins by creating a Dagger client with dagger.Connect(), as before.
  • It uses the client's Host().Directory() method to obtain a reference to the current directory on the host. This reference is stored in the src variable.
  • It initializes a new container from a base image with the Container().From() method and returns a new Container struct. In this case, the base image is the golang:latest image.
  • It writes the filesystem of the repository branch in the container using the WithDirectory() method of the Container.
    • The first argument is the target path in the container (here, /src).
    • The second argument is the directory to be written (here, the reference previously created in the src variable). It also changes the current working directory to the /src path of the container using the WithWorkdir() method and returns a revised Container with the results of these operations.
  • It uses the WithExec() method to define the command to be executed in the container - in this case, the command go build -o PATH, where PATH refers to the build/ directory in the container. The WithExec() method returns a revised Container containing the results of command execution.
  • It obtains a reference to the build/ directory in the container with the Directory() method.
  • It writes the build/ directory from the container to the host using the Directory.Export() method.

Try the tool by executing the commands below:

dagger run go run multibuild/main.go

The Go CI tool builds the current Go project and writes the build result to build/ on the host.

Use the tree command to see the build artifact on the host, as shown below:

tree build
build
└── multibuild

Step 5: Create a multi-build pipeline​

Now that the Go CI tool can build a Go application and output the build result, the next step is to extend it for multiple OS and architecture combinations.

Replace the multibuild/main.go file from the previous step with the version below (highlighted lines indicate changes):

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	if err := build(context.Background()); err != nil {
		fmt.Println(err)
	}
}

func build(ctx context.Context) error {
	fmt.Println("Building with Dagger")

	// define build matrix
	oses := []string{"linux", "darwin"}
	arches := []string{"amd64", "arm64"}

	// initialize Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		return err
	}
	defer client.Close()

	// get reference to the local project
	src := client.Host().Directory(".")

	// create empty directory to put build outputs
	outputs := client.Directory()

	// get `golang` image
	golang := client.Container().From("golang:latest")

	// mount cloned repository into `golang` image
	golang = golang.WithDirectory("/src", src).WithWorkdir("/src")

	for _, goos := range oses {
		for _, goarch := range arches {
			// create a directory for each os and arch
			path := fmt.Sprintf("build/%s/%s/", goos, goarch)

			// set GOARCH and GOOS in the build environment
			build := golang.WithEnvVariable("GOOS", goos)
			build = build.WithEnvVariable("GOARCH", goarch)

			// build application
			build = build.WithExec([]string{"go", "build", "-o", path})

			// get reference to build output directory in container
			outputs = outputs.WithDirectory(path, build.Directory(path))
		}
	}
	// write build artifacts to host
	_, err = outputs.Export(ctx, ".")
	if err != nil {
		return err
	}

	return nil
}

This revision of the Go CI tool does much the same as before, except that it now supports building the application for multiple OSs and architectures.

  • It defines the build matrix, consisting of two OSs (darwin and linux) and two architectures (amd64 and arm64).
  • It iterates over this matrix, building the Go application for each combination. The Go build process is instructed via the GOOS and GOARCH build variables, which are reset for each case via the Container.WithEnvVariable() method.
  • It creates an output directory on the host named for each OS/architecture combination so that the build outputs can be differentiated.

Try the Go CI tool by executing the commands below:

dagger run go run multibuild/main.go

The Go CI tool builds the application for each OS/architecture combination and writes the build results to the host. You will see the build process run four times, once for each combination. Note that the builds are happening concurrently, because the builds do not depend on eachother.

Use the tree command to see the build artifacts on the host, as shown below:

tree build
build/
├── darwin
│   ├── amd64
│   │   └── multibuild
│   └── arm64
│       └── multibuild
└── linux
    ├── amd64
    │   └── multibuild
    └── arm64
        └── multibuild

Another common operation in a CI environment involves creating builds targeting multiple Go versions. To do this, extend the Go CI tool further and replace the multibuild/main.go file from the previous step with the version below (highlighted lines indicate changes):

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	if err := build(context.Background()); err != nil {
		fmt.Println(err)
	}
}

func build(ctx context.Context) error {
	fmt.Println("Building with Dagger")

	// define build matrix
	oses := []string{"linux", "darwin"}
	arches := []string{"amd64", "arm64"}
	goVersions := []string{"1.20", "1.21"}

	// initialize Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		return err
	}
	defer client.Close()

	// get reference to the local project
	src := client.Host().Directory(".")

	// create empty directory to put build outputs
	outputs := client.Directory()

	for _, version := range goVersions {
		// get `golang` image for specified Go version
		imageTag := fmt.Sprintf("golang:%s", version)
		golang := client.Container().From(imageTag)
		// mount cloned repository into `golang` image
		golang = golang.WithDirectory("/src", src).WithWorkdir("/src")

		for _, goos := range oses {
			for _, goarch := range arches {
				// create a directory for each os, arch and version
				path := fmt.Sprintf("build/%s/%s/%s/", version, goos, goarch)
				// set GOARCH and GOOS in the build environment
				build := golang.WithEnvVariable("GOOS", goos)
				build = build.WithEnvVariable("GOARCH", goarch)

				// build application
				build = build.WithExec([]string{"go", "build", "-o", path})

				// get reference to build output directory in container
				outputs = outputs.WithDirectory(path, build.Directory(path))
			}
		}
	}
	// write build artifacts to host
	_, err = outputs.Export(ctx, ".")
	if err != nil {
		return err
	}
	return nil
}

This revision of the Go CI tool adds another layer to the build matrix, this time for Go language versions. Here, the build() function uses the Go version number to download the appropriate Go base image for each build. It also adds the Go version number to each build output directory on the host to differentiate the build outputs.

Try the Go CI tool by executing the commands below:

dagger run go run multibuild/main.go

The Go CI tool builds the application for each OS/architecture/version combination and writes the results to the host. You will see the build process run eight times, once for each combination. Note that the builds are happening concurrently, because each build in the DAG does not depend on any other build.

Use the tree command to see the build artifacts on the host, as shown below:

tree build
build/
├── 1.20
│   ├── darwin
│   │   ├── amd64
│   │   │   └── multibuild
│   │   └── arm64
│   │       └── multibuild
│   └── linux
│       ├── amd64
│       │   └── multibuild
│       └── arm64
│           └── multibuild
└── 1.21
    ├── darwin
    │   ├── amd64
    │   │   └── multibuild
    │   └── arm64
    │       └── multibuild
    └── linux
        ├── amd64
        │   └── multibuild
        └── arm64
            └── multibuild

tip

As the previous steps illustrate, the Dagger Go SDK allows you to author your pipeline entirely in Go. This means that you don't need to spend time learning a new language, and you immediately benefit from all the powerful programming capabilities and packages available in Go. For instance, this tutorial used native Go variables, conditionals and error handling throughout, together with the errgroup package for sub-task parallelization.

Conclusion​

This tutorial introduced you to the Dagger Go SDK. It explained how to install the SDK and use it with a Go module. It also provided a working example of a Go CI tool powered by the SDK, which is able to build an application for multiple OSs, architectures and Go versions in parallel.

Use the SDK Reference to learn more about the Dagger Go SDK.