Skip to main content

Build a CI Pipeline

CI pipelines often start simple, but eventually transform into a labyrinth of artisanal shell scripts and/or unmanageable YAML code. Dagger lets you replace those artisanal scripts and YAML with a modern API and cross-language scripting engine.

This quickstart will guide you through building a CI pipeline for an application using Dagger.

Requirements​

This quickstart will take you approximately 10 minutes to complete. You should be familiar with programming in Go, Python, TypeScript, PHP, or Java.

Before beginning, ensure that:

  • you have installed the Dagger CLI.
  • you know the basics of Dagger.
  • you have Git and a container runtime installed on your system and running. This can be Docker, but you can also use Podman, nerdctl, or other Docker-like systems.
  • you have a GitHub account (optional, only if configuring Dagger Cloud)
tip

Dagger SDKs generate native code-bindings for all dependencies from the Dagger API. If you're using an IDE, this gives you automatic type-checking, code completion and other key features when developing Dagger Functions. Setting up your IDE to work with Dagger is a highly recommended step to improve your experience and benefit from Dagger's universal type system.

Get the example application​

The example application is a skeleton Vue framework application that returns a "Hello from Dagger!" welcome page. Clone its repository and set it as the current working directory:

git clone https://github.com/dagger/hello-dagger
cd hello-dagger

Configure Dagger Cloud (optional)​

important

This step is optional and will create a Dagger Cloud account, which is free of charge for a single user. If you prefer not to sign up for Dagger Cloud, you can skip this section.

Dagger Cloud is an online visualization tool for Dagger workflows. It provides a web interface to visualize each step of your workflow, drill down to detailed logs, understand how long operations took to run, and whether operations were cached.

Create a new Dagger Cloud account by running dagger login:

dagger login

The Dagger CLI will invite you to authenticate your device by displaying a link containing a unique key. Click the link in your browser, and verify that you see the same key in the Dagger Cloud Web interface.

$ dagger login
Browser opened to: https://auth.dagger.cloud/activate?user_code=FCNP-SRLM
Confirmation code: FCNP-SRLM

Once you confirm your authentication code, your Dagger CLI will be authenticated and you will get redirected to your newly created Dagger Cloud organization.

After successfully creating your organization, all future Dagger workflows can be inspected in Dagger Cloud.

Initialize a Dagger module​

Bootstrap a new Dagger module in Go, Python, TypeScript, PHP, or Java by running dagger init in the application's root directory.

dagger init --sdk=go

This will generate a dagger.json module metadata file and a .dagger directory containing some boilerplate Dagger Functions as examples.

To see the generated Dagger Functions, run:

dagger functions

You should see information about two auto-generated Dagger Functions: container-echo and grep-dir.

important

By default, the Dagger module name is automatically generated from the name of the directory in which dagger init runs. In this case, the default name of the cloned application directory is hello-dagger, so the module name is HelloDagger. If you cloned the application into a different directory, add the --name=hello-dagger flag to dagger init to correctly set the Dagger module name.

Construct a pipeline​

Replace the generated Dagger module files as described below.

Replace the generated .dagger/main.go file with the following code, which adds four Dagger Functions to your Dagger module:

package main

import (
	"context"
	"fmt"
	"math"
	"math/rand"

	"dagger/hello-dagger/internal/dagger"
)

type HelloDagger struct{}

// Publish the application container after building and testing it on-the-fly
func (m *HelloDagger) Publish(
	ctx context.Context,
	// +defaultPath="/"
	source *dagger.Directory,
) (string, error) {
	_, err := m.Test(ctx, source)
	if err != nil {
		return "", err
	}
	return m.Build(source).
		Publish(ctx, fmt.Sprintf("ttl.sh/hello-dagger-%.0f", math.Floor(rand.Float64()*10000000))) //#nosec
}

// Build the application container
func (m *HelloDagger) Build(
	// +defaultPath="/"
	source *dagger.Directory,
) *dagger.Container {
	build := m.BuildEnv(source).
		WithExec([]string{"npm", "run", "build"}).
		Directory("./dist")
	return dag.Container().From("nginx:1.25-alpine").
		WithDirectory("/usr/share/nginx/html", build).
		WithExposedPort(80)
}

// Return the result of running unit tests
func (m *HelloDagger) Test(
	ctx context.Context,
	// +defaultPath="/"
	source *dagger.Directory,
) (string, error) {
	return m.BuildEnv(source).
		WithExec([]string{"npm", "run", "test:unit", "run"}).
		Stdout(ctx)
}

// Build a ready-to-use development environment
func (m *HelloDagger) BuildEnv(
	// +defaultPath="/"
	source *dagger.Directory,
) *dagger.Container {
	nodeCache := dag.CacheVolume("node")
	return dag.Container().
		From("node:21-slim").
		WithDirectory("/src", source).
		WithMountedCache("/root/.npm", nodeCache).
		WithWorkdir("/src").
		WithExec([]string{"npm", "install"})
}

In this Dagger module, each Dagger Function performs a different operation:

  • The publish Dagger Function tests, builds and publishes a container image of the application to a registry.
  • The test Dagger Function runs the application's unit tests and returns the results.
  • The build Dagger Function performs a multi-stage build and returns a final container image with the production-ready application and an NGINX Web server to host and serve it.
  • The build-env Dagger Function creates a container with the build environment for the application.

Run the pipeline​

Dagger Shell is the fastest way to interact with the Dagger API, allowing access to both core types and custom Dagger Functions using a familiar Bash-like syntax. Type dagger to launch Dagger Shell in interactive mode.

publish

This single command runs the application's tests, then builds and publishes it as a container image to the ttl.sh container registry. Here's what you should see:

Publish

You can test the published container image by pulling and running it with docker run:

Docker run

If you signed up for Dagger Cloud, the output of the previous command would have also included a link to visualize the workflow run on Dagger Cloud. Click the link in your browser to see a complete breakdown of the steps performed. Here's what you should see:

login

Interact with the build environment​

The build-env Dagger Function returns a Container type representing the application's build environment. One of the most interesting built-in functions of this type is terminal, which can be used to open an interactive terminal session with the running container.

To try this, chain an additional function call to terminal on the returned Container:

build-env | terminal --cmd=bash

This command builds the container image and then drops you into an interactive terminal running the bash shell. You can now directly execute commands in the running container, as shown below:

Build env with terminal

Run a container as a local service​

The build Dagger Function returns a Container type representing the built container image. Another interesting built-in function to explore here is the as-service function, which can be used to start a container as a local service and have any exposed ports forwarded to the host machine. This is similar to Docker Compose, except that you're using code instead of YAML to manage your services.

To try this, use the function chain below:

build | as-service | up --ports=8080:80

By default, Dagger will map each exposed container service port to the same port on the host. Since NGINX operates on port 80, which is often a privileged port on the host, the additional --ports 8080:80 argument re-maps container port 80 to unprivileged host port 8080.

Final container as service

You should now be able to access the application by browsing to http://localhost:8080 on the Dagger host (replace localhost with your Dagger host's network name or IP address if accessing it remotely). You should see a "Hello from Dagger!" welcome page, served by NGINX.

Test service

Next steps​

Congratulations! You've created your first CI pipeline with Dagger.

Now you have the tools to successfully take the next step: adopting Dagger in your project.

In the meantime, we encourage you to join our awesome community on Discord to introduce yourself and ask questions. And starring our GitHub repository is always appreciated!