Skip to main content

Replace a Dockerfile with Go (or Python, or Node.js)

Introduction​

This guide explains how to use a Dagger SDK to perform all the same operations that you would typically perform with a Dockerfile, except using Go, Python or Node.js. You will learn how to:

  • Create a Dagger client
  • Write a Dagger pipeline to:
    • Configure a container with all required dependencies and environment variables
    • Download and build the application source code in the container
    • Set the container entrypoint
    • Publish the built container image to Docker Hub
  • Test the Dagger pipeline locally

Requirements​

This guide assumes that:

  • You have a Go, Python or Node.js development environment. If not, install Go, Python or Node.js.
  • You have Docker installed and running on the host system. If not, install Docker.
  • You have a Dagger SDK installed for one of the above languages. If not, follow the installation instructions for the Dagger Go, Python or Node.js SDK.
  • You have the Dagger CLI installed in your development environment. If not, install the Dagger CLI.
  • You have a Docker Hub account. If not, register for a Docker Hub account.

Step 1: Understand the source Dockerfile​

To illustrate the process, this guide replicates the build process for the popular open source Memcached caching system using Dagger. It uses the Dockerfile and entrypoint script for the official Docker Hub Memcached image.

Begin by reviewing the source Dockerfile and corresponding entrypoint script to understand how it works. This Dockerfile is current at the time of writing and is available under the BSD 3-Clause License.

Broadly, this Dockerfile performs the following steps:

  • It starts from a base alpine container image.
  • It adds a memcache user and group with defined IDs.
  • It sets environment variables for the Memcached version (MEMCACHED_VERSION) and commit hash (MEMCACHED_SHA1).
  • It installs dependencies in the container.
  • It downloads the source code archive for the specified version of Memcached, checks the commit hash and extracts the source code into a directory.
  • It configures, builds, tests and installs Memcached from source using make.
  • It copies and sets the container entrypoint script.
  • It configures the image to run as the memcache user.

Step 2: Replicate the Dockerfile using a Dagger pipeline​

The Dagger SDK enables you to develop a CI/CD pipeline in one of the supported languages (Go, Python or Node.js) to achieve the same result as using a Dockerfile.

To see how this works, add the following code to your Go module as main.go. Replace the DOCKER-HUB-USERNAME placeholder with your Docker Hub username.

package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

const (
	nproc       = "1"
	gnuArch     = "arm64"
	publishAddr = "DOCKER-HUB-USERNAME/my-memcached"
)

func main() {
	ctx := context.Background()

	// create a Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		panic(err)
	}
	defer client.Close()

	// set the base container
	// set environment variables
	memcached := client.Container().
		From("alpine:3.17").
		WithExec([]string{"addgroup", "-g", "11211", "memcache"}).
		WithExec([]string{"adduser", "-D", "-u", "1121", "-G", "memcache", "memcache"}).
		WithExec([]string{"apk", "add", "--no-cache", "libsasl"}).
		WithEnvVariable("MEMCACHED_VERSION", "1.6.17").
		WithEnvVariable("MEMCACHED_SHA1", "e25639473e15f1bd9516b915fb7e03ab8209030f")

	// add dependencies to the container
	memcached = setDependencies(memcached)

	// add source code to the container
	memcached = downloadMemcached(memcached)

	// build the application
	memcached = buildMemcached(memcached)

	// set the container entrypoint
	entrypoint := client.Host().Directory(".").File("docker-entrypoint.sh")
	memcached = memcached.
		WithFile("/usr/local/bin/docker-entrypoint.sh", entrypoint).
		WithExec([]string{"ln", "-s", "usr/local/bin/docker-entrypoint.sh", "/entrypoint.sh"}).
		WithEntrypoint([]string{"docker-entrypoint.sh"}).
		WithUser("memcache").
		WithDefaultArgs([]string{"memcached"})

	// publish the container image
	addr, err := memcached.Publish(ctx, publishAddr)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Published to %s", addr)
}

func setDependencies(container *dagger.Container) *dagger.Container {
	return container.
		WithExec([]string{
			"apk",
			"add",
			"--no-cache",
			"--virtual",
			".build-deps",
			"ca-certificates",
			"coreutils",
			"cyrus-sasl-dev",
			"gcc",
			"libc-dev",
			"libevent-dev",
			"linux-headers",
			"make",
			"openssl",
			"openssl-dev",
			"perl",
			"perl-io-socket-ssl",
			"perl-utils",
		})
}

func downloadMemcached(container *dagger.Container) *dagger.Container {
	return container.
		WithExec([]string{"sh", "-c", "wget -O memcached.tar.gz https://memcached.org/files/memcached-$MEMCACHED_VERSION.tar.gz"}).
		WithExec([]string{"sh", "-c", "echo \"$MEMCACHED_SHA1  memcached.tar.gz\" | sha1sum -c -"}).
		WithExec([]string{"mkdir", "-p", "/usr/src/memcached"}).
		WithExec([]string{"tar", "-xvf", "memcached.tar.gz", "-C", "/usr/src/memcached", "--strip-components=1"}).
		WithExec([]string{"rm", "memcached.tar.gz"})
}

func buildMemcached(container *dagger.Container) *dagger.Container {
	return container.
		WithWorkdir("/usr/src/memcached").
		WithExec([]string{
			"./configure",
			fmt.Sprintf("--build=%s", gnuArch),
			"--enable-extstore",
			"--enable-sasl",
			"--enable-sasl-pwdb",
			"--enable-tls",
		}).
		WithExec([]string{"make", "-j", nproc}).
		WithExec([]string{"make", "test", fmt.Sprintf("PARALLEL=%s", nproc)}).
		WithExec([]string{"make", "install"}).
		WithWorkdir("/usr/src/memcached").
		WithExec([]string{"rm", "-rf", "/usr/src/memcached"}).
		WithExec([]string{
			"sh",
			"-c",
			"apk add --no-network --virtual .memcached-rundeps $( scanelf --needed --nobanner --format '%n#p' --recursive /usr/local | tr ',' '\n' | sort -u | awk 'system(\"[ -e /usr/local/lib/\" $1 \" ]\") == 0 { next } { print \"so:\" $1 }')",
		}).
		WithExec([]string{"apk", "del", "--no-network", ".build-deps"}).
		WithExec([]string{"memcached", "-V"})
}

There's a lot going on here, so let's step through it in detail:

  • The Go CI pipeline imports the Dagger SDK and defines a main() function. The main() function creates a Dagger client with dagger.Connect(). This client provides an interface for executing commands against the Dagger engine.
  • It initializes a new container from a base image with the client's Container().From() method and returns a new Container struct. In this case, the base image is the alpine:3.17 image.
  • It calls the WithExec() method to define the adduser, addgroup and apk add commands for execution, and the WithEnvVariable() method to set the MEMCACHED_VERSION and MEMCACHED_SHA1 container environment variables.
  • It calls a custom setDependencies() function, which internally uses WithExec() to define the apk add command that installs all the required dependencies to build and test Memcached in the container.
  • It calls a custom downloadMemcached() function, which internally uses WithExec() to define the wget, tar and related commands required to download, verify and extract the Memcached source code archive in the container at the /usr/src/memcached container path.
  • It calls a custom buildMemcached() function, which internally uses WithExec() to define the configure and make commands required to build, test and install Memcached in the container. The buildMemcached() function also takes care of deleting the source code directory at /usr/src/memcached in the container and executing memcached -V to output the version string to the console.
  • It updates the container filesystem to include the entrypoint script from the host using WithFile() and specifies it as the command to be executed when the container runs using WithEntrypoint().
  • Finally, it calls the Container.Publish() method, which executes the entire pipeline described above and publishes the resulting container image to Docker Hub.
warning

Like the source Dockerfile, this pipeline assumes that the entrypoint script exists in the current working directory on the host as docker-entrypoint.sh. You can either create a custom entrypoint script, or use the entrypoint script from the Docker Hub Memcached image repository.

Step 3: Test the Dagger pipeline​

Test the Dagger pipeline as follows:

  1. Log in to Docker on the host:

    docker login
    info

    This step is necessary because Dagger relies on the host's Docker credentials and authorizations when publishing to remote registries.

  2. Run the pipeline:

    dagger run go run main.go
warning

Verify that you have an entrypoint script on the host at ./docker-entrypoint.sh before running the Dagger pipeline.

The dagger run command executes the script in a Dagger session and displays live progress. This process will take some time. At the end of the process, the built container image is published on Docker Hub and a message similar to the one below appears in the console output:

Published to docker.io/.../my-memcached@sha256:692....

Browse to your Docker Hub registry to see the published Memcached container image.

Conclusion​

This tutorial introduced you to the Dagger SDKs. By replacing a Dockerfile with native code, it demonstrated how Dagger SDKs contain everything you need to develop CI/CD pipelines in your favorite language and run them on any OCI-compatible container runtime.

The advantage of this approach is that it allows you to use powerful native language features, such as (where applicable) static typing, concurrency, programming structures such as loops and conditionals, and built-in testing, to create powerful CI/CD tooling for your project or organization.

Use the API Key Concepts page and the Go, Node.js and Python SDK References to learn more about Dagger.