Go on Docker Swarm
Particubes is a platform dedicated to voxel games, which are games made out of little cubes, like Minecraft. The team consists of 10 developers that like to keep things simple. They write primarily Go & Lua, push to GitHub and use GitHub Actions for automation. The production setup is a multi-node Docker Swarm cluster running on AWS.
The Particubes team chose Dagger for continuous deployment because it was the easiest way of integrating GitHub with Docker Swarm. Every commit to the main branch goes straight to docs.particubes.com via a Dagger pipeline that runs in GitHub Actions. Let us see how the Particubes Dagger plan fits together.
Actions API
This is a high level overview of all actions in the Particubes docs Dagger plan:
We can see all available actions in a Plan by running the following command:
$ dagger do
Execute a dagger action.
Available Actions:
build Create a container image
clean Remove a container image
test Locally test a container image
deploy Deploy a container image
Client API
Dagger actions usually need to interact with the host environment where the Dagger client runs. The Particubes' plan uses environment variables and the filesystem.
This is an overview of all client interactions for this plan:
This is what the above looks like in the Dagger plan config:
package docs
import (
"dagger.io/dagger"
)
dagger.#Plan & {
client: {
// Locally, manual source of the .env or install https://direnv.net
env: {
GITHUB_SHA: string
SSH_PRIVATE_KEY_DOCKER_SWARM: dagger.#Secret
}
filesystem: {
"./": read: contents: dagger.#FS
"./merge.output": write: contents: actions.build.image.rootfs // Creates a build artifact for debug
}
network: "unix:///var/run/docker.sock": connect: dagger.#Socket // Docker daemon socket
}
}
The build Action
This is a more in-depth overview of the build action and how it interacts with the client in the Particubes docs Dagger plan:
This is what the above looks like in the Dagger plan config:
build: {
luaDocs: docker.#Dockerfile & {
source: client.filesystem."./lua-docs".read.contents
}
_addGithubSHA: core.#WriteFile & {
input: luaDocs.output.rootfs
path: "/www/github_sha.yml"
contents: #"""
keywords: ["particubes", "game", "mobile", "scripting", "cube", "voxel", "world", "docs"]
title: "Github SHA"
blocks:
- text: "\#(client.env.GITHUB_SHA)"
"""#
}
image: docker.#Image & {
rootfs: _addGithubSHA.output
config: luaDocs.output.config
}
}
GitHub Action integration
This is the GitHub Actions workflow config that invokes dagger, which in turn runs the full plan:
name: Dagger/docs.particubes.com
on:
push:
branches: [master]
jobs:
deploy:
runs-on: ubuntu-latest
env:
GITHUB_SHA: ${{ github.sha }}
SSH_PRIVATE_KEY_DOCKER_SWARM: ${{ secrets.SSH_PRIVATE_KEY_DOCKER_SWARM }}
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Install Dagger
uses: dagger/dagger-for-github@v3
with:
install-only: true
- name: Dagger project update
run: dagger project update
- name: Dagger do test
run: dagger do test --log-format plain
- name: Dagger do deploy
run: dagger do deploy --log-format plain
Since this is a Dagger pipeline, anyone on the team can run it locally with a single command:
dagger do
This is the first step that enabled the Particubes team to have the same CI/CD experience everywhere.
Full Particubes docs Dagger plan
This is the entire plan running on Particubes' CI:
package docs
import (
"dagger.io/dagger"
"dagger.io/dagger/core"
"universe.dagger.io/alpine"
"universe.dagger.io/bash"
"universe.dagger.io/docker"
"universe.dagger.io/docker/cli"
)
dagger.#Plan & {
client: {
// Locally, manual source of the .env or install https://direnv.net
env: {
GITHUB_SHA: string
SSH_PRIVATE_KEY_DOCKER_SWARM: dagger.#Secret
}
filesystem: {
"./": read: contents: dagger.#FS
"./merge.output": write: contents: actions.build.image.rootfs // Creates a build artifact for debug
}
network: "unix:///var/run/docker.sock": connect: dagger.#Socket // Docker daemon socket
}
actions: {
params: image: {
ref: "registry.particubes.com/lua-docs"
tag: "latest"
localTag: "test-particubes" // name of the image when being run locally
}
_dockerCLI: alpine.#Build & {
packages: {
bash: {}
curl: {}
"docker-cli": {}
"openssh-client": {}
}
}
#_verifyGithubSHA: bash.#Run & {
input: _dockerCLI.output
env: GITHUB_SHA: client.env.GITHUB_SHA
always: true
script: contents: #"""
TRIMMED_URL="$(echo $URL | cut -d '/' -f 1)"
curl --verbose --fail --connect-timeout 5 --location "$URL" >"$TRIMMED_URL.curl.out" 2>&1
if ! grep "$GITHUB_SHA" "$TRIMMED_URL.curl.out"
then
echo "$GITHUB_SHA not present in the $TRIMMED_URL response:"
cat "$TRIMMED_URL.curl.out"
exit 1
fi
"""#
}
build: {
luaDocs: docker.#Dockerfile & {
source: client.filesystem."./lua-docs".read.contents
}
_addGithubSHA: core.#WriteFile & {
input: luaDocs.output.rootfs
path: "/www/github_sha.yml"
contents: #"""
keywords: ["particubes", "game", "mobile", "scripting", "cube", "voxel", "world", "docs"]
title: "Github SHA"
blocks:
- text: "\#(client.env.GITHUB_SHA)"
"""#
}
image: docker.#Image & {
rootfs: _addGithubSHA.output
config: luaDocs.output.config
}
}
clean: cli.#Run & {
host: client.network."unix:///var/run/docker.sock".connect
always: true
env: IMAGE_NAME: params.image.localTag
command: {
name: "sh"
flags: "-c": #"""
docker rm --force "$IMAGE_NAME"
"""#
}
}
test: {
preLoad: clean
load: cli.#Load & {
image: build.image
host: client.network."unix:///var/run/docker.sock".connect
tag: params.image.localTag
env: DEP: "\(preLoad.success)" // DEP created wth preLoad
}
run: cli.#Run & {
host: client.network."unix:///var/run/docker.sock".connect
always: true
env: {
IMAGE_NAME: params.image.localTag
PORTS: "80:80"
DEP: "\(load.success)" // DEP created wth load
}
command: {
name: "sh"
flags: "-c": #"""
docker run -d --rm --name "$IMAGE_NAME" -p "$PORTS" "$IMAGE_NAME"
"""#
}
}
verify: #_verifyGithubSHA & {
env: {
URL: "localhost/github_sha"
DEP: "\(run.success)" // DEP created wth run
}
}
postVerify: clean & {
env: DEP: "\(verify.success)" // DEP created wth verify
}
}
deploy: {
publish: docker.#Push & {
dest: "\(params.image.ref):\(params.image.tag)"
image: build.image
}
update: cli.#Run & {
host: "ssh://ubuntu@3.139.83.217"
always: true
ssh: key: client.env.SSH_PRIVATE_KEY_DOCKER_SWARM
env: DEP: "\(publish.result)" // DEP created wth publish
command: {
name: "sh"
flags: "-c": #"""
docker service update --image registry.particubes.com/lua-docs:latest lua-docs
"""#
}
}
verify: #_verifyGithubSHA & {
env: {
URL: "https://docs.particubes.com/github_sha"
DEP: "\(update.success)" // DEP created wth run
}
}
}
}
}
What comes next ?
Particubes' team suggested that we create a dev action with hot reload, that way Dagger would even asbtract away the ramp-up experience when developing the doc
The latest version of this pipeline can be found at github.com/voxowl/particubes/pull/144