Contributing to Dagger
The best way to find a good contribution is to use Dagger for something. Then write down what problems you encounter. Could be as simple as a question you had, that the docs didn't answer. Or a bug in the tool, or a missing feature. Then pick an item that you're comfortable with in terms of difficulty, and give it a try. 🙂
You can ask questions along the way, we're always happy to help you with your contribution. The bigger the contribution, the earlier you should talk to maintainers to make sure you're taking the right approach and are not wasting your effort on something that will not get merged.
GitHub Workflow
The recommended workflow is to fork the repository and open pull requests from your fork.
1. Fork, clone & configure Dagger upstream
- Click on the Fork button on GitHub
- Clone your fork
- Add the upstream repository as a new remote
# Clone repository
git clone git@github.com:$YOUR_GITHUB_USER/dagger.git
# Add upstream origin
git remote add upstream git@github.com:dagger/dagger.git
2. Create a pull request
# Create a new feature branch
git checkout -b my_feature_branch
# Make changes to your branch
# ...
# Commit changes - remember to sign!
git commit -s
# Push your new feature branch
git push my_feature_branch
# Create a new pull request from https://github.com/dagger/dagger
3. Add release notes fragment
If this is a user-facing change, please add a line for the release notes.
You will need to have changie installed.
If this is a user-facing change in the 🚙 Engine or 🚗 CLI, run changie new in the top level directory.
Here is an example of what that looks like:
changie new
✔ Kind … Added
✔ Body … engine: add `Directory.Sync`
✔ GitHub PR … 5414
✔ GitHub Author … helderco
If there are code changes in the SDKs, run changie new in the corresponding directory, e.g. sdk/go, sdk/typescript, etc.
Remember to add & commit the release notes fragment. This will be used at release time, in the changelog. Here is an example of the end-result for all release notes fragments: https://github.com/dagger/dagger/blob/v0.6.4/.changes/v0.6.4.md
You can find an asciinema of how changie works on https://changie.dev
4. Update your pull request with latest changes
# Checkout main branch
git checkout main
# Update your fork's main branch from upstream
git pull upstream main
# Checkout your feature branch
git checkout my_feature_branch
# Rebase your feature branch changes on top of the updated main branch
git rebase main
# Update your pull request with latest changes
git push -f my_feature_branch
Scope of pull requests
We prefer small incremental changes that can be reviewed and merged quickly. It's OK if it takes multiple pull requests to close an issue.
The idea is that each improvement should land in Dagger's main branch within a few hours. The sooner we can get multiple people looking at and agreeing on a specific change, the quicker we will have it out in a release. The quicker we can get these small improvementes in a Dagger release, the quicker we can get feedback from our users and find out what doesn't work, or what we have missed.
The added benefit is that this will force everyone to think about handling partially implemented features & non-breaking changes. Both are great approaches, and they work really well in the context of Dagger.
"Small incremental changes ftw" -> Small pull requests that get merged within hours!
Commits
License
Contributions to this project are made under the Apache License 2.0 (Apache-2.0).
DCO
Contributions to this project must be accompanied by a Developer Certificate of Origin (DCO).
All commit messages must contain the Signed-off-by line with an email address
that matches the commit author. When committing, use the --signoff flag:
git commit -s
The Signed-off-by line must match the author's real name, otherwise the PR will be rejected.
Commit messages
Guidelines:
- Group Commits: Each commit should represent a meaningful change (e.g. implement feature X, fix bug Y, ...).
- For instance, a PR should not look like 1) Add Feature X 2) Fix Typo 3) Changes to features X 5) Bugfix for feature X 6) Fix Linter 7) ...
- Instead, these commits should be squashed together into a single "Add Feature" commit.
- Each commit should work on its own: it must compile, pass the linter and so on.
- This makes life much easier when using
git log,git blame,git bisect, etc. - For instance, when doing a
git blameon a file to figure out why a change was introduced, it's pretty meaningless to see a Fix linter commit message. "Add Feature X" is much more meaningful.
- This makes life much easier when using
- Use
git rebase -i mainto group commits together and rewrite their commit message. - To add changes to the previous commit, use
git commit --amend -s. This will change the last commit (amend) instead of creating a new commit. - Format: Use the imperative mood in the subject line: "If applied, this commit will your subject line here"
- Add the following prefixes to your commit message to help trigger automated processes1:
docs:for documentation changes only (e.g.,docs: Fix typo in X);test:for changes to tests only (e.g.,test: Check if X does Y);chore:general things that should be excluded (e.g.,chore: Clean up X);website:for the documentation website (i.e., the frontend code; e.g.,website: Add X link to navbar);ci:for internal CI specific changes (e.g.,ci: Enable X for tests);infra:for infrastructure changes (e.g.,infra: Enable cloudfront for X);fix: for improvements and bugfixes that do not introduce a feature (e.g.,fix: improve error message);feat: for new features (e.g.,feat: implement --cache-to feature to export cache)
Docs
Use relative file paths for links
Instead of using URLs to link to a doc page, use relative file paths instead:
❌ This is [a problematic link](/doc-url).
✅ This is [a good link](../relative-doc-file-path.md).
The docs compiler will replace file links with URLs automatically. This helps prevent broken internal links. If a file gets renamed, the compiler will catch broken links and throw an error. Learn more.
Vulnerability Scanning
We run trivy scanning of our engine image in GHA to scan for any CVEs present in any third-party binaries we build or that we include in the image (e.g. runc, CNI plugins, etc.). As of this writing, we currently only scan for Critical and High severity CVEs. If any of those are found the GHA job will fail.
It's sometimes possible that the vulnerability may require quite a bit of work to address, especially if it's coming from third-party binary or a transitive dependency in our go.mod that does not have a release with the fix yet.
- In this case, it's worth checking whether the specific vulnerability is actually relevant to us. If it's not or if you're unsure, reach out to the Dagger team on Github or Discord and we can figure out whether to address it or add it to an ignore list.
The rest of this section gives some guidance on fixing these vulnerabilities when they are relevant.
Vulnerability in a Dagger binary
If a vulnerability is reported in the Go stdlib, we'll want to upgrade the version of Go we use to build everything. As of this writing, this can be done by changing GolangVersion in ci/consts/versions.go.
Otherwise, if a vulnerability is reported in a Go dependency, you'll want to track down where the dependency is coming from.
This can become a bit complicated since it's possible for multiple versions of a Go module to be in the dependency DAG, with only a subset of the versions actually being vulnerable.
- If the dependency at the vulnerable version is directly listed in our
go.mod, then you should start by just upgrading it there. - After that, if the vulnerability is still reported, it may be coming from a transitive dependency.
- You can track those down with the
go mod graphcommand. For example, if the vulnerable module version isgolang.org/x/sys@v0.0.0-20211116061358-0a5406a5449c, you can rungo mod graph | grep 'golang.org/x/sys@v0.0.0-20211116061358-0a5406a5449c'.
- You can track those down with the
Vulnerability in a third-party binary
If a vulnerability is reported in the Go stdlib, we'll want to upgrade the version of Go we use to build everything. As of this writing, this can be done by changing GolangVersion in ci/consts/versions.go.
Otherwise, you'll want to check if the binary in question has a newer version with the vulnerability gone. The versions of these binaries are also controlled in ci/consts/versions.go.
If there isn't a newer version to upgrade to, we'll be in a tougher spot and may need some combination of upgrading to a non-released commit, sending patches upstream or (as a worst-case fallback) patching it ourselves. Reach out to the Dagger team on Github or Discord if you're unsure how to best proceed.
FAQ
How to run a development engine?
To automatically build and start a development engine (and stop any previously running one):
./hack/dev
The above will build the CLI and the engine, and provide configuration details to connect to the engine:
export _EXPERIMENTAL_DAGGER_CLI_BIN=bin/dagger
export _EXPERIMENTAL_DAGGER_RUNNER_HOST=docker-container://dagger-engine.dev
To run the dev CLI:
> ./bin/dagger version
dagger devel () linux/amd64
How to run tests locally?
To run all go tests against an engine built from local code (with some parallelism):
./hack/dev go test -parallel 4 -v -count=1 ./...
To run one go test against an engine built from local code:
./hack/dev go test -v -count=1 -run TestExtensionMount $(pwd)/core/integration/
When running core integration tests don't specify an individual file; always run with the full core/integration/ package or you will get errors (due to use of an init func in suite_test.go).
How to run linters locally?
To run all linters:
./hack/make lint
To list available linters:
> ./hack/make -l | grep lint
docs:lint lints documentation files
engine:lint lints the engine
lint runs all linters
sdk:all:lint runs all SDK linters
sdk:elixir:lint lints the Elixir SDK
sdk:go:lint lints the Go SDK
sdk:java:lint lints the Java SDK
sdk:typescript:lint lints the TypeScript SDK
sdk:python:lint lints the Python SDK
sdk:rust:lint lints the Rust SDK
The docs:lint is misleading as it only lints the Markdown in documentation (.md). Go snippets in documentation are linted in engine:lint while the others are linted in sdk:<name>:lint.
How to test SDK changes locally?
TypeScript:
- In
sdk/typescript, runnpm run build - In your
package.json, update@dagger.io/daggerto reference your local path. For example"@dagger.io/dagger": "<PATH TO DAGGER FORK>/dagger/sdk/typescript",
Python:
- While in a VirtualEnvironment, run
pip install <PATH TO DAGGER FORK>/sdk/python
Go:
- In your Go project, run
go mod edit -replace dagger.io/dagger=<PATH TO DAGGER FORK>/sdk/go - Then
go mod tidy