💾 Archived View for makeworld.space › gemlog › 2020-11-13-go_modules.gmi captured on 2024-08-18 at 17:12:43. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-11-30)
-=-=-=-=-=-=-
I've seen many people online talk about liking Go[1] and using it, but being confused by its dependency system, called Go modules. This blog post aims to provide a simple introduction with examples. It focuses mostly on Unix-based systems like Linux and macOS over Windows.
This post does not cover all possible ways of using Go modules. It's just a simple introduction with the most common use cases.
You should know about Semantic Versioning[2]. Pretty much all Go modules are expected to follow it, and it will help you when upgrading modules or releasing your own.
I assume you already have Go installed. Go modules have preliminary support since Go 1.11, but Go module support improves with with each release. I'd recommend installing at least Go 1.13, but install the latest version if you can.
To begin the setup, you need to be aware of some environment variables. By default `GOPATH` is set to `~/go`, and `GOBIN` is set to `~/go/bin`. I would not recommend changing this. There is also `GOMODCACHE` (added in Go 1.15) which defaults to `GOPATH[0]/pkg/mod`, which would be `~/go/pkg/mod` if you don't change any variables. It holds the cache of modules you've downloaded.
There is one other variable, `GO111MODULE`. This variable controls whether Go modules are enabled and being used to manage dependencies. If it's set to `auto` or `on`, you should be good. If it's set to `off`, then you must change it. If you're using a Go version earlier than 1.13, you must set it to `on`, as the older `auto` behavior is not appropriate for this tutorial.
To check your current setup, you can run `go env` and see all the variables Go is working with.
You can skip this section if you just want to start using Go modules. But this might be helpful for people coming from other languages like Python or Rust.
Most programming languages handle dependencies by having a official registry/repository of packages that users can upload to. Think of PyPI, CPAN, NPM, crates.io, and more. Go takes a much more decentralized approach. Modules can exist anywhere on the Web, and they are defined with an HTTP(S) path.
For example, a valid module path would be `github.com/username/repo`. It could also be `mydomain.com/foo`. In either case, to download the module Go will first try to download from a proxy (`GOPROXY`), and then fallback to looking at the HTML of the URL provided to find where the code lives. For more info on this process and the HTML required, you can read more here[3].
You will not need to bother with any of this for most Git hosting providers. GitHub, Gitlab, Gitea, etc., will all create the needed HTML for you, and you can just specify a module path like `github.com/username/repo`.
First, your module needs a folder. To keep things simple, the path to this folder should double as its module path. So if you're using GitHub, your username is `user`, and your project is called `foo`, your project should live at `$GOPATH/src/github.com/user/foo`, which by default will be `~/go/src/github.com/user/foo`.
Then go into your project folder and run `go mod init`.
$ mkdir -p $GOPATH/src/github.com/user/foo $ cd $GOPATH/src/github.com/user/foo $ go mod init go: creating new go.mod: module github.com/user/foo
If you're really against this layout, you're welcome to create your projects anywhere. The only difference for this entire tutorial is that you run `go mod init` differently. You will have to specify the module path, since Go can't infer it.
$ go mod init github.com/user/foo go: creating new go.mod: module github.com/user/foo
Running `go mod init` will create a `go.mod` file in the project folder. This is how Go declares its dependencies -- with a `go.mod` file, and also a `go.sum` file that we'll see later. You should always be uploading these files with the rest of your code.
$ cat go.mod module github.com/user/foo go 1.15
Besides declaring your dependencies, the `go.mod` file also tells Go that this project really is a module, and gives it its path.
Most of the time you shouldn't need to edit `go.mod` or `go.sum` by hand. Instead you should let it be changed by running commands.
Alright, now you have a project, presumably with some code, and a `go.mod` file. Time to add some dependencies. The main commands for that are `go get` and `go mod tidy`.
You use `go get` to add dependencies. Here are some examples:
go get github.com/bar/baz # Latest version go get github.com/bar/baz@master # Latest commit on master go get github.com/bar/baz@e0ffd3a # The module at commit e0ffd3a go get github.com/bar/baz@v1.2.3 # Version 1.2.3 (using git tags)
`go get` accepts some other flags that I don't need to cover here. One flag to avoid is `-u`, as it updates all the downstream dependencies of a module (not changing a major version), which could cause breakage.
My suggestion would be to use the commit or tag syntax as much as possible, so that you know explicitly what version of dependencies you're adding, and to not use any flags.
The other important command after adding dependencies is `go mod tidy`. This command cleans up your `go.mod` and `go.sum` files, removing unused entries and adding missing ones. It's good to run this right after adding a dependency, and before committing your code. You can run this command multiple times without any worry, if everything is in order than it will just change nothing.
You can now use the dependency you added in your code, using the module path.
import (
"github.com/bar/baz"
)
func main() {
baz.Something()
}
This is the part that gets a bit trickier. In Go, the module path changes with every major version update. So the path for v1.0.0 of baz, all the way to v1.99.99 of baz is still `github.com/bar/baz`. But once baz v2.0.0 is released, the module path is now `github.com/bar/baz/v2`. Note that's not `@v2`, but `/v2`.
This happens because major versions are supposed to represent breaking changes. So it's important to allow your project to be able to use v2, while one of your dependencies still uses v1. This is achieved by making the module paths different.
The only changes you need to make is to use a slightly different `go get` command, and to update your code import paths.
go get github.com/bar/baz/v2 # Latest version of v2 go get github.com/bar/baz/v2@e0ffd3a # The module at commit e0ffd3a (must be a commit after v2 release) go get github.com/bar/baz/v2@v2.2.3 # A specific v2 version
import (
"github.com/bar/baz/v2" // Uses the new path
)
func main() {
baz.Something() // Module name is still 'baz', not 'v2' or something
}
The simplest way to publish a module is to use git, and a git host. Examples include GitHub, Gitlab, sr.ht, etc. Upload your Go code there, and make sure it has at least a `go.mod` file. Now anyone can use it as a module!
You should probably have versioning though, and you can do that with git tags, or by using the releasing system on your git host. For example, on GitHub you can click the Releases tab, and then "Draft a new release".
Doing simple tags with git is pretty easy as well:
git tag v1.2.3 # Whatever version this commit is git push --tags
Make sure your release name follows Semantic Versioning[4], and starts with a `v`!
So, you've been working on a v1 module for a while, but now you need to make a breaking change. This means you need to release a new major version.
Keep in mind, this is not required from v0 to v1, read about Semantic Versioning[5] to know why. But if you're already on v1 and want to make a breaking change, then read on.
There are a couple different ways to do this. Assuming you don't need compatibility with Go versions older than 1.9.7 or 1.10.3, than this method should work fine. If you do, you can read more here[6].
1. Update your `go.mod` file to include a `/v2` at the end of the module path.
$ cat go.mod module github.com/user/foo/v2 go 1.15
2. Update all your internal import statements to use the new `/v2` path. Anytime you import your own module in your code, you need to update that import.
3. Commit, push, tag, push tag.
# Commit and push, and then... git tag v2.0.0 git push --tags
The Go blog has a series on using Go modules, here's the first post[7].
The official Go Wiki has a page on Modules[8].
There is also an official technical reference[9], which should be considered the definitive/primary source for any info. It applies to Go 1.15 and later.
3: https://golang.org/ref/mod#vcs
6: https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher
7: https://blog.golang.org/using-go-modules
8: https://github.com/golang/go/wiki/Modules