💾 Archived View for capsule.adrianhesketh.com › 2021 › 07 › 17 › go-cdk-building-go-lambda-functions captured on 2021-12-03 at 14:04:38. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-11-30)
-=-=-=-=-=-=-
Go Lambda functions are structured as an individual Linux executable that uses the `github.com/aws/aws-lambda-go/lambda` package's `Start` function to run the Lambda runtime to process requests.
Requests are then processed by the `handle` function that is passed into `lambda.Start`.
The signature of the `handle` method is different depending on the event that's triggering the invocation. For example, API Gateway expects a return value, but SQS or EventBridge don't.
While it's possible to trigger AWS Lambda functions with arbitrary JSON payloads using the AWS CLI, SDK or in the console, the vast majority of time, your Lambda function will be triggered by an AWS service, like API Gateway.
AWS provides structured types for Lambda triggers in the `github.com/aws/aws-lambda-go/events` package, so your Lambda handler will look something like this:
package main
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
)
func main() {
// lambda.Start(HandleDynamoDBStream)
lambda.Start(HandleAPIGateway)
}
func HandleDynamoDBStream(ctx context.Context, event events.DynamoDBEvent) error {
//TODO: Put your logic here.
}
func HandleAPIGateway(ctx context.Context, req events.APIGatewayProxyRequest) (resp events.APIGatewayProxyResponse, err error) {
//TODO: Put your logic here.
}
The best way to build Go Lambda function is to use the `github.com/aws/aws-cdk-go/awscdk/awslambdago` package. is a higher level construct than `github.com/aws/aws-cdk-go/awscdk/awslambda` and will automatically build your Lambda functions.
Even though I've used the Node.js equivalent, I didn't spot that this package existed, and wasted a bit of time using the lower level `awslambda` package until my colleague Matthew Murray pointed me at this.
If you don't have Go installed, it will try and build the functions in a Docker container.
The Go executables can be made smaller by stripping out the symbol table and debug information (`-s`) and omitting the DWARF symbol table (`-w`). This doesn't make stack traces unreadable, so it's quite appropriate for production use.
bundlingOptions := &awslambdago.BundlingOptions{
GoBuildFlags: &[]*string{jsii.String(`-ldflags "-s -w"`)},
}
// notFound.
notFound := awslambdago.NewGoFunction(stack, jsii.String("notFoundHandler"), &awslambdago.GoFunctionProps{
Runtime: awslambda.Runtime_GO_1_X(),
Entry: jsii.String("../api/notfound"),
Bundling: bundlingOptions,
})
This package is only useful if you already have a process that has built the Lambda functions already.
In most cases, you'll want to use the `github.com/aws/aws-cdk-go/awscdk/awslambdago` package instead which builds your code for you.
If you use the the `awslambda` package, you must set:
notFound := awslambda.NewFunction(stack, jsii.String("notFoundHandler"), &awslambda.FunctionProps{
Runtime: awslambda.Runtime_GO_1_X(),
Code: awslambda.Code_FromAsset(jsii.String("../api/notfound"), &awss3assets.AssetOptions{}),
Handler: jsii.String("lambdaHandler"),
})
I use a Mac as my work machine, but at the time of writing Lambda requires the Go program to be compiled for Linux on the x64 processor architecture.
On my Mac, if I build a Go program with the `go build` command, it gets compiled for that operating system and processor architecture. To build it for Linux on x64, the `GOOS` (Go operating system) and `GOARCH` (Go architecture) environment variables need to be set:
GOOS=linux GOARCH=amd64 go build
That command produces a Go binary for a Linux x64 architecture that can run on Lambda, but the name of the executable will be the name of the current directory. To force the name to be `lambdaHandler`, the `-o` flag can be used.
GOOS=linux GOARCH=amd64 go build -o lambdaHandler .
Finally, the Go executable can be made smaller by stripping out the symbol table and debug information (`-s`) and omitting the DWARF symbol table (`-w`). This doesn't make stack traces unreadable, so it's quite appropriate for production use.
GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o lambdaHandler .
The flags are documented at [0]
Now we have the command to run, it's just a case of going into each directory (`cd`) and building the program with the command before we run `cdk deploy`.
My first pass at this was to use 4 lines of shell script. Unfortunately, it needs a lot of unpacking to understand...
find ./api -type f -name "*main.go" |
xargs --no-run-if-empty dirname |
xargs readlink -f |
awk '{print "cd "$1" && GOOS=linux GOARCH=amd64 go build -ldflags=\"-s -w\" -o lambdaHandler main.go\0"}' |
sh -v
First, the `find` command finds all of the `main.go` files in the `./api` path and the output is piped into `xargs`.
The first `xargs` calls `dirname` for each `main.go` file and strips `/xxxx/main.go` back to just the directory name `/xxxx/`.
Next, `xargs` is used again to run `readlink` to get the fully qualified path of each directory instead of the relative path.
`awk` is then used to construct a shell command print out the `go build` command for each line.
Finally, the output is passed into `sh -v` which shows both the output of the `go build` command, but also prints out the command that was executed.
Unfortunately, while the commands work great on Linux (and therefore Github Actions) they don't work on MacOS, because `xargs` and `readlink` don't have the same rich set of options that their Linux counterparts provide.
If your team is building Go programs, you might decide to run a build using Go instead. This has a couple of benefits:
But it has a downside of being a *lot* longer than 5 lines:
package main
import (
"fmt"
"io/fs"
"os"
"os/exec"
"path"
"path/filepath"
"time"
)
func main() {
start := time.Now()
fmt.Println("Finding Go Lambda function code directories...")
dirs, err := getDirectoriesContainingMainGoFiles("./api")
if err != nil {
fmt.Printf("Failed to get directories containing main.go files: %v\n", err)
os.Exit(1)
}
fmt.Printf("%d Lambda entrypoints found...\n", len(dirs))
for i := 0; i < len(dirs); i++ {
fmt.Printf("Building Lambda %d of %d...\n", i+1, len(dirs))
err = buildMainGoFile(dirs[i])
if err != nil {
fmt.Printf("Error: %v\n", err)
os.Exit(1)
}
}
fmt.Printf("Built %d Lambda functions in %v\n", len(dirs), time.Now().Sub(start))
}
func buildMainGoFile(path string) error {
// GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o lambdaHandler .
cmd := exec.Command("go", "build", "-ldflags=-s -w", "-o", "lambdaHandler")
cmd.Dir = path
cmd.Env = append(os.Environ(),
"GOOS=linux",
"GOARCH=amd64")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return fmt.Errorf("error running command: %w", err)
}
if exitCode := cmd.ProcessState.ExitCode(); exitCode != 0 {
return fmt.Errorf("non-zero exit code: %v", exitCode)
}
return nil
}
func getDirectoriesContainingMainGoFiles(srcPath string) (paths []string, err error) {
filepath.Walk(srcPath, func(currentPath string, info fs.FileInfo, err error) error {
if info.IsDir() {
// Continue.
return nil
}
d, f := path.Split(currentPath)
if f == "main.go" {
paths = append(paths, d)
}
return nil
})
if err != nil {
err = fmt.Errorf("failed to walk directory: %w", err)
return
}
return
}
I just named the file `build-lambda.go` and run it with `go run build-lambda.go` in my `Makefile` before running `deploy` to make sure that all of the Lambda functions have been built.
build-lambda: go run build-lambda.go deploy: build-lambda cd cdk && cdk deploy
Hopefully this will save you a few minutes on your next Go, Lambda and CDK project!
Github Actions CI/CD for Go AWS CDK projects