Cuberfile reference

The Cuberfile is a simple way to describe a deployment of an app on Kubernetes (in a way that is similar to a Capfile for Capistrano or a Procfile for Heroku).

You can apply it to the Kubernetes cluster (i.e. create or update your app) by executing cuber deploy.

The Cuberfile is also read by other Cuber commands in order to detect the current app name and the Kubernetes cluster.

You can keep the Cuberfile in the same folder of your app and track it with Git, or, if you prefer, you can keep it in a separate folder / repository. If you need different deployments or environments for the same application, you can create multiple Cuberfiles in different folders (e.g. cuber/staging/Cuberfile, cuber/production/Cuberfile).

A Cuberfile defines a DSL, described below, but you can also use Ruby code inside it, since the Cuberfile is just a Ruby file (this can be useful for example to read data from separate files or other sources).

app name

The name of your app.

It can only include lowercase letters, digits or dashes.

It is used as a namespace for your app on Kubernetes.

app 'my-app'

release version

The name of an existing release, in case you want to publish a past release of your app.

It’s only useful for rollbacks or to lock to a specific version. For normal deployments it is not necessary and you can omit it.

A release is an existing Docker image, built by a previous deployment, and still available in the Docker registry.

Each release has a unique name, thus ensuring that you can always rollback to a previous build. The release name is made of two parts: the first seven characters are the short commit hash, from the Git commit, while the digits after the dash represent a timestamp (YYYYMMDDHHMMSS). In this way you always know what version of your app is inside a release and when exactly the Docker image was built. Note that the commit hash alone would not be enough to uniquely identify an image, since the same application code can be transformed into an image in different ways (e.g. different Dockerfile, different or updated base image, different Buildpacks or other different configurations).

You can see the current release running in Kubernetes using cuber info.

You can see a list of all the available releases by visiting your Docker repository.

release 'abcdefg-20211201170000'

repo url, branch: nil

The URL (or local path) of your Git repository.

This repository must contain your app code and it will be cloned on your local machine in order to build the Docker image of your app.

The default branch of the repository will be used (usually HEAD points to the master or main branch). Otherwise you can pass a branch option with the name of a branch or tag.

Note that git clone must be able to authenticate to the repository in order to clone it.

# Use a local path (e.g. current directory)
repo '.'

# Use a remote URL
repo 'https://github.com/username/my-app.git'

buildpacks builder

Build a Docker image of your app automatically using Cloud Native Buildpacks (Buildpacks.io).

You can choose which builder you want to use (e.g. Heroku, Google, Paketo).

A builder is a set of buildpacks: a builder usually contains at least one buildpack for each language supported (e.g. Ruby, Node.js, Python, PHP, Java, Go, etc.).

You can see a list of the builders using pack builder suggest.

If you don’t include buildpacks in the Cuberfile, the dockerfile will be used to build the image of your application.

# Use Heroku buildpacks on Ubuntu 20 LTS (recommended)
buildpacks 'heroku/buildpacks:20'

# Use Paketo buildpacks
buildpacks 'paketobuildpacks/builder:full'

dockerfile path

The name (or relative path) of the Dockerfile inside your app repository.

The default behavior, if this option is not set, is to use the Dockerfile from your Git repository.

If you don’t want to use a Dockerfile you can use buildpacks instead.

dockerfile 'Dockerfile'

Here’s a Dockerfile for Rails application for example:

FROM ruby:3.0
ARG RAILS_ENV=production
RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash -
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y nodejs yarn postgresql-client default-mysql-client
RUN mkdir /app
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN gem install bundler
RUN bundle config set without 'development test'
RUN bundle install
COPY . .
RUN SECRET_KEY_BASE=`bin/rake secret` rails assets:precompile
EXPOSE 8080
CMD ["rails", "server", "-b", "0.0.0.0", "-p", "8080"]

Note that EXPOSE and CMD are useful for documentation, but ignored by Cuber. The port where your application must be listening is defined at runtime by the PORT environment variable. The command executed inside Kubernetes is defined with the proc option.

image name

The Docker repository where the image will be stored.

Include only the registry and image name, without any tag.

The tag will be added automatically by Cuber and represents the release name.

# Use the official Docker registry
image 'username/my-app'

# Use another container registry
image 'gcr.io/username/my-app'

cache enabled

You can enable or disable the Docker cache and Buildpacks cache.

When the cache is disabled the base image is also refreshed.

Turning the cache off, before running cuber deploy, is useful to get the latest security patches (otherwise, since the tags for the base image are often reused on Docker Hub, you may not get the most recent image).

In any case keeping the Docker cache enabled (default) will greatly improve the speed of a deployment.

If you want to skip the build process entirely and use an existing image, see the release option.

# Disable the build cache
cache false

dockerconfig path

The Docker config, which must contain the credentials to access to the Docker registry.

The path is an absolute path or relative to the directory where the Cuberfile is located.

The default is ~/.docker/config.json, otherwise you can set it here.

dockerconfig 'dockerconfig.json'

Make sure that your Docker config file contains the auth field with the username and password of the Docker registry. Depending on your OS, the default ~/.docker/config.json may only include a reference to an external secret store (like Keychain on MacOS), but the secret is not directly in the file. Since the file will be uploaded to Kubernetes, you need to include the auth directly in the file. In order to do this create a file (e.g. dockerconfig.json) like this:

{
  "auths": {
    "https://index.docker.io/v1/": {
      "auth": "c3R...zE2"
    }
  }
}

The auth value can generated with echo -n 'username:password' | base64.

kubeconfig path

The kubeconfig file contains the address and credentials to access to the Kubernetes cluster.

The path is an absolute path or relative to the directory where the Cuberfile is located.

Usually you can find your kubeconfig file in the ~/.kube folder.

Otherwise you can download it from your cloud provider using their online dashboard or their CLI.

kubeconfig 'kubeconfig.yml'

migrate cmd, check: nil

You can run a migration command during each release (e.g. database migrations).

Usually the migration command is executed in parallel with the deployment, so some pods may start before it is completed and some after it is completed.

If you want to ensure that your new app version starts only when the migration is completed, you can provide a check command: in this case your new release will not start until the check passes.

migrate 'rails db:migrate', check: 'rake db:abort_if_pending_migrations'

In the above example, we run the database migrations for a Rails application: the check command exits with a successful status (0) only when the migration is completed. If the check command fails (returns a non-zero exit code), your app does not start and the check command is retried. Only when the check command is successful your new release is started.

Note that the migration command runs inside the new release: this means that it always sees the most recent app code and the new env variables (as defined by the current Cuberfile).

As an alternative to migrate, if you want a manual, but simpler solution, you can use cuber run. For example you can deploy your new app version, without specifying a migrate option in the Cuberfile, and then run the migration command manually:

$ cuber run rails db:migrate
$ cuber restart

proc name, cmd, scale: 1, cpu: nil, ram: nil, term: 60, env: {}

Run and scale any process (web server, background workers, etc.) on Kubernetes.

This is probably the most important configuration, since it defines the processes to run on Kubernetes.

It is similar to what you would define in a Procfile, but it also includes other options.

# Start your web server or app
proc :web, 'your web server command'

# Start a specific web server (puma) and scale it to 10 pods
# Also start some background jobs (sidekiq) and scale it to 20 pods
proc :web, 'bundle exec puma', scale: 10
proc :worker, 'bundle exec sidekiq', scale: 20

# You can set resources and limits (CPU and RAM)
# e.g. 1 vCPU and 2GB of RAM
proc :web, 'bundle exec puma', cpu: 1, ram: 2

# You can grant a longer grace period for shutdown (e.g. for long running jobs)
proc :slowjobs, 'bundle exec sidekiq -t 590', term: 600

# You can also override the global env variables, only for a specific proc
proc :myproc, 'example command', env: { EXAMPLE: 'example123' }

web has a special meaning, since it’s the only process connected to the load balancer and thus accessible from the outside world. Your app must listen on the port defined by the PORT environment variable (which is defined automatically by Cuber).

All other proc names don’t have any special meaning. Just remember that the names can only contain lowercase letters.

The scale option controls the number of pods that run for that specific process: Kubernetes pods are somewhat similar to VMs and Heroku dynos and they are the unit that runs your application containers. You can easily scale them when you need more computational power. However remember that pods run inside your Kubernetes cluster and you will also need to scale it accordingly.

The cpu and ram options set the size of the pods. The cpu option sets the number of vCPUs reserved for each pod (but the pod can also use additional CPU if it’s available). For example, cpu: 1 means 1 vCPU, cpu: 2 means 2 vCPU, etc. You can also use fractions of a CPU: for example cpu: 0.5 means 500m of CPU time. The ram option sets the memory allocated for each pod (a pod cannot exceed that limit). For example, ram: 4 means 4GB of RAM. You can also use fractions (e.g. ram: 2.5, ram: 0.25).

In general, environment variables should be always defined globally using the global env option, however you can also pass them to a specific proc using the env: argument like in the example above.

cron name, schedule, cmd

Create cron jobs that run on Kubernetes.

The cron jobs spins up a new instance (pod) of your app every time it runs: this means that the cron job has access to your app code, tasks, environment, database, etc. like any other proc.

In order to define the cron job schedule you can use the cron syntax (if you are not familiar with it we recommend to try an online tool like crontab.guru). You can also use the special words @hourly, @daily, @weekly, @monthly, @yearly.

# Run a command every minute
cron :example, '* * * * *', 'my command'

# Run a command every day at 3am
cron :anotherjob, '0 3 * * *', 'my command'

# In a Rails app, for example, run a rake task every day
cron :mytask, '@daily', 'rake my:task'

env key, value, secret: false

Define environment variables (or secrets) for your application.

These env variable will be defined inside Kubernetes and are available to your application (inside proc, cron, migrate, etc.).

If the environment variable is a secret, you can set secret: true, for increased security inside Kubernetes.

If you keep the Cuberfile inside Git, then you should avoid to include the secret values directly in it: you can read the secret values from an external source, like a file or an environment variable already defined in your CI/CD environment.

# Define an environment variable EXAMPLE with the value "example123"
env 'EXAMPLE', 'example123'

# Define an environment variable RAILS_MASTER_KEY
# Since it is a secret value, we keep it out of Git and we read it from an external source
# We also mark it with secret: true to inform Kubernetes that it is a secret
env 'RAILS_MASTER_KEY', File.read('config/credentials/production.key').strip, secret: true

# Set a secret (MY_SECRET) that will be available in Kubernetes
# reading the value from an env variable (LOCAL_SECRET) defined in your CI/CD environment
env 'MY_SECRET', ENV['LOCAL_SECRET'], secret: true

health url

Configure the health checks for the web proc.

If the proc (container) doesn’t respond with successful status codes, it is considered not ready to accept new HTTP requests.

The health checks are always sent over HTTP, however if you specify https in the url, then the header X-Forwarded-Proto: https is added to the request (to simulate an HTTPS request with SSL termination).

health 'http://example.com/health'

lb key, value

Set special options or configurations for the load balancer (or for the ingress).

Although the load balancer for your app is created automatically, and works out of the box, your cloud provider may allow you to control additional settings of the load balancer using Kubernetes Annotations.

These settings are specific to a single provider and are not standardized.

Using lb you can pass these provider-specific settings to the Kubernetes LoadBalancer.

Depending on your provider, these settings can be useful for configuring SSL/TLS, static IPs, the load balancer size, etc.

For more information search “Kubernetes LoadBalancer Annotations” inside your cloud provider documentation (e.g. DigitalOcean, GKE, etc.).

# Enable HTTPS on DigitalOcean
# https://docs.digitalocean.com/products/kubernetes/how-to/configure-load-balancers/
lb 'service.beta.kubernetes.io/do-loadbalancer-protocol', 'https'
lb 'service.beta.kubernetes.io/do-loadbalancer-certificate-id', '1234-5678-9012-3456'

# Set a static IP on GKE
# https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer
lb 'kubernetes.io/ingress.global-static-ip-name', 'my-static-ip-name'

# Enable HTTPS on EKS
# https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/
lb 'service.beta.kubernetes.io/aws-load-balancer-backend-protocol', 'http'
lb 'service.beta.kubernetes.io/aws-load-balancer-ssl-cert', 'arn:aws:acm:REGION:USERID:certificate/ID'
lb 'service.beta.kubernetes.io/aws-load-balancer-ssl-ports', 'https'

ingress enabled

On some providers you may want to use a Kubernetes Ingress instead of a Kubernetes LoadBalancer.

By default Cuber will create a load balancer, which is compatible with all providers.

However, on some cloud providers (e.g. Google Cloud), you have more flexibility and more options if you use an Ingress.

You can easily enable it with this option.

# Enable Ingress (e.g. recommended for GKE)
ingress true

You can also pass some custom settings to the Ingress using lb.

ssl crt, key

Upload an SSL/TLS certificate and private key to the Kubernetes cluster and enable HTTPS.

If the ingress option is enabled, then this certificate is used automatically to encrypt the traffic between the client (browser) and your website.

If the ingress option is not enabled, then this option is only useful in combination with the lb options offered by your provider and you need to reference this secret explicitly by name.

The name of this secret inside Kubernetes is ssl.

# Upload an SSL/TLS certificate and private key
# If ingress is enabled, then HTTPS and this certificate are used automatically
ssl 'ssl/cert.pem', 'ssl/key.pem'

# Otherwise, if you are using a load balancer, reference the secret by name
# For example, on Vultr, use the secret named "ssl" for the load balancer
lb 'service.beta.kubernetes.io/vultr-loadbalancer-protocol', 'http'
lb 'service.beta.kubernetes.io/vultr-loadbalancer-https-ports', '443'
lb 'service.beta.kubernetes.io/vultr-loadbalancer-ssl', 'ssl'

Note that for some providers (e.g. DigitalOcean, EKS) the ssl option is useless, because they force you to keep the certificate outside Kubernetes: in that case you will need to upload your SSL/TLS certificate using their online dashboard or using their CLI and then reference it using lb.

For simple testing you can use a self-signed certificate:

$ openssl req -newkey rsa:2048 -x509 -sha256 -days 365 -nodes -out cert.pem -keyout key.pem -subj "/CN=example.com/O=example.com"

And then you can try HTTPS with curl -v -k https://HOSTIP.

In production you need to upload a real certificate: you can buy one from a certificate authority or, alternatively, use a Cloudflare Origin certificate and then use Cloudflare in front of your website. Some cloud providers may also support automatic TLS certificates for the load balancer using Let’s Encrypt: in that case follow the provider instructions (in combination with lb settings if needed).