Multiple Kubernetes clusters for Auto DevOps

When using Auto DevOps, you can deploy different environments to different Kubernetes clusters.

The Deploy Job template used by Auto DevOps defines three environment names:

  • review/ (every environment starting with review/)
  • staging
  • production

These environments are tied to jobs using Auto Deploy, so they must have different deployment domains. You must define separate KUBE_CONTEXT and KUBE_INGRESS_BASE_DOMAIN variables for each of the three environments.

Deploy to different clusters

To deploy your environments to different Kubernetes clusters:

  1. Create Kubernetes clusters.
  2. Associate the clusters to your project:
    1. Install a GitLab Agent on each cluster.
    2. Configure each agent to access your project.
  3. Install NGINX Ingress Controller in each cluster. Save the IP address and Kubernetes namespace for the next step.
  4. Configure the Auto DevOps CI/CD Pipeline variables
    • Set up a KUBE_CONTEXT variable for each environment. The value must point to the agent of the relevant cluster.
    • Set up a KUBE_INGRESS_BASE_DOMAIN. You must configure the base domain for each environment to point to the Ingress of the relevant cluster.
    • Add a KUBE_NAMESPACE variable with a value of the Kubernetes namespace you want your deployments to target. You can scope the variable to multiple environments.

For deprecated, certificate-based clusters:

  1. Go to the project and select Infrastructure > Kubernetes clusters from the left sidebar.
  2. Set the environment scope of each cluster.
  3. For each cluster, add a domain based on its Ingress IP address.
note
Cluster environment scope is not respected when checking for active Kubernetes clusters. For a multi-cluster setup to work with Auto DevOps, you must create a fallback cluster with Cluster environment scope set to *. You can set any of the clusters you’ve already added as a fallback cluster.

Example configurations

Cluster name Cluster environment scope KUBE_INGRESS_BASE_DOMAIN value KUBE CONTEXT value Variable environment scope Notes
review review/* review.example.com path/to/project:review-agent review/* A review cluster that runs all Review Apps.
staging staging staging.example.com path/to/project:staging-agent staging Optional. A staging cluster that runs the deployments of the staging environments. You must enable it first.
production production example.com path/to/project:production-agent production A production cluster that runs the production environment deployments. You can use incremental rollouts.

Test your configuration

After completing configuration, test your setup by creating a merge request. Verify whether your application deployed as a Review App in the Kubernetes cluster with the review/* environment scope. Similarly, check the other environments.