Skip to Content

Migrating from Nginx Ingress Controller to Chainguard-maintained images

You must update the Kubernetes NGINX Ingress Controller to use the Chainguard‑maintained images to ensure compatibility and supportability.

Warning

Important

  • You might experience downtime while upgrading the NGINX Ingress Controller.
  • In this documentation, NGINX Ingress Controller refers to the Open-Source NGINX Ingress Controller maintained by Kubernetes.

Related topics

Statement of direction: End of support for Kubernetes NGINX Ingress controller

Best practice
Run the following command to check the current version of nginx-ingress-controller and record it. Make sure to save this information in case you need to revert the upgrade.

kubectl -n ingress-nginx get ds -o yaml | grep -i image:attach:xwiki:IT-Operations-Management.On-Premises-Deployment.BMC-Helix-IT-Operations-Management-Deployment.itomdeploy261.Upgrading.Migrating-from-Nginx-Ingress-Controller-to-Chainguard-maintained-images.WebHome@filename

Before you begin

Create a temporary directory and back up the secrets, configmap, and daemonset resources from the ingress-nginx namespace.

  1. To create a temporary directory at a location of your choice, run the following command:
    mkdir -p /opt/ingress-nginx-backup
    cd /opt/ingress-nginx-backup
  2. Navigate to the directory
  3. To back up the secret (my-tls-secret), run the following command:
    kubectl -n ingress-nginx get secrets my-tls-secret -o yaml > my-tls-secret.yaml
  4. To back up the configmap (ingress-nginx-controller), run the following command:
    kubectl -n ingress-nginx get cm ingress-nginx-controller -o yaml > ingress-nginx-controller_cm.yaml
  5.  To back up the daemonset, run the following command:
    kubectl -n ingress-nginx get ds ingress-nginx-controller -o yaml > ingress-nginx-controller_ds.yaml

To upgrade to Chainguard-maintained images

To upgrade your existing Kubernetes NGINX Ingress Controller deployment to use the Chainguard‑maintained images, you must update the image reference in the existing daemonset.

Important

The NGINX Ingress Controller image is published by BMC to Docker Hub and is built and maintained following Chainguard‑based practices.

  1. To get the ingress-class used for the NGINX Ingress Controller, run the following command:
    kubectl get ds -n ingress-nginx -o yaml | grep -i "\-\-ingress-class"
  2. Make sure that you synchronize the following images with your local Harbor repository:
    docker.io/bmchelix/lp0lz:ingress-nginx-controller-1.14.5
    docker.io/bmchelix/lp0lz:kube-webhook-certgen-1.14.5
    docker.io/bmchelix/lp0lz:ingress-nginx-controller-v1.13.9
    docker.io/bmchelix/lp0lz:kube-webhook-certgen-v1.13.9
  3. Download the attached ingress-scc.yaml file.
  4. Apply the ingress-scc.yaml file by using the following command:
    kubectl apply -f ingress-scc_<version>.yaml
    For example, kubectl apply -f ingress-scc_24.2.yaml
  5. Download the attached deploy.yaml file corresponding to the Ingress Controller version you are deploying:

  6. Make sure that the ingress-class that you verified in Step 1 is specified in the deploy.yaml file.

    spec:
          containers:
          - args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --ingress-class=nginx

    If it is not specified, add it:
    ingress-class=nginx

  7. To delete the jobs (ingress-nginx-admission-create and ingress-nginx-admission-patch), run the following command:

    kubectl delete job ingress-nginx-admission-create ingress-nginx-admission-patch -n ingress-nginx --ignore-not-found=true

    Important
    Ignore the errors or warnings if the ingress-nginx-admission-create and ingress-nginx-admission-patch jobs are not present.

  8. Make the following changes in the deploy.yaml file:

    1. Change the kind field of the ingress-nginx-controller from Deployment to DaemonSet

      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        labels:
          app.kubernetes.io/component: controller
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/part-of: ingress-nginx
          app.kubernetes.io/version: <version of the deploy.yaml>
        name: ingress-nginx-controller
        namespace: ingress-nginx

    2. (If upgrading to Ingress Controller 1.9.3 or later) Under Kind: Daemonset, change the spec.strategy field to spec.updateStrategy.

      spec:
        minReadySeconds: 0
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 1
          type: RollingUpdate

    3. In the args section, set the default certificate to my-tls-secret:

      spec:
            containers:
            - args:
              - /nginx-ingress-controller
              - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
              - --election-id=ingress-controller-leader
              - --controller-class=k8s.io/ingress-nginx
              - --ingress-class=nginx
              - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
              - --validating-webhook=:8443
              - --validating-webhook-certificate=/usr/local/certificates/cert
              - --validating-webhook-key=/usr/local/certificates/key
              - --default-ssl-certificate=ingress-nginx/my-tls-secret           # <<<<<<<<<<<<<<

    4. (If upgrading to Ingress Controller 1.9.5 or later) Under Kind: Daemonset, locate securityContext, and then set the value of the flag allowPrivilegeEscalation as true.

      securityContext:
                allowPrivilegeEscalation: true

       Important
      If you are using an external image repository, make sure that the ingress controller and webhook image references in the deploy.yaml file are updated to point to the images hosted in your external repository.

  9. Apply the Security Context Constraints (SCC) to service accounts by running the following commands:

    1. If the version of your OpenShift cluster is lower than 4.14:

      oc adm policy add-scc-to-user ingress-scc -z default -n ingress-nginx
      oc adm policy add-scc-to-user ingress-scc -z ingress-nginx-admission -n ingress-nginx
      oc adm policy add-scc-to-user ingress-scc -z ingress-nginx -n ingress-nginx

    2. If the version of your OpenShift cluster is 4.14 or higher:

      oc -n ingress-nginx annotate job.batch/ingress-nginx-admission-patch openshift.io/required-scc="ingress-scc"
      oc -n ingress-nginx annotate job.batch/ingress-nginx-admission-create openshift.io/required-scc="ingress-scc"
      oc -n ingress-nginx annotate daemonset.apps/ingress-nginx-controller openshift.io/required-scc="ingress-scc"
  10. Create a Docker registry secret in the ingress-nginx namespace to allow the cluster to authenticate and pull the Chainguard-maintained image by using the following commands:

    kubectl create secret docker-registry bmc-dtrhub \
      -n ingress-nginx \
      --docker-server=docker.io \
      --docker-username=<docker-username> \
      --docker-password='<docker-password>'
     

    Important
    If you are using an external image repository instead of Docker Hub, replace docker.io with the appropriate image registry host when creating the Docker registry secret.

  11. To patch the daemonset to reference the Docker registry pull secret, run the following command:

    kubectl patch ds ingress-nginx-controller -n ingress-nginx \
      --type='json' \
      -p='[{"op":"add","path":"/spec/template/spec/imagePullSecrets","value":[{"name":"bmc-dtrhub"}]}]'
     

    Important
    If imagePullSecrets is already defined, merge the new secret with the existing list instead of applying this patch.

  12. To apply the deploy.yaml, run the following command:kubectl apply -f deploy.yaml
  13. To verify that the NGINX Ingress Controller pods are running on all worker nodes, run the following command:kubectl -n ingress-nginx get pods

  14. To verify the version of the NGINX Ingress Controller from one of the pod logs, run the following command

  15. kubectl logs <ingress controller pod> -n ingress-nginx | less

    Where -n ingress-nginx is the ingress namespace.
    The pod logs should have the same version of the NGINX Ingress Controller that you installed.

  16. To make sure that the same ports of the NGINX Ingress Controller service are configured in the load balancer, run the following command:

    kubectl -n ingress-nginx get service

  17. To verify the version of the Ingress Controller, run the following command:

    kubectl -n ingress-nginx describe pod <pod name> | grep -i image

    Example

    kubectl -n ingress-nginx describe pod ingress-nginx-controller-cb44m | grep -i image

    Sample output:

    kubectl -n ingress-nginx describe pod ingress-nginx-controller-86px9 | grep -i image
        Image: docker.io/bmchelixhub/lp0lz:ingress-nginx-controller-v1.14.5

To validate the ingress controller configurations

Important
If upgrading the NGINX Ingress Controller to version 1.9.3 and higher, perform this step.

To validate the ingress controller configurations, perform the following steps:

  1. Identify the configmap name by running the following command

    kubectl get cm -n <ingress_nginx_namespace>

  2. Change the configmap name to use the configmap in your environment by running the following command:

    kubectl edit cm ingress-nginx-controller -n ingress-nginx

    data:
      enable-underscores-in-headers: "true"
      annotations-risk-level: "Critical" (only applicable for Ingress version 1.12.x and above)
      proxy-body-size: 250m
      server-name-hash-bucket-size: "1024"
      ssl-redirect: "false"
      use-forwarded-headers: "true"
      worker-processes: "40"
      allow-snippet-annotations: "true"
      large-client-header-buffers: "4 64k"

    Important
    The configurations shown above are mandatory. Apart from these, you can retain any other configurations according to your requirements.

  3. Restart the NGINX Ingress Controller
    1. Run the following command to restart the daemonset:
      kubectl -n ingress-nginx rollout restart ds ingress-nginx-controller
    2. Run the following command to restart the deployment:kubectl -n ingress-nginx rollout restart deployment ingress-nginx-controller
  4. Verify the upgrade.
    1. To monitor the pods as they restart with the updated Chainguard-maintained image, run the following command:
      kubectl -n ingress-nginx get pods -w
    2. Verify the updated image is running, and the output must include an updated image reference:
      image:attach:xwiki:IT-Operations-Management.On-Premises-Deployment.BMC-Helix-IT-Operations-Management-Deployment.itomdeploy261.Upgrading.Migrating-from-Nginx-Ingress-Controller-to-Chainguard-maintained-images.WebHome@filename docker.io/bmchelix/lp0lz:ingress-nginx-controller-v1.14.5
    3. To verify the updated Chainguard-maintained image is successfully pulled,  run the following command:
      kubectl describe pod -n ingress-nginx <ingress-pod-name> | grep -i image
      The expected output must include the updated Chainguard-maintained image
      image:attach:xwiki:IT-Operations-Management.On-Premises-Deployment.BMC-Helix-IT-Operations-Management-Deployment.itomdeploy261.Upgrading.Migrating-from-Nginx-Ingress-Controller-to-Chainguard-maintained-images.WebHome@filename docker.io/bmchelix/lp0lz:ingress-nginx-controller-v1.14.5

 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

BMC Helix IT Operations Management deployment 26.1