Upgrading Nginx Ingress Controller

Before you begin

Back up of the secrets, configmap, and the daemonset of the ingress-nginx namespace:

1. To create a temporary directory at a location of your choice, run the following command:

   mkdir -p <directory location>/<directory name>

   Example:

   mkdir -p /opt/ingress-nginx-backup
2. Log in to the directory.

3. To take back up of secret (my-tls-secret), run the following command:

   kubectl get secrets my-tls-secret -n ingress-nginx -o yaml >> my-tls-secret.yaml

4. To back up the configmap (ingress-nginx-controller), run the following command:

   kubectl -n ingress-nginx get cm ingress-nginx-controller -o yaml >> ingress-nginx-controller.yaml

5. To back up the daemonset, run the following command:

   kubectl -n ingress-nginx get ds ingress-nginx-controller -o yaml >> ingress-nginx-controller_ds.yaml

   The backed-up files are saved in the directory that you created, for example, ingress-nginx-backup.

To upgrade NGINX Ingress Controller

1. To get the ingress-class used for the NGINX Ingress Controller, run the following command:

   kubectl get ds -n ingress-nginx -o yaml | grep -i "\-\-ingress-class"

   Look for --ingress-class in the command output.
   Example: --ingress-class=nginx

2. Based on your Kubernetes or OpenShift version, run one of the following commands to get the deploy.yaml file for the NGINX Ingress Controller:

   • To get deploy.yaml file for Nginx Ingress Controller version 1.7.0:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml

   • To get deploy.yaml file for Nginx Ingress Controller version 1.8.1:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.1/deploy/static/provider/cloud/deploy.yaml

   • To get deploy.yaml file for Nginx Ingress Controller version 1.9.3:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.3/deploy/static/provider/cloud/deploy.yaml

     Important

     If you are not able to download Nginx Ingress Controller version 1.9.3, use version 1.9.5 or 1.9.6.

   • To get deploy.yaml file for Nginx Ingress Controller version 1.9.5:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.5/deploy/static/provider/cloud/deploy.yaml

   • To get deploy.yaml file for Nginx Ingress Controller version 1.9.6:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.6/deploy/static/provider/cloud/deploy.yaml

   • To get deploy.yaml file for Nginx Ingress Controller version 1.11.1:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.1/deploy/static/provider/cloud/deploy.yaml

   • To get deploy.yaml file for Nginx Ingress Controller version 1.11.2:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.2/deploy/static/provider/cloud/deploy.yaml

   • To get deploy.yaml file for Nginx Ingress Controller version 1.11.5:

     wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.5/deploy/static/provider/cloud/deploy.yaml

   
   

3. Make sure that the ingress-class that you verified in Step 1 is specified in the deploy.yaml file.

   spec:
         containers:
        - args:
          - /nginx-ingress-controller
          - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
          - --election-id=ingress-controller-leader
          - --controller-class=k8s.io/ingress-nginx
          - --ingress-class=nginx

   If it is not specified, add it:
   ingress-class=nginx

4. To delete the jobs (ingress-nginx-admission-create and ingress-nginx-admission-patch), run the following command:

   kubectl delete job ingress-nginx-admission-create ingress-nginx-admission-patch -n ingress-nginx --ignore-not-found=true
   

   Important

   Ignore the errors or warnings if the ingress-nginx-admission-create and ingress-nginx-admission-patch jobs are not present. 

5. Make the following changes in the deploy.yaml file:

   1. Change the kind field of the ingress-nginx-controller from Deployment to DaemonSet

      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        labels:
          app.kubernetes.io/component: controller
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/part-of: ingress-nginx
          app.kubernetes.io/version: <version of the deploy.yaml>
        name: ingress-nginx-controller
       namespace: ingress-nginx

   2. (If upgrading to Ingress Controller 1.9.3 or later) Under Kind: Daemonset, change the spec.strategy field to spec.updateStrategy.

      spec:
        minReadySeconds: 0
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 1
          type: RollingUpdate

   3. In the args section, set the default certificate to my-tls-secret:

      spec:
            containers:
           - args:
             - /nginx-ingress-controller
             - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
             - --election-id=ingress-controller-leader
             - --controller-class=k8s.io/ingress-nginx
             - --ingress-class=nginx
             - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
             - --validating-webhook=:8443
             - --validating-webhook-certificate=/usr/local/certificates/cert
             - --validating-webhook-key=/usr/local/certificates/key
             - --default-ssl-certificate=ingress-nginx/my-tls-secret           # <<<<<<<<<<<<<<

   4. (If upgrading to Ingress Controller 1.9.5 or later) Under Kind: Daemonset, locate securityContext, and then set the value of the flag allowPrivilegeEscalation as true.

      securityContext:
                allowPrivilegeEscalation: true
6. To apply the deploy.yaml, run the following command:kubectl apply -f deploy.yaml
7. Apply the Security Context Constraints (SCC) to service accounts by running the following commands:

   1. If the version of your OpenShift cluster is lower than 4.14:

      oc adm policy add-scc-to-user ingress-scc -z default -n ingress-nginx
      oc adm policy add-scc-to-user ingress-scc -z ingress-nginx-admission -n ingress-nginx
      oc adm policy add-scc-to-user ingress-scc -z ingress-nginx -n ingress-nginx

   2. If the version of your OpenShift cluster is 4.14 or higher:

      oc -n ingress-nginx annotate job.batch/ingress-nginx-admission-patch openshift.io/required-scc="ingress-scc"
      oc -n ingress-nginx annotate job.batch/ingress-nginx-admission-create openshift.io/required-scc="ingress-scc"
      oc -n ingress-nginx annotate daemonset.apps/ingress-nginx-controller openshift.io/required-scc="ingress-scc"

8. To verify that the NGINX Ingress Controller pods are running on all worker nodes, run the following command:

9. kubectl -n ingress-nginx get pods

10. To verify the version of the NGINX Ingress Controller from one of the pod logs, run the following command:

    kubectl logs <ingress controller pod> -n ingress-nginx | less

    Where -n ingress-nginx is the ingress namespace.
    The pod logs should have the same version of the NGINX Ingress Controller that you installed.
     

11. To make sure that the same ports of the NGINX Ingress Controller service are configured in the load balancer, run the following command:

    kubectl -n ingress-nginx get service

12. To verify the version of the Ingress Controller, run the following command:

    oc -n ingress-nginx describe pod <pod name> | grep -i image

    Example:

    oc -n ingress-nginx describe pod ingress-nginx-controller-cb44m | grep -i image

    Sample output:
    
    In the example, the version of the Ingress Controller installed is 1.9.6. 
     

13. To get the ingress-nginx-controller service port details, run the following command:

    oc -n ingress-nginx get svc

    Example output:

    For example, you will see output like below:
    $ oc -n ingress-nginx get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    ingress-nginx-controller LoadBalancer 10.43.43.12 XXX.XXX.XXX.XXX 80:31764/TCP,443:31864/TCP 24h
    ingress-nginx-controller-admission ClusterIP 10.43.46.181 <none> 443/TCP 24h

    Note the port details from the command output. For example:

    • Node port 31764 maps to port 80 and must be used for HTTP connection.

    • Node port 31864 maps to port 443 and must be used for HTTPS connection.

14. The Validating Webhook Configuration blocks the deployment of a few Ingress objects. To delete the Validating Webhook Configuration, run the following command:

    kubectl delete  ValidatingWebhookConfiguration ingress-nginx-admission

To validate the ingress controller configurations

To validate the ingress controller configurations, perform the following steps:

1. Identify the configmap name by running the following command:

   kubectl get cm -n <ingress_nginx_namespace>

2. Change the configmap name to use the configmap in your environment by running the following command:

   kubectl edit cm ingress-nginx-controller -n ingress-nginx
   
   data:
     enable-underscores-in-headers: "true"
     proxy-body-size: 250m
     server-name-hash-bucket-size: "1024"
     ssl-redirect: "false"
     use-forwarded-headers: "true"
    worker-processes: "40"
    allow-snippet-annotations: "true"
    large-client-header-buffers: "4 64k"

   Important

   The configurations shown above are mandatory. Apart from these, you can retain any other configurations according to your requirements.

3. To restart the daemonset, run the following command:

   oc -n ingress-nginx rollout restart ds ingress-nginx-controller

Where to go from here