Deploying and configuring the NGINX Open Source Ingress Controller


Before deploying

Some content is unavailable due to permissions.

(

Some content is unavailable due to permissions.

), you must deploy and configure an NGINX Ingress Controller. The Ingress Controller serves as a reverse proxy and load balancer in Kubernetes. It works by implementing a Kubernetes Ingress, which is an API object that defines rules for routing external traffic to services within the Kubernetes cluster. 

The NGINX Ingress Controller adds an abstraction layer to traffic routing by accepting traffic from external sources, such as the internet, and load-balancing it to Kubernetes pods running inside the platform. 

When a request arrives from an external source, the Ingress Controller determines which service should handle the request based on the rules defined in the Ingress resource. It then routes the traffic to the appropriate service and monitors the status of pods. If necessary, it automatically updates load-balancing rules. 

Important

Starting with

Some content is unavailable due to permissions.

 version 24.3, we support the enterprise edition of the F5 NGINX Plus Ingress Controller.
You can use either the enterprise edition of the F5 NGINX Plus Ingress Controller or the open-source version.
For more information about the NGINX Plus Ingress Controller, see Configuring the NGINX Plus Ingress Controller.

The certified versions of NGINX Ingress Controller with the Kubernetes and OpenShift orchestration platform are as follows:

Nginx Ingress Controller version

Supported Kubernetes version

OpenShift version

1.7.0 and 1.11.5

1.25  

4.12

1.8.1 and 1.11.5

1.25 and 1.26

4.13

1.9.6 and 1.11.5

1.26 and 1.27

4.13, 4.14, and 4.15

1.10.1 and 1.11.5

1.27 and 1.28

4.14 and 4.15

1.11.1 and 1.11.5

1.28, 1.29, and 1.30

4.15

1.11.2 and 1.11.5

1.30

4.15 and 4.16

Important

You will need cluster-admin privileges to deploy the NGINX Ingress Controller.

To deploy the NGINX Ingress Controller for Openshift

  1. Download the attached ingress-scc.yaml file.
  2. Apply the ingress-scc.yaml file by using the following command: oc apply -f ingress-scc_<version>.yaml
    Example:
    oc apply -f ingress-scc_24.2.yaml
  3. Based on your OpenShift version, run one of the following commands to get the deploy.yaml file for the NGINX Ingress Controller:

    To get deploy.yaml file for Nginx Ingress Controller:
    wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-<version>/deploy/static/provider/cloud/deploy.yaml

    If you change the ingress configuration, make sure that the INGRESS_CLASS value in the configs/infra.config file matches the class in your ingress definition.

  4. Make the following changes in the deploy.yaml file: 
    1. Change the kind field of the ingress-nginx-controller from Deployment to DaemonSet.
      Daemonset ensures that the controller machine runs on every worker node.
      The namespace for the NGINX Ingress Controller is ingress-nginxapiVersion: apps/v1
      kind: DaemonSet
      metadata:
        labels:
          app.kubernetes.io/component: controller
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/part-of: ingress-nginx
          app.kubernetes.io/version: <version of the deploy.yaml>
        name: ingress-nginx-controller
       namespace: ingress-nginx
    2. Under kind: Daemonset, change the spec.strategy field to spec.updateStrategy spec:
        minReadySeconds: 0
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 1
          type: RollingUpdate
    3. Under kind: Daemonset, locate securityContext, and then set the value of the flag allowPrivilegeEscalation as truesecurityContext:
                allowPrivilegeEscalation: true
  5. Apply the deploy.yaml file by using the following command: kubectl apply -f deploy.yaml
  6. To verify that the pods are running on all worker nodes, run the following command: kubectl get ds -n ingress-nginx

    Sample output: 

    NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
    ingress-nginx-controller   11        11        11      11           11          kubernetes.io/os=linux   15d

    The values of the columns Desired, Current, Ready, Up-to-date, and Available should be equal to the number of worker nodes.

  7. (Skip this step if you have attached an SSL certificate at the load balancer and are sending HTTP or HTTPS traffic to ingress.)
    Create a secret with the certificate and key to be mounted on the Ingress Controller pods by using the following command:

    kubectl create secret tls my-tls-secret --cert=/path/to/cert.pem --key=/path/to/privkey.pem -n ingress-nginx

    Important

    Make sure cert.pem contains the signed certificate and the complete chain of CA certs.

    ​​​​​​

  8. (Skip this step if you have attached an SSL certificate at the load balancer and are sending HTTP or HTTPs traffic to ingress.)
    Edit the daemonset as described:
    1. Add the secret that you created in the args section.
    2. Run the following command:

      kubectl edit daemonset ingress-nginx-controller -n ingress-nginx
    3. In the output, set the ingress-class parameter according to your requirement:

      ...
         spec:
            containers:
            - args:
              - /nginx-ingress-controller
              - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
              - --election-id=ingress-controller-leader
              - --controller-class=k8s.io/ingress-nginx
              - --ingress-class=nginx
              - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
              - --validating-webhook=:8443
              - --validating-webhook-certificate=/usr/local/certificates/cert
              - --validating-webhook-key=/usr/local/certificates/key
              - --default-ssl-certificate=ingress-nginx/my-tls-secret
      ...

       

  9. To configure the NGINX Ingress Controller, perform the following steps:
    1. Identify the configmap name by running the following command:

      kubectl get cm -n <ingress_nginx_namespace>
    2. Change the configmap name to use the configmap in your environment by running the following command:

      kubectl edit cm <ingress_nginx_configmap> -n  <ingress_nginx_namespace>

      data:
        enable-underscores-in-headers: "true"
        proxy-body-size: 250m
        server-name-hash-bucket-size: "1024"
        ssl-redirect: "false"
        use-forwarded-headers: "true"
        worker-processes: "40"
        allow-snippet-annotations: "true"
       large-client-header-buffers: "4 64k"

      Important

      The configurations shown above are mandatory. Apart from these, you can retain any other configurations according to your requirement.

  10. To apply the changes made to the daemonset, restart it:

    kubectl -n ingress-nginx rollout restart ds ingress-nginx-controller
  11. To verify the version of the Ingress Controller, run the following command:

    kubectl -n ingress-nginx describe <pod name> | grep -i image

    Example:

    kubectl -n ingress-nginx describe pod ingress-nginx-controller-cb44m | grep -i image

    Sample output:
    24.2_K8s.png
    In the example, 
    the version of the Ingress Controller installed is 1.9.6. 

  12. (Skip this step if you have attached an SSL certificate at the Load Balancer) Update the service ingress-nginx-controller and add load balancer IP as an external IP by using the following command:

    kubectl patch service/ingress-nginx-controller -n ingress-nginx -p '{"spec":{"externalIPs":["xxx.xxx.xxx.xxx"]}}'

    Here, xxx.xxx.xxx.xxx is the IP that the LB_HOST value resolves to.

  13. To get the ingress-nginx-controller service port details, run the following command:

    kubectl -n ingress-nginx get svc

    Example output:

    For example, you will see output like below:
    $ kubectl -n ingress-nginx get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    ingress-nginx-controller LoadBalancer 10.43.43.12 XXX.XXX.XXX.XXX 80:31764/TCP,443:31864/TCP 24h
    ingress-nginx-controller-admission ClusterIP 10.43.46.181 <none> 443/TCP 24h

    Note the port details from the command output. For example:

    • Node port 31764 maps to port 80 and must be used for HTTP connection.
    • Node port 31864 maps to port 443 and must be used for HTTPS connection.
  14. Based on the output of the previous command (step 12), configure the load balancer to send HTTP or HTTPS traffic to the correct ports. 

If you have multiple

Some content is unavailable due to permissions.

namespaces in the same cluster, the Validating Webhook Configuration blocks deployment of a few Ingress objects. To delete the Validating Webhook Configuration, run the following command:

kubectl delete  ValidatingWebhookConfiguration ingress-nginx-admission

To deploy the NGINX Ingress Controller for Kubernetes

  1.  Run the following command and verify if the ingress-nginx namespace is present:
    kubectl get ns

  2. If you have the ingress-nginx namespace, delete it by running the following commands:
    kubectl delete ds -n ingress-nginx ingress-nginx-controller
    kubectl -n ingress-nginx delete svc ingress-nginx-controller-admission
    kubectl delete clusterrole ingress-nginx
    kubectl delete ClusterRoleBinding ingress-nginx
    kubectl delete IngressClass nginx
    kubectl delete ValidatingWebhookConfiguration ingress-nginx-admission
    kubectl delete ns ingress-nginx

  3. Based on the version of your Kubernetes, run one of the following commands to get the deploy.yaml file for the NGINX Ingress Controller:
    To get deploy.yaml file for Nginx Ingress Controller:
    wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-<version>/deploy/static/provider/cloud/deploy.yaml
    Where, version is the Nginx Ingress Controller. Example:
    To get deploy.yaml file for Nginx Ingress Controller version 1.11.2

    wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.2/deploy/static/provider/cloud/deploy.yaml
    To get deploy.yaml file for Nginx Ingress Controller version 1.11.5
    wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.5/deploy/static/provider/cloud/deploy.yaml

  4. Make the following changes in the deploy.yaml file:
    1. Change the kind field of the ingress-nginx-controller from Deployment to DaemonSet.
      Daemonset ensures that the controller machine runs on every worker node.
      The namespace for the NGINX Ingress Controller is 
      ingress-nginx.

      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        labels:
          app.kubernetes.io/component: controller
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/part-of: ingress-nginx
          app.kubernetes.io/version: <version of the deploy.yaml>
        name: ingress-nginx-controller
       namespace: ingress-nginx
    2. (If deploying NGINX Ingress Controller 1.9.3 or later) Under kind: Daemonset, change the spec.strategy field to spec.updateStrategy

      spec:
        minReadySeconds: 0
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 1
          type: RollingUpdate
    3. (If deploying NGINX Ingress Controller 1.9.5 or later) Under kind: Daemonset, locate securityContext, and then set the value of the flag allowPrivilegeEscalation as true.

      securityContext:
                allowPrivilegeEscalation: true
  5. Apply the deploy.yaml file by using the following command:

    kubectl apply -f deploy.yaml
  6. To verify that the pods are running on all worker nodes, run the following command:

     kubectl get ds -n ingress-nginx

    Sample output:

    NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
    ingress-nginx-controller   11        11        11      11           11          kubernetes.io/os=linux   15d

    The values of the columns Desired, Current, Ready, Up-to-date, and Available should be equal to the number of worker nodes.

  7. (Skip this step if you have attached an SSL certificate at the load balancer and are sending HTTP or HTTPS traffic to ingress.)
    Create a secret with the certificate and key to be mounted on the Ingress Controller pods by using the following command:

    kubectl create secret tls my-tls-secret --cert=/path/to/cert.pem --key=/path/to/privkey.pem -n ingress-nginx

    Important

    Make sure cert.pem contains the signed certificate and the complete chain of CA certs.

     

  8. (Skip this step if you have attached an SSL certificate at the load balancer and are sending HTTP or HTTPs traffic to ingress.)
    Edit the daemonset as described:
    1. Add the secret that you created in the args section.
    2. Run the following command:

      kubectl edit daemonset ingress-nginx-controller -n ingress-nginx
    3. In the output, set the ingress-class parameter according to your requirement:

      ...
         spec:
            containers:
            - args:
              - /nginx-ingress-controller
              - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
              - --election-id=ingress-controller-leader
              - --controller-class=k8s.io/ingress-nginx
              - --ingress-class=nginx
              - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
              - --validating-webhook=:8443
              - --validating-webhook-certificate=/usr/local/certificates/cert
              - --validating-webhook-key=/usr/local/certificates/key
              - --default-ssl-certificate=ingress-nginx/my-tls-secret
      ...

       

  9. To configure the NGINX Ingress Controller, perform the following steps:
    1. Identify the configmap name by running the following command:

      kubectl get cm -n <ingress_nginx_namespace>
    2. Change the configmap name to use the configmap in your environment by running the following command:

      kubectl edit cm <ingress_nginx_configmap> -n  <ingress_nginx_namespace>

      data:
        enable-underscores-in-headers: "true"
        proxy-body-size: 250m
        server-name-hash-bucket-size: "1024"
        ssl-redirect: "false"
        use-forwarded-headers: "true"
        worker-processes: "40"
        allow-snippet-annotations: "true"
       large-client-header-buffers: "4 64k"

      Important

      The configurations shown above are mandatory. Apart from these, you can retain any other configurations according to your requirement.

  10. To apply the changes made to the daemonset, restart it:

    kubectl -n ingress-nginx rollout restart ds ingress-nginx-controller
  11. To verify the version of the Ingress Controller, run the following command:

    kubectl -n ingress-nginx describe <pod name> | grep -i image

    Example:

    kubectl -n ingress-nginx describe pod ingress-nginx-controller-cb44m | grep -i image

    Sample output:
    24.2_K8s.png
    In the example, 
    the version of the Ingress Controller installed is 1.9.6. 
     

  12. (Skip this step if you have attached an SSL certificate at the Load Balancer) Update the service ingress-nginx-controller and add load balancer IP as an external IP by using the following command:

    kubectl patch service/ingress-nginx-controller -n ingress-nginx -p '{"spec":{"externalIPs":["xxx.xxx.xxx.xxx"]}}'

    Here, xxx.xxx.xxx.xxx is the IP that the LB_HOST value resolves to.

  13. To get the ingress-nginx-controller service port details, run the following command:

    kubectl -n ingress-nginx get svc

    Example output:

    For example, you will see output like below:
    $ kubectl -n ingress-nginx get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    ingress-nginx-controller LoadBalancer 10.43.43.12 XXX.XXX.XXX.XXX 80:31764/TCP,443:31864/TCP 24h
    ingress-nginx-controller-admission ClusterIP 10.43.46.181 <none> 443/TCP 24h

    Note the port details from the command output. For example:

    • Node port 31764 maps to port 80 and must be used for HTTP connection.
    • Node port 31864 maps to port 443 and must be used for HTTPS connection.
  14. Based on the output of the previous command (step 12), configure the load balancer to send HTTP or HTTPS traffic to the correct ports. 
  15. The Validating Webhook Configuration blocks the deployment of a few Ingress objects. To delete the Validating Webhook Configuration, run the following command:

    kubectl delete  ValidatingWebhookConfiguration ingress-nginx-admission

 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*