Deploying and configuring the ingress controller for OpenShift or Kubernetes

The ingress controller is a load balancer for Kubernetes environments. Before deploying BMC Helix IT Operations Management, you must deploy and configure the ingress controller.

The certified versions of NGINX Ingress Controller with the Kubernetes and OpenShift orchestration platform are as follows:

Nginx Ingress Controller version
Supported Kubernetes version
OpenShift version
1.7.0
1.24 or 1.25
4.11 or 4.12
1.8.1
1.26
4.13
1.9.3 
1.27
4.14
1.9.5
1.28
4.14
1.9.6
1.29
NA

Related topics

Preparing-for-deployment

System-requirements

To deploy the ingress controller for Openshift

  1. Download the attached ingress-scc.yaml file.

  1. Apply the ingress-scc.yaml file by using the following command:

    oc apply -f ingress-scc_<version>.yaml

    Example:

    oc apply -f ingress-scc_24.1.yaml
  2. Based on the version of your OpenShift, run one of the following commands to get the deploy.yaml file for the NGINX Ingress Controller:

    • To get deploy.yaml file for Nginx Ingress Controller version 1.7.0:
      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml
    • To get deploy.yaml file for Nginx Ingress Controller version 1.8.1:
      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.1/deploy/static/provider/cloud/deploy.yaml
    • To get deploy.yaml file for Nginx Ingress Controller version 1.9.3:
      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.3/deploy/static/provider/cloud/deploy.yaml
      ImportantIf you are not able to download Nginx Ingress Controller version 1.9.3, use version 1.9.5 or 1.9.6.
    • To get deploy.yaml file for Nginx Ingress Controller version 1.9.5:
      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.5/deploy/static/provider/cloud/deploy.yaml
    • To get deploy.yaml file for Nginx Ingress Controller version 1.9.6:
      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.6/deploy/static/provider/cloud/deploy.yaml

    If you change the ingress configuration and decide not to use the attached deploy.yaml file, make sure that the INGRESS_CLASS value in the configs/infra.config file matches the class in your ingress definition.
  3. Make the following changes in the deploy.yaml file:
    1. Change the kind property of the ingress-nginx-controller from Deployment to DaemonSet.
      Daemonset ensures that the controller machine runs on every worker node. It improves the reliability and efficiency.
      The namespace for the ingress controller is       ingress-nginx.

    1. (If deploying NGINX Ingress Controller 1.9.3 or later) Under kind: Daemonset, change the spec.strategy field to spec.updateStrategy:

      spec:
        minReadySeconds: 0
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 1
          type: RollingUpdate

    2. (If deploying NGINX Ingress Controller 1.9.5 or later) Under kind: Daemonset, locate securityContext, and then set the value of the flag allowPrivilegeEscalation as true.

      securityContext:
                allowPrivilegeEscalation: true

  2. Apply the deploy.yaml file by using the following command:

    oc apply -f deploy.yaml

  3. To verify that the pods are running on all worker nodes, run the following command:

    oc get ds -n ingress-nginx

    Sample output:

    NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
    ingress-nginx-controller   11        11        11      11           11          kubernetes.io/os=linux   15d

    Columns Desired, Current, Ready, Up-to-date, and Available should have a value equal to the number of worker nodes, except for tainted nodes such as Master.

  4. Apply the Security Context Constraints (SCC) to service accounts by running the following commands:

    1. If the version of your OpenShift cluster is lower than 4.14:

      oc adm policy add-scc-to-user ingress-scc -z default -n ingress-nginx
      oc adm policy add-scc-to-user ingress-scc -z ingress-nginx-admission -n ingress-nginx
      oc adm policy add-scc-to-user ingress-scc -z ingress-nginx -n ingress-nginx

    2. If the version of your OpenShift cluster is 4.14 or higher:

      oc -n ingress-nginx annotate job.batch/ingress-nginx-admission-patch openshift.io/required-scc="ingress-scc"
      oc -n ingress-nginx annotate job.batch/ingress-nginx-admission-create openshift.io/required-scc="ingress-scc"
      oc -n ingress-nginx annotate daemonset.apps/ingress-nginx-controller openshift.io/required-scc="ingress-scc"

  5. (Skip this step if you have attached an SSL certificate at the Load Balancer)
    Create a secret from the trusted certificate and key. Depending on your cluster, run the following command:

    Important

    Ensure that the cert.pem file contains the full chain.

    oc create secret tls my-tls-secret --cert=/path/to/cert.pem --key=/path/to/privkey.pem -n ingress-nginx

  6. (Skip this step if you have attached an SSL certificate at the Load Balancer)
    In the ingress-controller, under daemonset, edit the args section to set the default certificate to my-tls-secret:

    oc edit daemonset ingress-nginx-controller -n ingress-nginx
    ...
       spec:
          containers:
          - args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
            - --default-ssl-certificate=ingress-nginx/my-tls-secret
    ...


  7. Configure the ingress controller. Perform the following steps:

    1. Identify the configmap name by running the following command:

      oc get cm -n <ingress_nginx_namespace>

    2. Change the configmap name to use the configmap in your environment by running the following command:

      oc edit cm <ingress_nginx_configmap> -n  <ingress_nginx_namespace>

      data:
        enable-underscores-in-headers: "true"
        proxy-body-size: 250m
        server-name-hash-bucket-size: "1024"
        ssl-redirect: "false"
        use-forwarded-headers: "true"
       worker-processes: "40"
       allow-snippet-annotations: "true"

      Important

      The configurations shown above are mandatory. Apart from these, you can retain any other configurations according to your requirements.

  8. Verify the version of the ingress controller from one of the pods' logs by using the following command:

    oc logs ingress-nginx-controller-XXXXX | less

  9. (Skip this step if you have attached an SSL certificate at the Load Balancer) Update the service ingress-nginx-controller and add load balancer IP as an external IP by using the following command:

    oc patch service/ingress-nginx-controller -n ingress-nginx -p '{"spec":{"externalIPs":["xxx.xxx.xxx.xxx"]}}'

    Here, xxx.xxx.xxx.xxx is the IP that the LB_HOST value resolves to.

  10. (Skip this step if you have attached an SSL certificate at the Load Balancer) Update the load balancer settings to point to the correct ports of ingress-nginx-controller service. Check the ingress-nginx-controller service ports by using the following command:

    oc -n ingress-nginx get svc

    Example output:

    For example, you will see output like below:
    $ oc -n ingress-nginx get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    ingress-nginx-controller LoadBalancer 10.43.43.12 XXX.XXX.XXX.XXX 80:31764/TCP,443:31864/TCP 24h
    ingress-nginx-controller-admission ClusterIP 10.43.46.181 <none> 443/TCP 24h

  11. The Validating Webhook Configuration blocks the deployment of a few Ingress objects. To delete the Validating Webhook Configuration, run the following command:

    oc -n ingress-nginx delete  ValidatingWebhookConfiguration ingress-nginx-admission



To deploy the ingress controller for Kubernetes

  1. Run the following command and check if the ingress-nginx namespace is present:

    kubectl get ns

  2. If you have the ingress-nginx namespace, delete it by running the following commands:

    kubectl delete ds -n ingress-nginx nginx-ingress-controller
    kubectl delete svc ingress-nginx-controller-admission -n ingress-nginx
    kubectl delete clusterrole ingress-nginx
    kubectl delete ClusterRoleBinding ingress-nginx
    kubectl delete IngressClass nginx
    kubectl delete ValidatingWebhookConfiguration ingress-nginx-admission
    kubectl delete ns ingress-nginx

  3. (Only if you are on Kubernetes version 1.24 or earlier) Download the psp.yaml file from github by using the following command:

    wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/docs/examples/psp/psp.yaml

  4. (Only if you are on Kubernetes version 1.24 or earlier) If you have the restricted-psp property enabled by default, apply the psp.yaml file by using the following command:

    kubectl apply -f psp.yaml

    Important

    To check if the pod security policy is enabled, run the following command:
    kubectl get psp

    You must find a PSP with the name ingress-nginx.

  5. Based on the version of your Kubernetes, run one of the following commands to get the deploy.yaml file for the NGINX Ingress Controller:

    • To get deploy.yaml file for Nginx Ingress Controller version 1.7.0:

      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml

    • To get deploy.yaml file for Nginx Ingress Controller version 1.8.1:

      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.1/deploy/static/provider/cloud/deploy.yaml

    • To get deploy.yaml file for Nginx Ingress Controller version 1.9.3:

      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.3/deploy/static/provider/cloud/deploy.yaml

      Important

      If you are not able to download Nginx Ingress Controller version 1.9.3, use version 1.9.5 or 1.9.6.

    • To get deploy.yaml file for Nginx Ingress Controller version 1.9.5:

      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.5/deploy/static/provider/cloud/deploy.yaml

    • To get deploy.yaml file for Nginx Ingress Controller version 1.9.6:

      wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.6/deploy/static/provider/cloud/deploy.yaml
  6. Make the following changes in the deploy.yaml file:
    1. Change the kind property of the ingress-nginx-controller from Deployment to DaemonSet.
      Daemonset ensures that the controller machine runs on every worker node. It improves the reliability and efficiency.
      The namespace for the ingress controller is       ingress-nginx.

    1. (If deploying NGINX Ingress Controller 1.9.3 or later) Under kind: Daemonset, change the spec.strategy field to spec.updateStrategy:

      spec:
        minReadySeconds: 0
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 1
          type: RollingUpdate

    2. (If deploying NGINX Ingress Controller 1.9.5 or later) Under kind: Daemonset, locate securityContext, and then set the value of the flag allowPrivilegeEscalation as true.

      securityContext:
                allowPrivilegeEscalation: true

  2. Apply the deploy.yaml file by using the following command:

    kubectl apply -f deploy.yaml

  3. To verify that the pods are running on all worker nodes, run the following command:

     kubectl get ds -n ingress-nginx

    Sample output:

    NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
    ingress-nginx-controller   11        11        11      11           11          kubernetes.io/os=linux   15d

    Columns Desired, Current, Ready, Up-to-date, and Available should have a value equal to the number of worker nodes, except for tainted nodes such as Master.

  4. (Skip this step if you have attached an SSL certificate at the Load Balancer)
    Create a secret with the certificate and key to be mounted on the ingress controller pods by using the following command:

    kubectl create secret tls my-tls-secret --cert=/path/to/cert.pem --key=/path/to/privkey.pem -n ingress-nginx

    Important

    Ensure that the cert.pem file has the full chain in it.

  5. (Skip this step if you have attached an SSL certificate at the Load Balancer)
    Edit the daemonset as described below:
    1. Add the secret that you created in the args section.

    2. Run the following command:

      kubectl edit daemonset ingress-nginx-controller -n ingress-nginx

    3. In the output, set the ingress-class parameter according to your requirement:

      ...
         spec:
            containers:
            - args:
              - /nginx-ingress-controller
              - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
              - --election-id=ingress-controller-leader
              - --controller-class=k8s.io/ingress-nginx
              - --ingress-class=nginx
              - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
              - --validating-webhook=:8443
              - --validating-webhook-certificate=/usr/local/certificates/cert
              - --validating-webhook-key=/usr/local/certificates/key
              - --default-ssl-certificate=ingress-nginx/my-tls-secret
      ...

  6. Configure the ingress controller. Perform the following steps:

    1. Identify the configmap name by running the following command:

      kubectl get cm -n <ingress_nginx_namespace>

    2. Change the configmap name to use the configmap in your environment by running the following command:

      kubectl edit cm <ingress_nginx_configmap> -n  <ingress_nginx_namespace>

      data:
        enable-underscores-in-headers: "true"
        proxy-body-size: 250m
        server-name-hash-bucket-size: "1024"
        ssl-redirect: "false"
        use-forwarded-headers: "true"
       worker-processes: "40"
       allow-snippet-annotations: "true"

      Important

      The configurations shown above are mandatory. Apart from these, you can retain any other configurations according to your requirement.

  7. (Skip this step if you have attached an SSL certificate at the Load Balancer) Update the service ingress-nginx-controller and add load balancer IP as an external IP by using the following command:

    kubectl patch service/ingress-nginx-controller -n ingress-nginx -p '{"spec":{"externalIPs":["xxx.xx.xx.xxx"]}}'

    Here, xxx.xxx.xxx.xxx is the IP that the LB_HOST value resolves to.

  8. (Skip this step if you have attached an SSL certificate at the Load Balancer) Update the load balancer settings to point to the correct ports of ingress-nginx-controller service. Check the ingress-nginx-controller service ports by using the following command:

    kubectl -n ingress-nginx get svc

    Example output:

    For example, you will see output like below:
    $ kubectl -n ingress-nginx get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    ingress-nginx-controller LoadBalancer 10.43.43.12 XXX.XXX.XXX.XXX 80:31764/TCP,443:31864/TCP 24h
    ingress-nginx-controller-admission ClusterIP 10.43.46.181 <none> 443/TCP 24h

  9. The Validating Webhook Configuration blocks the deployment of a few Ingress objects. To delete the Validating Webhook Configuration, run the following command:

    kubectl -n ingress-nginx delete  ValidatingWebhookConfiguration ingress-nginx-admission




 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*