Deploying the Nginx Ingress in On-prem Kubernetes for bare-metal and EC2 instance(Standalone) with Letz Encrypt(Auto-Renew certificate).

Introduction

Kubernetes Ingresses allow you to flexibly route traffic from outside your Kubernetes cluster to Services inside of your cluster. This is accomplished using Ingress Resources, which define rules for routing HTTP and HTTPS traffic to Kubernetes Services, and Ingress Controllers, which implement the rules by load balancing traffic and routing it to the appropriate backend Services.

In this guide, we’ll set up the Kubernetes-maintained Nginx Ingress Controller(hostNetwork configuration), and create some Ingress Resources to route traffic to several dummy backend services. Once we’ve set up the Ingress, we’ll install a cert-manager into our cluster to manage and provision TLS certificates for encrypting HTTP traffic to the Ingress.

This will make sure zero cost for your certificate and less maintenance

Pre-requisites

Before we get started installing the Ingress controller

* Ensure you have Kubernetes 1.15+ cluster on bare-metal or in ec2 instance.

* The wget command-line utility installed on your local machine. You can install wget using the package manager built into your operating system.

Installation and configuration

Clone the project locally to your Linux machine.

git clone git@github.com:ahamedyaserarafath/zerocost-kube-https.git

Step 1 Setting Up Dummy Backend Services(Change it according to your service)

We’ll first create and roll out dummy echo Services to which we’ll route external traffic using the Ingress.

kubectl apply -f http_ingress.yaml
kubectl get svc echo1

Verify that the Service started correctly by confirming that it has a ClusterIP, the internal IP on which the Service is exposed.

Output
service/echo1 created
deployment.apps/echo1 created

Now that our dummy echo web services are up and running, we can move on to rolling out the Nginx Ingress Controller.

Step 2 – Setting Up the Kubernetes Nginx Ingress Controller with HostNetwork

In this step, we’ll roll out v0.43.0 of the Kubernetes-maintained Nginx Ingress Controller. Note that there are several Nginx Ingress Controllers; the Kubernetes community maintains the one used in this guide and Nginx Inc. maintains Kubernetes-ingress. The instructions in this tutorial are based on those from the official Kubernetes Nginx Ingress Controller Installation Guide.

In this ingress controller, we added host network is enabled by default

kubectl apply -f ingress_controler.yaml

We use apply here so that in the future we can incrementally apply changes to the Ingress Controller objects instead of completely overwriting them. To learn more about applying, consult Managing Resources from the official Kubernetes docs.

You should see the following output:

Output
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created

Confirm that the Ingress Controller Pods have started:

kubectl get pods -n ingress-nginx \
  -l app.kubernetes.io/name=ingress-nginx --watchOutput
NAME                                       READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-xtbbt       0/1     Completed   0          32s
ingress-nginx-admission-patch-wb8mk        0/1     Completed   1          32s
ingress-nginx-controller-9b98b6577-xrq4k   1/1     Running     0          40s

Hit CTRL+C to return to your prompt.

Now, confirm that the NodePort was successfully created by fetching the Service details with kubectl:

kubectl get svc --namespace=ingress-nginxOutput
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.109.108.65   <none>        80:30228/TCP,443:30028/TCP   65s
ingress-nginx-controller-admission   ClusterIP   10.103.80.152   <none>        443/TCP                      65s

Step 3 – Setting Up the Kubernetes Nginx Ingress Resource(HTTP)

Let’s begin by creating a minimal Ingress Resource to route traffic directed at a given subdomain to a corresponding backend Service.

Here, we’ve specified that we’d like to create an Ingress Resource called sample-ingress, and route traffic based on the Host header. An HTTP request Host header specifies the domain name of the target server.

Sample for your reference

  rules:
    - host: app.sample.com
      http:
        paths:

to

  rules:
    - host: <your_domain>
      http:
        paths:

You can now create the Ingress using kubectl:

kubectl apply -f http_ingress.yaml

You’ll see the following output confirming the Ingress creation:

Output
ingress.networking.k8s.io/sample-ingress created

Note: Wait for few mins till the external IP is assigned to ingress, you can check that by the below commands

kubectl get ingress

You’ll see the following output confirming the Ingress created with Nodeport IP:

NAME             CLASS    HOSTS            ADDRESS       PORTS   AGE
sample-ingress   <none>   app.sample.com   192.168.0.7   80      5s

To test the Ingress, navigate to your DNS management service and create A records for app.sample.com(<your_domain>) pointing to the Node IP(In case of EC2 instance open the 80 Port)

Once you’ve created the necessary app.sample.com(<your_domain>) DNS records, you can test the Ingress Controller and Resource you’ve created using the curl command-line utility.

From your local machine, curl the echo1 Service:

curl http://app.sample.com

You should get the following response from the echo1 service:

Output
echo1

From your local machine, curl the echo1 Service if the domain is not mapped:

curl -kv http://localhost -H 'Host: app.sample.com'

You should get the following response from the echo1 service:

Output
* Rebuilt URL to: localhost/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: app.sample.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sat, 30 Jan 2021 18:13:51 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 6
< Connection: keep-alive
< X-App-Name: http-echo
< X-App-Version: 0.2.3
<
echo1

Step 4 — Installing and Configuring Cert-Manager

In this step, we’ll install v0.16.1 of cert-manager into our cluster. cert-manager is a Kubernetes add-on that provisions TLS certificates from Let’s Encrypt and other certificate authorities (CAs) and manages their lifecycles. Certificates can be automatically requested and configured by annotating Ingress Resources, appending a TLS section to the Ingress spec, and configuring one or more Issuers or ClusterIssuers to specify your preferred certificate authority. To learn more about Issuer and ClusterIssuer objects, consult the official cert-manager documentation on Issuers.

Install cert-manager and its Custom Resource Definitions (CRDs) like Issuers and ClusterIssuers by following the official installation instructions. Note that a namespace called cert-manager will be created into which the cert-manager objects will be created:

kubectl apply --validate=false -f cert-manager.yaml
or 
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.16.1/cert-manager.yaml

You should see the following output:

Output
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
. . .
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created

To verify our installation, check the cert-manager Namespace for running pods:

kubectl get pods --namespace cert-managerOutput

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-271ad6d964-hb512              1/1     Running   0          20s
cert-manager-cainjector-1eeee9dd7c-f46gf   1/1     Running   0          90s
cert-manager-webhook-589b9w9ddd-wf9l6      1/1     Running   0          98s

This indicates that the cert-manager installation succeeded.

Before we begin issuing certificates for our echo1.example.com and echo2.example.com domains, we need to create an Issuer, which specifies the certificate authority from which signed x509 certificates can be obtained. In this guide, we’ll use the Let’s Encrypt certificate authority, which provides free TLS certificates and offers both a staging server for testing your certificate configuration and a production server for rolling out verifiable TLS certificates.

We then specify an email address to register the certificate and create a Kubernetes Secret called letsencrypt-prod to store the ACME account’s private key. We also use the HTTP-01 challenge mechanism. To learn more about these parameters, consult the official cert-manager documentation on Issuers.

Note: If you reached these steps make sure your domain is mapped to NodePort otherwise certification will not be issued by letsencrypt and make sure 80 and 443 port is open.

Open a file called prod_issuer.yaml in your favorite editor and edit the email id:

  ...
    email: <your_email_address_here>
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
  ...

Roll out this Issuer using kubectl:

kubectl create -f prod_issuer.yaml

You should see the following output:

Output
clusterissuer.cert-manager.io/letsencrypt-prod created

Now that we’ve created our Let’s Encrypt staging and prod ClusterIssuers, we’re ready to modify the Ingress Resource we created above and enable TLS encryption for the echo1.example.com(<your_domain>) paths.

Step 5 — Issuing Production Let’s Encrypt Certificates

To issue a staging TLS certificate for our domains, we’ll annotate https_ingress_letzencrypt.yaml with the ClusterIssuer created in Step 4. This will use ingress-shim to automatically create and issue certificates for the domains specified in the Ingress manifest.

Open up echo_ingress.yaml in your favorite editor:

...
  - hosts:
    - app.sample.com
    secretName: app-secret
  rules:
    - host: app.sample.com
      http:
...

to

...
  - hosts:
    - <your_domain>
    secretName: app-secret
  rules:
    - host: <your_domain>
      http:
...

We also add a tls block to specify the hosts for which we want to acquire certificates and specify a secretName. This secret will contain the TLS private key and issued certificate. Be sure to swap out app.sample.com with the domain for which you’ve created DNS records.

When you’re done making changes, save and close the file.

We’ll now push this update to the existing Ingress object using kubectl apply:

kubectl apply -f echo_ingress.yaml

You should see the following output:

Output
ingress.networking.k8s.io/sample-ingress configured

You can use kubectl describe to track the state of the Ingress changes you’ve just applied:

kubectl describe ingress

Once the certificate has been successfully created, you can run a describe on it to further confirm its successful creation:

kubectl describe certificate

Once you see the following output, the certificate has been issued successfully:

Output
  Normal  Issuing    28s                 cert-manager  Issuing certificate as Secret was previously issued by ClusterIssuer.cert-manager.io/letsencrypt-prod
  Normal  Reused     28s                 cert-manager  Reusing private key stored in existing Secret resource "app-secret"
  Normal  Requested  28s                 cert-manager  Created new CertificateRequest resource "app-secret-90agn"
  Normal  Issuing    2s (x2 over 4m52s)  cert-manager  The certificate has been successfully issued

We’ll now perform a test using curl to verify that HTTPS is working correctly:

curl http://app.sample.com
or 
curl -kv http://localhost -H 'Host: app.sample.com'

You should see the following:

Output
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.15.9</center>
</body>
</html>

This indicates that HTTP requests are being redirected to use HTTPS.

Run curl on HTTPS:

curl https://app.sample.com
or
curl -kv https://localhost -H 'Host: app.sample.com'

You should now see the following output:

Output
echo1

You can run the previous command with the verbose -v flag to dig deeper into the certificate handshake and to verify the certificate information.

At this point, you’ve successfully configured HTTPS using a Let’s Encrypt certificate for your Nginx Ingress.

Conclusion

In this guide, you set up an Nginx Ingress to load balance and route external requests to backend Services inside of your On-prem/EC2 Kubernetes cluster.

You also secured the Ingress by installing the cert-manager certificate provisioner and setting up a Let’s Encrypt certificate

There are many alternatives to the Nginx Ingress Controller. To learn more, consult Ingress controllers from the official Kubernetes documentation.

https://github.com/ahamedyaserarafath/zerocost-kube-https

GitHub Link, Please feel free to fork and use it

Categorized in:

DevOps

Tagged in:

,