Provision and deploy the Consul on AWS using Terraform and Docker.

Introduction

In this post, we will deploy a highly available three-node Consul cluster to AWS. We will use Terraform to provision a set of EC2 instances and accompanying infrastructure. The instances will be built from a basic ubuntu 18.04 ami. We will install the docker and deploy the consul and open the respective ports. We will deploy Docker containers to each EC2 host, containing an instance of the Consul server.

Consul – We will achieve high availability (HA) by clustering three Consul server nodes across the three Elastic Cloud Compute (EC2) instances. We use consul for service discovery in ECS which will be explained in the last part.

Pre-requisites

Before we get started installing the Consul stack on AWS.

  • Ensure the latest version of terraform is installed
  • Create the IAM access in AWS to provision the ec2 instance,vpc, subnet, internet gateway, security group, iam.

Installation and configuration

Clone the project locally to your host.

$ git clone https://github.com/ahamedyaserarafath/terraform_consul.git

The AWS will provision and those are added as a part of variables, if you wish to change please feel free to change in variable.tf alone.

In this project, we used the following provision.

  • EC2 AMI – ami-0dad20bd1b9c8c004
  • EC2 Instance Type – t2.micro
  • Region – Singapore
  • VPC – 11.0.0.0/16
  • Subnet – 11.0.1.0/24,11.0.2.0/24,11.0.0.0/24
  • Port Opened – 8300,8301,8302,8500,8600,22

Steps to run the provisioning in Terraform

  • Clone the repo
$ git clone https://github.com/ahamedyaserarafath/terraform_consul.git
  • Terraform initialize a working directory
$ terraform init
  • Terraform to create an execution plan
$ terraform plan
  • Terraform apply to provision in aws
$ terraform apply

Note: The above command will provision the ec2 instance and install the Consul

Install Consul registry in ECS

Do create an ELB for those three consul servers with 8500

  • Edit the ecs_registrator.sh and add the respective server IP or ELB, make sure ECS can communicate with the same.
CONSUL_IP=54.254.252.6 
to 
CONSUL_IP=<ELB/IP>
  1. Run the below command where dockerfile exists(Here gliderlabs/registration is used)
$ docker build -t ecs_consul_registrator .
  • Push the ecs_consul_registrator to ECR, please follow the AWS Documents to push the docker image.
  • Create a task with ecs_consul_registrator Container and mount the below file for service discovery of the ECS cluster.
...
Container Path    	Source Volume 	Read only
/tmp/docker.sock	  sourcedocker
...
Name - sourcedocker
Source Path -/var/run/docker.sock
...

JSON parser for your reference

       ...
       "mountPoints": [
        {
          "readOnly": null,
          "containerPath": "/tmp/docker.sock",
          "sourceVolume": "sourcedocker"
        }
        ...
        "volumes": [
        {
          "name": "sourcedocker",
          "host": {
            "sourcePath": "/var/run/docker.sock"
        }
        ...
https://github.com/ahamedyaserarafath/terraform_consul

GitHub Link, Please feel free to fork and use it

Categorized in:

DevOps

Tagged in:

, , ,