Cloud agnostic approach with Terraform

If you’ve been working with both AWS and Azure you should have noticed that each of them has some advantages.

Tools like Terraform might be very helpful if you’re not familiar with both CloudFormation and Azure RM.

However don’t consider Terraform as a nonpareil (this is not true at all), it is simple tool for simple tasks.

So in this  post I’ll tell you about Terraform terms and concepts and show example with AWS & Azure.

For a bit more complicated infrastructure you’ll have to use CloudFormation and Azure RM

In very general, Terraform is orchestrator which allows you not to use APIs if you don’t want.

But what if you already have infrastructure to manage? Well, Terraform has ability to import resources, but it’s really poor at the time, so we’ll start from scratch in this example.

As an orchestrator Terraform supports lots of providers, f.e. AWS, Azure, GitHub and even MySQL.

If you’re using cloud provider you might want to create the most popular resource – virtual machines. As soon as VM is created you want it to do some tasks in your infrastructure, so you have to install and configure some software on it.

To achieve it we’ll use provisioners  to run DSC tool, Ansible in my example.

And if you want to get rid of provisioners and providers you can use modules to get what you need very fast or you can even create your own module and distribute it over Terraform registry, GitHub or AWS S3.

Whatever way you’ll chose you will deal with state concept. To tell the long story short (as you have notices, Terraform has very good documentation), state is used to map your configuration entities (like “db_svr”) to real entities (like “i-07a4dbf12c50dae76”) and store metadata f.e. dependencies and time stamps.

If you’re working alone or if your team executes terraform from single machine (like Jenkins) storing states locally will be fine, but if you’re working with team you must consider centralised storage like Terraform Enterprise or AWS S3.

Terraform syntax is very easy when you’re using Terraform format and a bit more complicated using JSON.

Before start coding update your global gitignore with this records –

.terraform/ – in this dir plugins are stored, no need to put binaries to VCS
*.tfstate* – states putted to VCS can produce conflicts, so use Remote state
*.tfvars – this files will keep your secrets like API keys, passwords, connection strings, etc. and VCS is not a choice for that

Well, now it’s time to have a look at some real examples and we’ll start with AWS because they provide default network and you don’t have to create storage accounts.

Because we don’t want to maintain huge configs let’s create config for variables:

variable "access_key" {}
variable "secret_key" {}

variable "region" {
  default = "us-east-1"
}

variable "amis" {
  type = "map"
  default = {
    "us-east-1" = "ami-cd0f5cb6"
    "us-west-2" = "ami-6e1a0117"
  }
}

We can have multiple providers, so let’s create dedicated config to describe them:

provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region     = "${var.region}"
}

Finally let’s create our main config which will create our instance:

resource "aws_instance" "terraform_instance" {
  ami = "${lookup(var.amis, var.region)}"
  instance_type = "t2.micro"  
  key_name = "aws-lab"

  connection {
    type = "ssh"
    user = "ubuntu"
    private_key = "${file("/Users/kagarlickij/.ssh/aws-lab.pem")}"
  }

    provisioner "remote-exec" {
    inline = [
      "sudo apt-get update -y",
      "sudo apt-get install python -y",
    ]
  }

    provisioner "local-exec" {
        command = "sleep 120; ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u ubuntu --private-key /Users/kagarlickij/.ssh/aws-lab.pem -i '${aws_instance.terraform_instance.public_ip},' install_nginx.yml"
    }

}

output "public_ip" {
  value = "${aws_eip.terraform_eip.public_ip}"
}

As you can see, we’ll use Ansible to install required software, here’s super simple playbook for nginx installation:

---
- hosts: all
  become: true

  tasks:
    - name: Installs nginx web server
      apt: pkg=nginx state=installed update_cache=true
      notify:
        - start nginx

  handlers:
    - name: start nginx
      service: name=nginx state=started

For our learning we have to create one more resource (Elastic IP) and attach it to instance created above:

resource "aws_eip" "terraform_eip" {
  instance = "${aws_instance.terraform_instance.id}"
}

 

Ok, let’s install necessary stuff with terraform init command:

Before we start applying it, let’s check what will be done using terraform plan command:

Looks good, so execute terraform apply:

Does it looks better than CloudFormation or Azure RM output? As for me yes, much better =)

And you can destroy resources in the same simple and obvious way via terraform destroy (but don’t forget to type “yes” manually):

Now let’s have a look at Azure which is a bit more hard. First of all you have to get auth stuff (for AWS we used API keys).

Follow Terraform instruction or this script to get subscription_id , client_id , client_secret , tenant_id

Start with defining variables:

variable "subscription_id" {}
variable "client_id" {}
variable "client_secret" {}
variable "tenant_id" {}

variable "location" {
    default = "East US"
}

variable "resource_group_name" {
    default = "terraform"
}

variable "vnet_cidr" {
    default = "10.1.0.0/16"
}

variable "subnet1_cidr" {
    default = "10.1.0.0/24"
}

variable "vm_username" {
    default = "terraform"
}

variable "vm_size_web" {
    default = "Standard_DS1_v2"
}

Define provider:

provider "azurerm" {
  subscription_id = "${var.subscription_id}"
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
  tenant_id = "${var.tenant_id}"
}

Create resource group:

resource "azurerm_resource_group" "terraform_rg" {
  name = "${var.resource_group_name}"
  location = "${var.location}"
}

Create VNet and subnets:

resource "azurerm_virtual_network" "terraform_vnet" {
  name = "terraform_vnet"
  address_space = ["${var.vnet_cidr}"]
  location = "${var.location}"
  resource_group_name   = "${azurerm_resource_group.terraform_rg.name}"
}

resource "azurerm_subnet" "terraform_subnet_1" {
  name = "terraform_subnet_1"
  address_prefix = "${var.subnet1_cidr}"
  virtual_network_name = "${azurerm_virtual_network.terraform_vnet.name}"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"
}

For this example let’s create Network Security Group (we’ve skipped Security Groups for AWS because they’re too simple):

resource "azurerm_network_security_group" "terraform_network_security_group_web" {
  name = "terraform_network_security_group_web"
  location = "${var.location}"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"

  security_rule {
    name = "SSH"
    priority = 1000
    direction = "Inbound"
    access         = "Allow"
    protocol = "Tcp"
    source_port_range       = "*"
      destination_port_range     = "22"
      source_address_prefix      = "*"
      destination_address_prefix = "*"
  }

  security_rule {
    name = "AllowHTTP"
    priority= 2000
    direction= "Inbound"
    access = "Allow"
    protocol = "Tcp"
    source_port_range       = "*"
      destination_port_range     = "80"
      source_address_prefix      = "Internet"
      destination_address_prefix = "*"
  }
}

Now create Network Interface:

resource "azurerm_network_interface" "terraform_network_interface_web" {
  name = "terraform_network_interface_web"
  location = "${var.location}"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"
  network_security_group_id = "${azurerm_network_security_group.terraform_network_security_group_web.id}"

  ip_configuration {
    name = "terraform_network_interface_ip_configuration_web"
    subnet_id = "${azurerm_subnet.terraform_subnet_1.id}"
    private_ip_address_allocation = "dynamic"
    public_ip_address_id= "${azurerm_public_ip.terraform_public_ip_web.id}"
  }
}

Finally create Public IP:

resource "azurerm_public_ip" "terraform_public_ip_web" {
  name = "terraform_public_ip_web"
  location = "${var.location}"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"
  public_ip_address_allocation = "static"
  domain_name_label = "terraform-public-ip-web"
}

We’re done with network part, and I don’t want to use Managed disks in this demo, so let’s create Storage account and container:

resource "random_id" "storage_account" {
  byte_length = 8
}

resource "azurerm_storage_account" "terraform_storage" {
  name = "tfsta${lower(random_id.storage_account.hex)}"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"
  location = "${var.location}"
  account_type = "Standard_LRS"
}

resource "azurerm_storage_container" "terraform_storage_container" {
  name = "terraformstoragecontainer"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"
  storage_account_name = "${azurerm_storage_account.terraform_storage.name}"
  container_access_type = "private"
}

Looks like everything is done and it means that we can create VM (pay extra attention on SSH for Linux VM, it’s not as obvious as when using AWS):

resource "azurerm_virtual_machine" "virtual_machine_web" {
  name = "virtual_machine_web"
  location = "${var.location}"
  resource_group_name = "${azurerm_resource_group.terraform_rg.name}"
  network_interface_ids = ["${azurerm_network_interface.terraform_network_interface_web.id}"]
  vm_size = "${var.vm_size_web}"

  delete_os_disk_on_termination = true

  storage_image_reference {
    publisher = "Canonical"
    offer = "UbuntuServer"
    sku = "16.04-LTS"
    version = "latest"
  }

  storage_os_disk {
    name = "osdisk-1"
    vhd_uri = "${azurerm_storage_account.terraform_storage.primary_blob_endpoint}${azurerm_storage_container.terraform_storage_container.name}/osdisk-1.vhd"
    caching = "ReadWrite"
    create_option = "FromImage"
  }

  os_profile {
    computer_name = "web"
    admin_username = "${var.vm_username}"
  }

  os_profile_linux_config {
    disable_password_authentication = true
    ssh_keys = [{
      path     = "/home/${var.vm_username}/.ssh/authorized_keys"
      key_data = "${file("/Users/kagarlickij/.ssh/azure_ssh.pub")}"
    }]
  }

  connection {
    type = "ssh"
    user = "${var.vm_username}"
    private_key = "${file("/Users/kagarlickij/.ssh/azure_ssh")}"
    agent = true
    timeout = "1m"
    host = "${azurerm_public_ip.terraform_public_ip_web.fqdn}"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update -y",
      "sudo apt-get install python -y",
    ]
  }

  provisioner "local-exec" {
    command = "sleep 120; ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u '${var.vm_username}' --private-key /Users/kagarlickij/.ssh/azure_ssh -i '${azurerm_public_ip.terraform_public_ip_web.fqdn},' install_nginx.yml"
  }
}

output "ip_address" {
  value = "${azurerm_public_ip.terraform_public_ip_web.ip_address}"
}

Ansible playbook is the same:

---
- hosts: all
  become: true

  tasks:
    - name: Installs nginx web server
      apt: pkg=nginx state=installed update_cache=true
      notify:
        - start nginx

  handlers:
    - name: start nginx
      service: name=nginx state=started

Init, Plan and Apply steps are the same so let’s have a look at results:

Now you’ve seen power of Terraform and it’s time to clone or fork my public repos for AWS and Azure and start using it in you lab or dev env!

I hope this info will be useful for you, and if you need any help feel free to use contact form on the main page.

.