Infrastructure as Code and Secrets Management

Lesson 1 Overview

  • In this lab, we will learn how to deploy AWS Infrastructure as Code using the Opensource tool Terraform by Hashicorp. We will also learn to use Vault, another Hashicorp tool, to securely manage secrets and passwords in a highly automated cloud application environment.
  • Cloud Shift Strategies is a proud partner of both [Hashicorp](http://hashicorp.com) and Amazon Web Services.

Prerequisites

  • A basic understanding of AWS services like EC2
  • A basic understanding of the principals of Docker and Continuous Integration servers will be helpful

What to bring

  • Laptop with an ssh client
  • A mouse might be helpful for fast copy/pasting
  • An AWS Free Tier account aws.amazon.com/free (ideally, one account per concurrent lab users and no production resources deployed)

What we’ll build

  • We will build a very simple Flask / Python application, not unlike the applications we created in the Flask Lambda Lab and Flask Docker Lab. This lab has much less focus on the application and much more focus on building the infrastructure and securing application secrets
  • We will use terraform to build a secure and fault tolerant and scalable application infrastructure in AWS EC2. We will use terraform.io to deploy a separate VPC, 6 subnets, Application Load Balancers, Auto Scaling Groups and an Aurora RDS Cluster.
  • We will use vault to store database secrets and a secure way. Vault will provision database accounts & passwords with a short lifespan so that database passwords don’t have to be stored in Version Control or even on the server.

Architecture

While this lab isn’t focused on AWS architecture specifically, we thought it would be a good opportunity to build the infrastructure in a robust and secure manner

Network Architecture Design Elements

  • A dedicated VPC for the application
  • A segmented network
  • Public subnet for Load Balancers with inbound internet access
  • Private subnets for DB with no inbound internet access
  • All subnets have outbound internet access
  • Public subnet internet accessible through Internet Gateway
  • Private subnet get to internet w/ NAT Gateways
  • Using 3 availability Zones for redundancy
  • The “default VPC” has a peering connection w/ our application VPC (for bastion)

Zero Trust Network

  • The bastion host allows ssh in only (should limit inbound IP in real world)
  • The vault host allows 8200 from web and bastion
  • ALB allows 80 from world
  • The web servers allow 80 only from ALB
  • Web servers do not store db passwords or connectivity on disk
  • The database servers allow 3306 from vault, web and bastion only
  • Database credentials are created by vault dynamically with a 1 hour lease

Lesson Outline

  • This lab is organized into 4 lessons

Lesson 2 – Build a Developer workstation

  • We’ll start by launching a small Linux instance in AWS EC2 that we’ll use to deploy our infrastructure. Normally, the developer would use her local PC for this activity, but in this lab downloading & configuring the tools on the Laptop would take more time than we have.
  • our provisioning server will have AWS administrator access to our AWS account so that it has the authority to provision and destroy all the resources defined. As such, this machine should be tightly controlled (or destroyed) when not in use.

Lesson 3 – Infrastructure as Code

  • We’ll use terraform.io to build the AWS infrastructure described in the figure above. Terraform is a powerful open source infrastructure automation tool based on the declarative HCL language.
  • It has many benefits including
  • Simple and Powerful
    • Safely and predictably create, change, and improve infrastructure
    • Collaborate & share configurations stored in version control
    • Evolve your infrastructure and track the history of infrastructure versions
  • Automation Friendly
    • Infrastructure as code increases operator productivity and transparency
    • Human and machine readable elegant user experience for operators
    • Maps resource dependencies because minor change could have potential cascading effects
  • Once Safe Workflow
    • Separates plan from apply – reduces mistakes and uncertainty at scale
    • Supports all major infrastructure providers (AWS, Google, Azure, OpenStack, VMware, and more)

Lesson 4 – Vault Secrets Management

  • We’ll use vaultproject.io to manage our secrets. Our database usernames and passwords will be created dynamically. The Vault server will be configured from the bastion host
  • Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing.
  • Vault can store your existing secrets, or it can dynamically generate new secrets to control access to third-party resources or provide time-limited credentials for your infrastructure.
  • Vault handles leasing, key revocation, key rolling, and auditing. Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service, or generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials, and more.

Lesson 5 – Cleanup

  • Finally, we’ll destroy all the resources that we created, including the developer workstation

Let’s get started with Lesson 2 now!