Course Review: Managing IaC With Terraform

As you probably have seen from previous posts, Terraform as a technology has peaked my interest.  Luckily for me, safaribooks put on a 3-hour course on this very topic.  The course was presented by Yevgeniy Brikman and it was very well done!

What I liked?

The presenter was absolutely awesome.  Very knowledgeable and what I liked was the presentation went quickly from the basics to some of the more advanced terraform concepts.  He talked about some issues that I ran up against when I was using terraform.  For example, he touched on how he does credential management from both local servers and build servers.  He also talked about how he organizes his tf files and how he integrates modules into his workflow.  Finally, I really liked the project he showed at the end for using go to "test" your terraform scripts in an automated fashion.

What I didn't like?

The O'Reilly platform requires flash to get the audio.  Wow, #howisthisstillathing ?

I am really liking terraform and it's ability to deploy resources cross-platform.  I am hoping to experiment more with it to see how I can bring this technology to my clients.  If you are interested in the course, I think there is another one coming up in a few weeks.  Check out https://www.safaribooksonline.com/live-training/courses/managing-infrastructure-as-code-with-terraform/0636920103172/ for more info.

Building an Azure SQL Server with Terraform

I want to spend some more time learning Terraform, and I am hoping to build out a script to deploy the infrastructure for a pet project of mine, SleepySecurityNews.  I'd link the site here, but I'm almost certain it isn't functional right now as I have not maintained it in quite some time.

The first thing that my app requires is a SQL server database.  From an Azure perspective, this translates into two resources, an Azure SQL Server and an Azure SQL Database.  This is also how it is represented in Terraform.  The two resources are:

•  azurerm_sql_server
•  azurerm_sql_database

 So first, a couple of notes that I learned while doing this:

Variables

Terraform has the concept of variables.  Like ARM templates,  you can specify defaults in the original definition.  You can also pass in variables via a file, an environment variable, or on the command line at "apply" time.  It seems pretty cool.

Variables are stored in state, which by default is a local file on disk.  When you setup an Azure SQL Server, you do have to specify an admin password in the provider.  You will need to refer to the documentation on sensitive data to understand the implications.  This is something I need to look into further and will try to address in a separate blog post.

Concat doesn't work on strings, use format for that

Like ARM templates, there are several functions that you can call within the template.  Of course I instinctively used concat to merge two strings together.  Nope!  Apparently this feature was removed and the recommended advice is to use the format command to put the strings together.  Makes sense!

Timing Error

When I deployed the script, I got a weird timing error.  Terraform deployed the database server, and then when it tried to deploy the database itself, I got the following error:

Essentially, it was telling me that the db server does not exist (when it was trying to deploy the database itself).  Re-running the script again without any changes resolved the issue.  I could see this as a problem going forward, and will keep an eye when building out chains with dependent resources.

In any event, here is the full code that I used to deploy:

provider "azurerm" {
    subscription_id = ""
    client_id = ""
    client_secret = ""
    tenant_id = ""
}

variable "ssn_environment" {
    type = "string"
    default = "dev"
    description = "environment short"
}

variable "ssn_db_name" {
    type = "string"
    default = "ssn"
    description = "The name of the sql server db to use"
}

variable "ssn_db_server_name" {
    type = "string"
    default = "ssndbusw"
    description = "The name of the sql server to use"
}

variable "ssn_resource_group_name" {
    type = "string"
    default = "ssn_terraform"
    description = "The name of the resource group to deploy all resources"
}

variable "ssn_db_edition" {
    type = "string"
    default = "Basic"
    description = "The sql server edition"
}

variable "ssn_location" {
    type = "string"
    default = "West US"
    description = "The location to deploy resources to"
}

variable "ssn_db_server_login" {
    type = "string"
    default = "ssndbadmin"
    description = "The db server login name to use"
}

variable "ssn_db_password" {
    type = "string"
    default = ""
    description = "The password to use"
}

resource "azurerm_sql_server" "ssn_dbserver" {
    name = "${format("%s%s",var.ssn_environment,var.ssn_db_server_name)}"
    resource_group_name = "${var.ssn_resource_group_name}"
    location = "${var.ssn_location}"
    version = "12.0"
    administrator_login = "${var.ssn_db_server_login}"
    administrator_login_password = "${var.ssn_db_password}"
}

resource "azurerm_sql_database" "ssn_db" {
    name = "${var.ssn_db_name}"
    resource_group_name = "${var.ssn_resource_group_name}"
    server_name = "${format("%s%s",var.ssn_environment,var.ssn_db_server_name)}"
    edition = "${var.ssn_db_edition}"
    location = "${var.ssn_location}"
}

When running the apply, I specified a var on the command line.  Something like the following would work:

terraform.exe apply -var "ssn_db_password=<enter password here>"

In conclusion, we ran through a more complex Terraform setup that provisioned an Azure SQL server and database.  During this process, I learned more about variables, a bit about interpolation functions within the Terraform language, and ran into a timing issue.  I hope you are enjoying this journey in learning Terraform!

Getting started with Terraform

When I look back at my time with Azure, I've made the progression from portal, to powershell, to ARM templates.  As I am writing more and more infrastructure-as-code, I'm always looking for better ways to execute deployments as part of a build/release pipeline.  For a while now, Terraform has been on my list of technologies to look in to.  The goal of this post is to provide a brief intro into terraform and build a simple storage account via template.

What is Terraform?

According to their website, Terraform is a "tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions."

Terraform excites me for a few reasons, namely:

• The fact that there is a plan step, allowing you to confirm changes
• The fact that it is open-source
• The fact that you can add on other providers (custom or otherwise

Obviously, Terraform is an abstraction language, and comes with the various downfalls of this approach.  It'll be interesting to see where the limitations are.

Installing Terraform

There are two ways you can access terraform, and it really depends on what you are trying to do.  Firstly, and probably the easiest, is you can download a binary.  The second is that Hashicorp does release a docker container with the executable on it.   

Initially I had opted for the docker container, but I was having issues with my docker for windows setup and sharing drives, so I resorted to using the executable.

Configuring the AzureRM Provider

Terraform has the concepts of providers, and you need to initialize one to connect to AzureRM.  The provider configuration looks something like the following:

provider "azurerm" {
  subscription_id = "..."
  client_id       = "..."
  client_secret   = "..."
  tenant_id       = "..."
}

The authentication mechanism is using Service principals, and you will have to set something up.  There is a script that you can download to automate the process, but I just ran the commands manually.

What I like about the providers it that you can default the pulling of credentials to use environment variables.  This is a step up from putting the provider details in code.  I'm sure there are more secure ways of passing in credentials which I haven't gotten to yet.  In order to mitigate risk here, I only gave access to this service principal to the required resource group.

The storage account resource

The definition for this was surprisingly easy.  Here is my code:

resource "azurerm_storage_account" "shamtestterraform" {
    name = "shamtestterraform1"
    resource_group_name = "terraformTest"
    location = "westus"
    account_type = "Standard_LRS"

    tags {
        environment = "test"
    }
}

You can find information on the resource definition using this link.

Deploying via Terraform

There are a couple of steps that you can perform to start a deployment via Terraform.

Step 1 is to run "terraform init"

Essentially this step confirms the provider is setup correctly and installs required modules/code to make everything run.

Step 2 is to run "terraform plan"

This is one of the main reasons why I like the idea of Terraform.  Essentially this step will refresh the current state and then compare the code being applied to the that state.  It will produce a report as to what will be created/updated/destroyed.

Here is a screenshot of what this looks like:

Pretty cool.

Step 3 is to run "terraform apply"

This essentially applies to code to the targets.  Creating a storage account is pretty simple and resulted in only one event in Azure.

What is interesting to note is that this does not create an ARM template and then do a deployment.  It is calling the API directly.  At a later date, I want to crack open the code and have a look and what it is actually doing.

Now that you are deployed, you could run "terraform destroy" to remove any created resources.

In conclusion, the goal of this post was to experiment with Terraform on a very simple Azure deployment.  So far, I like what I see, and I hope to create more with Terraform in the future.

Static Code Analysis for ARM Templates?

In a previous post I discussed the fact that ARM templates have an interesting flow when it comes to dealing with passwords.  The recommendation was that the ARM templates should be reviewed on each check-in.  This got me thinking that maybe we need a static code analysis tool for ARM templates.  The goal of this post is to discuss this in more detail.

For a long time now, code analysis tools (particularly static ones) have been in use for many managed and un-managed languages.  It is so popular that code analysis is built-in to Visual Studio 2017.  In fact, security guidance generally recommends the use of these tools.  Taking this further, many development shops use this type of analysis for not only enforcing code quality but also enforcing style guides, etc.  As ARM templates are "infrastructure-as-code" it only makes sense to try and extend these processes to those artifacts as well.

I think there are two avenues that static analysis could assist with ARM templates.  Firstly, it could identify common security concerns that may be found in these templates.  The first one that comes to mind is how passwords are handled.  As my previous post discussed, you can pass in passwords via securestring, or you can use keyvault references.  The better method is to use keyvault, and so we could write a test that ensures no parameters of type "securestring" exists.  This is just one example.

The second use for this would be around style.  One example is the use of description attributes on parameter elements.  You could issue a warning (for example) if a parameter was missing this attribute.  This could help enforcing good design guides at scale.

I know that Microsoft has been working on expanding Azure Resource Policies, and while a step in the right direction, they focus more on what can be created  in Azure (and what properties can/should/must be set) than how a template is written.  I do see a need where rules are potentially enforced in Azure (via ARP) and also, potentially, enforced in a build-step.  The build-step aspect would provide rapid feedback to the developers of policy violations (usability, security, or otherwise).

Ultimately, coming up with a full set of tests is outside the scope of this blog post.  I'm really looking for some feedback from the community on the viability of something like this.  For completeness, here is some example code that could be used to write tests.

#
# Script.ps1
#

param(
 [Parameter(Mandatory=$true)]
 [string]$filePath
)

class ARMParameter{
 [string]$name
 [string]$type
 [string]$metadata
 ARMParameter([string]$name,[string]$type,[string]$metadata){
  $this.name = $name
  $this.type = $type
  $this.metadata = $metadata
 }
 [string] ToString(){
  return "{0}-{1}" -f $this.name,$this.type
 }
}

class ARMResource{
}

Class ARMTemplate{
 hidden [PSCustomObject]$json
 hidden [ARMParameter[]]$parameters

 hidden [Void] ParseParameters(){
  Write-Verbose "Entering ParseParameters"
  $parsedParameters = @()
  $parametersRaw = $this.json.parameters
  foreach ($parameterRaw in $parametersRaw.psobject.Properties){
   Write-Verbose "parsing $parameterRaw"
   $name = $parameterRaw.Name
   $type = ""
   $metadata = ""
   foreach ($parameterRawContent in $parameterRaw.value.psobject.Properties){
    if ($parameterRawContent.Name -eq "type"){
     $type = $parameterRawContent.value
    }

    if ($parameterRawContent.Name -eq "metadata"){
     $metadata = $parameterRawContent.value.description
    }
   }
   $parsedParameters += [ARMParameter]::new($name,$type,$metadata)
  }
  $this.parameters = $parsedParameters
 }

 ARMTemplate([string]$filePath){
  Write-Verbose "Creating ARMTemplate Object"
  $this.json = ConvertFrom-Json -InputObject ((Get-Content -Path $filePath) -Join "`n")
  $this.ParseParameters()
 }

 [PsCustomObject[]] GetParametersByType([string]$type){
  return $this.parameters | ? {$_.type -eq $type}
 }

 [PsCustomObject[]] GetParameters(){
  return $this.parameters
 }
}


if (!(Test-Path $filePath)){
 Write-Error ("$filePath does not exist")
 exit
}
$armTemplate = [ARMTemplate]::new($filePath)

Write-Host "Test 1: Parameters with securestring" -ForegroundColor Cyan
$parameterWithTypeSecureString = $armTemplate.GetParametersByType("securestring")
if ($parameterWithTypeSecureString.length -eq 0){
 Write-Host "PASSED: No parameters with securestring type found" -ForegroundColor Green
} else {
 Write-Host "FAILED: Review Required: Paramters found with securestring type" -ForegroundColor Red
 Write-Host $parameterWithTypeSecureString
@"
Notes
===========
 Since these securestring parameters are passed in via the template, a reviewer should ensure that the pipeline for deployment
 handles the password in a secure manner.  An example would be a build server that retrieves the password from a secret store.
 Pipelines that give the developer access to the password for deployment purposes, or that store the credentials in plain-text
 should be avoided.
"@  
 
}


Write-Host "Test 2: All parameters should have metadata flag" -ForegroundColor Cyan
$parameters = $armTemplate.GetParameters()
$parametersWithNoMetadata = $parameters.Where({[string]::IsNullOrEmpty($_.metadata)})
if ($parametersWithNoMetadata.length -gt 0){
 Write-Host "Failed: Review Required: Metadata description missing from parameter fields" -ForegroundColor Red
 Write-Host "The following properties should have a metadata description added"
 Write-Host $parametersWithNoMetadata
@"
Notes
===========
 All parameters should contain a metadata tag with a description tag that defines the purpose of the property and where it 
 is used in the template.
"@  
} else {
 Write-Host "Passed" -ForegroundColor Green
}

So what are some thoughts around this concept?  Is is worth creating something like this where a team could create "standards" both from a usability perspective and a security perspective, and have it enforced (likely as part of a build pipeline)?  Does one of these exist in the marketplace and I am simply unaware?

Running AzurePS on Linux with Docker

I decided to play around with my linux box a bit today, and needed a project to help me on my way. Previously, I had fooled around with the azure-cli on Linux, but didn't really like the experience at all. Since Microsoft has made a big push into the world of linux, I decided to take a look at how I could install the Azure powershell on my linux box.

In order to further complicate my life, I did not want to "sully" my main operating system.  Docker is a good way for keeping some isolation between the tests that I do, and so, I decided to install Azure PS on a docker container.

First things first, I am running fedora 25 on my desktop.  Installing docker was pretty easy, especially if you use their get script.  I downloaded it first and had a boo, I hate the move these days to encouraging admins to run downloaded files directly from the internet in a root term.

Step 1)  Installing Docker on Fedora

You can find more information on how to install docker on Fedora here.

Be sure to go through the entire install process and add your regular user to the docker group (that you end up creating).  This is a good idea so you don't have to run every terminal as root.

Step 2)  Pulling a base image

Once docker is installed, you will need to pull a base image to work with.  The powershell-core team has released an installer for centos, but nothing for fedora.  Don't get me wrong, I tried installing it on Fedora but ran into dependency hell.  Most of the solutions on the internet were pointing at downloading the icu lib from some 3rd party repos and installing it so you had the correct version.  No thanks.

I decided to backtrack and install the centos build.

 docker pull docker.io/centos 

Step 3) Installing Powershell in the docker image

The next step is to run the centos image in interactive mode so you can start to manipulate it.

 docker run -i -t docker.io/centos 

You will want to download the powershell rpm.  You can find that here.  I had already previously downloaded it so I used the "docker cp" command to copy it into my running container.

Follow the instructions for installing the powershell rpm, which can be found here.

Step 4)  Test Powershell

Once you have powershell installed, run powershell from the command line to see it pop open.  Quick note, SELinux was complaining to me as soon as I ran this command.  It seems that powershell requires use of the "getsession" object, which it objected to.  I ran the recommend commands (which essentially look at the denials and issue grants for them).

After this, I see powershell, and could run commands!

Pretty neat!

Step 5)  Install Azure Modules

If you were under a rock for the last few months (like me) you would have probably missed that there are now two versions of powershell, desktop and core.  Core is not to be confused with Windows Server Core, which actually runs powershell desktop (sigh!).  The core (argh!) difference is that one is built to run on .NET core and the other requires the full .NET.  You can see more here.

It is important to note at this point that the main Azure modules require powershell desktop, and are not compatible with core.  You need to install a different set of Azure modules that were designed for core.  Spoiler alert, not all the modules are there at time of writing.  You can find the instructions for installing the core version of the Azure modules here.

Step 6) Profit

Well not really.  After you get everything installed and in your path, you should be able to run the AzureRM commands from the powershell prompt from your linux box. (That previous sentence was not even conceivable 5 years ago).

And there you go!  You are ready to rock.  If you want to skip all the pain and download my version, you can by pulling from my docker repo.