Sunday, July 9, 2017

Azure ARM Templates and Passwords

Password management in CI/CD pipelines is an important task in creating a secure devops workflow.   The goal of this post is to chat a little bit about ARM templates and how passwords are used within them.

How are passwords passed in?

Regardless of the keyvault integration options in ARM templates, the base template itself accepts a password as a securestring and uses it like that during its processes.

Here is an example of an Azure sql server being deployed via template.

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "password": {
      "type": "securestring"
    }
  },
  "variables": {
    "blahName": "[concat('blah', uniqueString(resourceGroup().id))]"
  },
  "resources": [
    {
      "name": "[variables('blahName')]",
      "type": "Microsoft.Sql/servers",
      "location": "[resourceGroup().location]",
      "apiVersion": "2014-04-01-preview",
      "dependsOn": [ ],
      "tags": {
        "displayName": "blah"
      },
      "properties": {
        "administratorLogin": "blah",
        "administratorLoginPassword": "[parameters('password')]"
      },
      "resources": [
        {
          "name": "AllowAllWindowsAzureIps",
          "type": "firewallrules",
          "location": "[resourceGroup().location]",
          "apiVersion": "2014-04-01-preview",
          "dependsOn": [
            "[resourceId('Microsoft.Sql/servers', variables('blahName'))]"
          ],
          "properties": {
            "startIpAddress": "0.0.0.0",
            "endIpAddress": "0.0.0.0"
          }
        }
      ]
    }

  ],
  "outputs": {
    "password": {
      "type": "string",
      "value": "[parameters('password')]"
    }
  }
}

Can I directly output the password, in it's unsecured form?

You bet.  If you notice in the template above I pass in a securestring but output the same variable as a string.  Here is what the output looks like:


It is important to note that this is possible regardless of if the keyvault references are used or not.

Do the passwords show up in debug mode?

ARM template deployments have a debug level that you can set.  In the debug level, you can ask for it to log the requests, the responses, or all.  When passing securestring passwords, the passwords are converted to plaintext before being transmitted across the wire.  As such, they show up in the debug logs.  For example:


How do I do this securely?

Good question.  Ultimately, because even the keyvault-integration interacts with ARM templates as a securestring, it makes it hard to build a fully secured pipeline.  Here are some thoughts:

Deployment pipeline separation

Essentially, I think ARM template deployments should go through a secured build/release pipeline that does not allow developers access to edit the deployment process.  This protects against developers needing direct access to the secrets (either in the pipeline or in keyvault) and protects against developers using the debug features to pull the requests with the unsecured password in it.

ARM Template separation

At this point, there are many ways to leak the confidential data in an ARM template.  I showed one above, by using the output feature.  It is important to note that this would probably work in a separate deployment pipeline since the pipeline itself would log this event and the developer probably has access to the deployment logs for "debugging purposes".  I think the solution here is more of a process one than a technical one.  Here are some suggestions:

  • Separate project for ARM templates
  • Code review on each check-in
  • Non-developer resource to modify build/release pipeline to use new version of the template
In conclusion, password management in raw ARM templates leaves much to be desired.  The fact that passwords can be leaked in many ways requires firm processes in the build/release pipeline to protect secrets.  Your process may still require multiple people to ensure requirements such as segregation of duties are met appropriately.