Azure Metadata Service Scheduled Events: An initial look

For a little while now, the Scheduled Events functionality in the Azure Metadata service has been in preview.  I recently had a chance to play with this on a test VM and the goal of this post is to walk through some basics.

From a setup perspective, configuring scheduled events for your VM is quite easy.  All it takes is a get request from your VM after setup to a private, non-routable IP address.  As per the documentation, this server may take up to 2 minutes to initialize, so be aware of that.

There are three types of events that are handled by this server: Freeze, Restart, and Shutdown.  The idea behind the service is that you could have a task running continually to watch this endpoint waiting for events.  When it does find an event, you could handle whatever is required (say gracefully shutdown the service or fail to a secondary) and then action the event by sending a POST request back to the endpoint.  Pretty neat for being able to handle HA situations.

I found a couple of interesting things while playing around with this service.

- Deallocating a machine clears any pending events.  This makes sense based on my understanding of what deallocation actually does.  Further, when you hit the endpoint again upon reallocating the VM it seems to "re-register" with the service

- Issuing a manual restart when a restart event is pending does not clear the restart event.  This makes sense and lends itself to the documented workflow that you should ask the Azure fabric to run the pending event by "approving it"

- I also noticed that there is a resources element in the JSON file but it seems to append an _ to the resource name. See the following image:

I think there are some interesting use cases for this service.  Some thoughts that I have:

- It would be nice to see this added as part of an ARM template, maybe as a VM extension
- It will be interesting to see how this integrates with toolsets like DSC.  I would image that you would have to disable them for some period to avoid auto-correction like capabilities.  The issue with this is that you don't get an event to trigger when the scheduled event is finished (or has just finished)
- I would love to see something more global (IE: reporting in the portal along with email alerting).

It is cool to see Azure moving to deliver more host health information and this service is no different.  I think there are some workflow pieces to work out as this gets integrated into a production environment.

Azure Automation Runbooks: Who ran me?

One of the interesting challenges with Azure automation is enforcing security throughout the runbook process.  As I have mentioned in previous posts, permissions within the system are not super granular and automation jobs execute in a service account context.

It turns out there is a way to at determine who actually executed a particular runbook.  You could use this identity in authorization/authentication decisions within the runbook as required.  It is important to note that this is a string reference to the email address of the person running the runbook.  This is set by the system, so trust it if you wish!

The first part is to get the job id of the currently running job.  You can accomplish this by looking at the $PSPrivateMetadata object which contains a JobId.  For example,

$jobId = $PSPrivateMetadata.JobId.Guid
$jobId

The second part is using this job ID and the Get-AzureRMAutomationJob cmdlet to determine who ran the runbook.  For example,

$job = Get-AzureRmAutomationJob -Id $jobId -ResourceGroupName "resourceGroup" -AutomationAccountName "automationAccount"
$job 

Keep in mind that you do need to log in to Azure to run the above command.  After this, the output looks like this:

As you can see, there is a StartedBy property that contains my email address.  You can now use that string to make decisions in your automation runbooks.

One interesting thing to note is that "StartBy" appears blank when you use the "Test Draft In Azure" functionality of the Automation Authoring Toolkit.

Azure Automation Development Process Considerations

In a recent post I discussed a case for multiple Azure automation accounts depending on the environment that is being targeted.  In this post, I would like to expand on some process considerations for developing on the Azure automation platform.

1) I prefer the ISE add-on modules over the built-in editor

While there are probably many methods of creating Azure Automation runbooks, the two most common are to use the build-in portal editor or the Powershell ISE Authoring Add-on.  From a development perspective, I much prefer the ISE add-on.  It has a very easy workflow for creating and testing runbooks.  When I am doing development, I am usually connected to the dev version of my automation account.  I can then do something like the following:

- Edit runbook locally (Puts runbook in EDIT)
- Upload runbook via built-in command (Puts runbook in EDIT)
- Use the built-in test to run a job
- When I'm satisfied, click on the publish command (Puts runbook in PUBLISHED)

What I really like about this method is that intellisense works on all child runbooks (provided you have them downloaded).  I have always found the intellisense in the portal version to be lacking in almost all usecases.

There are a few things that I would love to see improved in this add-on.  First of all, when you open a runbook it automatically gets put in "EDIT" mode in the portal.  I really don't like this as it should only be in edit once I have actually started editing or on first "upload" to Azure.  The second thing that is tricky is that you cannot "sync to source control" from the ISE plugin.  You can sync from, but that isn't the use case. 

In order to keep github in sync, I generally log into the portal after my edits are complete (and I'm happy with them).  From there

- Edit the runbook via the UI
- Click the Github sync button
- Click the publish button

This process is painful, and I'm hoping to build some code to create a better process.  The main blocker is the fact that the runbook used to sync automation to github (and vice versa) is not a "callable" runbook.

2) Use ARM templates to keep multiple Automation Accounts in sync

When I am doing work in Azure automation, especially when we are trying to automate IT operations, I am always trying to determine the best location for variables and credentials in the system.  Obviously Azure Automation credential store is a no-brainer, but variables can be placed in several locations depending on the need and ability of the team supporting it.  In order to keep automation accounts in sync, I find ARM templates to be the best way.  The process would look like the following:

- Make edits by hand using the portal until a pattern/process has been established
- Use the "automation script" functionality of the portal to export an ARM template
- Deploy the ARM template to test and production as required

Automation has pretty rich ARM template support, as you can see here.

3) Use workflow where possible, where not possible, use workflow anyways

I'll be honest, most of my powershell scripting these days is starting to go down the path of workflow.  While workflow can have it's challenges, I find it a much better way to write my IT automation scripts.  There are two main reasons why I tend to write workflow over regular scripts

- Parallel processing saves money
Remember that Azure Automation charges by the minute that a job is running.  In many cases, parallel processing can greatly increase the speed at which your runbooks finish.

- Easier to enforce order
I find the sequence/parallel tags in workflow a convenient way to enforce order of operations in IT automation processes.

If you are interested in learning more, there is a great resource that details some of the considerations for writing workflow in Azure Automation.