An intro look at logging for Azure Storage

As I am sure you know by now, Azure storage is implemented as a service.  Because of this, Azure storage is accessible over the internet to any location in the world.  Given sufficient authentication (IE: Azure storage key or SAS tokens) you can access any storage account.  There is no way to make this communication completely private, and therefore, most "prevention" type of security controls are not applicable to this type of deployment.  The goal of this post is to chat a little bit about Azure storage logs and how we can use them to gain some understanding of what is going on with our storage accounts.

The key questions I would like to understand are the following:

• Can I determine when my keys are being recycled?
• Can I determine who is accessing my storage account?
• Can I determine what is being accessed from my storage account?
• Can I determine how my storage account is being accessed?

Before we dive into answering those questions, let's talk a little bit about the logs that are available within Azure storage.

The first log that we can look at is the activity log for the Azure storage account.  This log will capture all operations that were executed on a storage account, essentially representing the log of the control plane on a given resource.  For more information on these logs, please see https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-activity-logs.  In this log I would expect to see CRUD operations on settings/configuration relating to the particular resource.  Specifically, I would probably want to be looking at these logs to assist with understanding my key management operations.

Here is an example of the activity log for a given azure storage account:

As you can see from the image above, you can very quickly identify the operation type and who initiated the event.  Clicking on a particular event will give more detailed information that conforms to the following schema: https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-activity-logs#event-schema.  Of particular note will be fields such as httpRequest which contain the client IP addresses of the action, and the correlationId/EventID which can be used for further troubleshooting .

The second log that we can look at is the Azure storage diagnostics logs.  This log, when enabled, can capture metrics on the storage account as well as transactional level details on actions done on the storage account.  This log represents the actions conducted against a storage account at the data plane level.  For more information on these logs, please see https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/enabling-storage-logging-and-accessing-log-data.  In this log, I would expect to see information about CRUD actions against resources within the Azure storage account.  It is important to note that these logs are stored inside a special container within the Azure storage account, and can be accessed by downloading them (via Azure storage explorer) to your desktop and analyzing them.  For information on the format of these logs, please see https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/storage-analytics-log-format.

Okay, now that we have a brief understanding of the logging options in Azure storage, let's have a look at answering the questions posed at the beginning of this post.

Can I determine when my keys are being recycled?

As this is an action against the resource itself, we can turn to the Azure activity log to see this event.  Here is a snippit of what this event looks like.

From the snippit above we can quickly see the event that occurred, the date/time, who initiated it, the scope of the authorization and so on.  One key piece of information that is missing is that we cannot see which key was actually regenerated.  This is mostly likely because the regenerate key action takes with "keytype" parameter as a body element rather than on the query string.  Here is a snippit from powershell:

From the MSDN docs (https://msdn.microsoft.com/en-us/library/azure/dn495112.aspx) you can see that the KeyType parameter can be either primary or secondary.

Can I determine who is accessing my storage account?

To answer this question, we can turn to the blob diagnostic logging.  In the log, there is a field for the IP address requesting the blob. 

Can I determine what is being accessed from my storage account?

Once again, the blob diagnostic logging reveals this information via the request-url and the requested-object-key parts of the log.

Can I determine how my storage account is being accessed?

Once again, the blob diagnostic log does capture this information in the authentication-type parameter.

The issue here again is that there is no reference to which storage key is used, rather just a record with the word "authenticated" in it. 

In conclusion, it looks like between the audit log and the diagnostic log, once an put together a picture of key events in the system and start to better understand the access/usage of the storage account.