Since we started using some Azure Services in our department, it came also the need to stay on top of the costs of the things we are doing. I went right away to the Cost Management + Billing Center to find out that while it provides the Budget feature where we can send alerts if it reaches certain thresholds, I couldn’t send daily reports with the costs accrued to date.
In this post, I’ll show you the solution I built to help me and my managers to keep on top of the expenses in Azure on a daily basis.
Cost Management + Billing Budgets
As I mentioned, the Budget feature in Azure only notifies a user when certain thresholds are met and we were looking for a daily constant reminder of the spending in Azure.
To create a budget you specify the scope by selecting a management group, subscription os resource group , the name, the reset period, start date, expiration date, and the value you want to track.
Then you configure how the alerts are going to be sent, by specifying one or more emails.
You can also create an action group which will give you options to send data to a Logic App, Azure Functions and Webhooks in general among a few other options as below.
But again, even though this budget feature would fit well in some scenarios, in our case for a daily tracking was not enough.
Our Budget Notification Solution
There are a couple of ways that I found of doing this. One is through calling the Azure Management APIs and Sam Bowen-Hughes wrote about how to use them in his blog:
https://medium.com/@sambowenhughes/getting-started-with-the-microsoft-azure-billing-apis-aa27af11c1d0.
Since due to some restrictions in the kind of access that I have to these APIs, I used the export capabilities of the Cost Management service in Azure to achieve the same results.
The overall design for this will be:
- Export file generated from Cost Management to Azure Storage;
- Event Grid to notify a Logic App that a new file has been created;
- Logic App to handle the email notification;
- Azure Function to aggregate the data;
Cost Management – Export File
To export the file you click on the Add button in the Exports blade and the following window will show up:
In there you specify the name of the export and the type of the export. Since I wanted a daily report, I selected that option. Please, note that one of the first things you need to do is to select the scope on which the export file will be generated. You specify a management group, a subscription or even a resource group, so basically it depends on the kind of granularity that you want.
After defining the scope and the interval you want the export file, select the storage account where the file will be stored.
Event Grid + Logic App
With that done, open the Storage Account you should have created beforehand and configured an Event Grid subscription. To facilitate the creation, use one of the templates provided in the Events blade as below:
When you click in Logic Apps, it will send you to a Logic App creation where you need to sign-in with your Azure AD credential or with a Service Principal. The Service Principal is the best option.
After the sign-in, specify which subscription, resource type and resource name you want to receive events about as below:
I also added a few more conditions where I just wanted to know about BlobCreated events and that the Subject contained the name of the folder that I specified in the configuration of the Export File in Cost Management.
The remaining of the Logic App workflow is to get the contents of the file from the Blob Storage we were just notified about, call an Azure Function to organize the data to us, and create HTML tables so that we can present in a formatted HTML email.
The action to send an email is like this:
Azure Function – Aggregator
The file generated by the Export File in the Cost Management service looks like below. For brevity and to be easier to read, I just left here the columns I’m interested about, but the file contains a lot more information about the Azure resources.
|
1 2 3 4 5 |
ResourceGroup ,MeterCategory ,PreTaxCost ,ResourceType rg-rg1-sb ,Azure App Service ,0018 ,/.../microsoft.web/sites/site-rg1-sb rg-rg2-sb ,Service Bus ,1.50 ,/.../Microsoft.EventHub/namespaces/ehubrg2sb rg-rg2-sb ,Storage ,0.0055 ,/.../Microsoft.Storage/storageAccounts/sarg2sb rg-rg2-sb ,Bandwidth ,4.7080 ,/.../microsoft.web/sites/fapp-rg2-sb |
The first thing that the Azure function needs to do is to capture the files and transform it in lines so we can do the aggregation. To capture the lines you need to do the following code:
|
1 2 3 4 |
string body = await new StreamReader(req.Body).ReadToEndAsync(); string[] csvLines = body.Split('\r'); var headers = csvLines[0].Split(',').ToList<string>(); |
Once we have the lines, then we can navigate through them and choose only the columns we are interested in by creating a collection:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
foreach (var line in csvLines.Skip(1)) { if (string.IsNullOrEmpty(line.Replace("\n", ""))) continue; var lineObject = new JObject(); var lineAttr = line.Split(','); AzureResourceItem azResource = new AzureResourceItem(); for (int x = 0; x < headers.Count; x++) { switch (headers[x]) { case "ResourceGroup": azResource.ResourceGroup = lineAttr[x]; break; case "MeterCategory": azResource.ResourceType = lineAttr[x]; break; case "PreTaxCost": azResource.Cost = decimal.TryParse(lineAttr[x], out decimal cost)? cost: 0; break; default: break; } } resultSet.rows.Add(azResource); } |
With the collection ready, then we can start using some LINQ commands to help us do the aggregation in an easier as the code below shows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
foreach (var item in resultSet.rows) { if (azureTotals.ResourceGroupTotals.FirstOrDefault(x => x.Name == item.ResourceGroup) == null) azureTotals.ResourceGroupTotals.Add( new AzureGroupTotal { Name = item.ResourceGroup, TotalCost = resultSet.rows.Where(x => x.ResourceGroup == item.ResourceGroup).Sum(x => x.Cost) }); if (azureTotals.ResourceTypeTotals.FirstOrDefault(x => x.Name == item.ResourceType) == null) azureTotals.ResourceTypeTotals.Add( new AzureGroupTotal { Name = item.ResourceType, TotalCost = resultSet.rows.Where(x => x.ResourceType == item.ResourceType).Sum(x => x.Cost) }); } return (ActionResult)new OkObjectResult(azureTotals); |
Summary
With a few lines of code and configuring a few resources, I have my Azure Daily Budget Report so I and the people of the department I work for can stay on top of the Azure expenses and in case some unexpected costs start to happen, we will receive a notification sooner than then standard Budget notifications in Azure.
