Skip to main content

Azure Budget Filters: A Key Tool for Effective Cloud Cost Management

· 12 min read

Azure Budgets are a vital tool that can be used to keep on top of your Cloud financial management (FinOps) Microsoft Azure platform potential and actual costs.

The most effective Azure Budgets - are the ones that you use!

 Azure Budgets

In the realm of Cost Management, budgets play a pivotal role in facilitating the planning and implementation of organizational accountability. These tools enable proactive communication regarding expenses and support the management of costs by closely monitoring spending trends over extended periods.

One can set up alerts based on current or projected expenditures to maintain adherence to the established organizational spending limit. Upon surpassing the budget thresholds, notifications are promptly triggered. Such occurrences neither impact any of the available resources nor interrupt any consumption processes.

By leveraging budgets, it becomes possible to perform detailed cost analysis and track expenses effectively.

Azure Back to School - Azure Budget Filters

Be aware of the delay with the Cost & Usage data, as there may be a difference between what you end up seeing in the Portal and the Budget itself - so make sure you account for this to be advised as early as possible:

Cost and usage data is typically available within 8-24 hours and budgets are evaluated against these costs every 24 hours.

Be sure to get familiar with Cost and usage data update specifics. When a budget threshold is met, email notifications are normally sent within an hour of the evaluation.

Note: Azure Budgets are not supported on Subscriptions, where you can't access Microsoft Cost Management, i.e. Azure Sponsorship subscriptions.

A time to clean the windows - Budget Scopes

When creating an Azure Budget, you can specify a Scope. A scope is the level of your hierarchy (i.e., if it's a Resource Group Budget, it cannot report on resources at the Subscription, you would have to create a Subscription or Management Group scoped Budget).

When you create an Azure Budget, they can be made at the following Scopes:

Microsoft Azure Budget Scopes

Most people, when creating scopes, will create a Scope at the Subscription and/or Resource Group level - there is no right or wrong answer when it comes to your Azure Budget Scope - this needs to work for you and your organisation, ie if you have a Project per Resource Group - then it would make sense to create a Budget per Resource Group, the same for Subscriptions.

You can also have multiple Azure Budgets at the same or different scopes, so a combination of Budgets may be the most effective. An example could be a Subscription Budget that may go to a Product Owner, but a Management Group could go to Finance or the Technology teams.

Keep in mind, that Budgets on their own are just a forecasting and alerting tool, they won't stop resources from running, if it goes over an alert threshold, out of the box - the Budget doesn't touch your resources, merely gives you an opportunity to proactively react to them, before costs become a problem.

Time to scrum the floor - Create an Azure Budget

Let's go through the process of creating an Azure Budget, using the Azure Portal.

  1. Log in to the Microsoft Azure Portal
  2. In the Search bar above, search for Budgets
  3. Click on Budgets
  4. Change the Scope
  5. Cost Management - Azure Budget
  6. You can select a Management Group, Subscription, or Resource Group for the Scope, by clicking on each - in my example, I have a Management Group named: mg-landingzones, which I am going to select.
  7. Click Select
  8. Now that the Scope has been set, we can add our Budget to the specified Scope, click + Add
  9. We will come back to Filters, in another section - but for the Budget details you will need:
    • Name this is the name of your Budget, make sure its something meaningful (ie Monthly-Budget_MG-LandingZones)
    • Reset period (Monthly/Quarterly or Annual, this is the period that the Budget resets back to $0 - you can't go wrong with Monthly)
    • Creation date (the date that the Budget will start)
    • Expiration date (the date that the Budget will stop)
  10. Budget Amount(this is the overall; amount that you are planning on your resources to spend)
  11. Azure Portal - Create budget
  12. Once you have entered in your Budget details, click Next to configure your Alert conditions. The alert conditions are where you can specify, what you want to alert on ie 50% or 80% of the overall budget amount. Actual - is when it financially reaches that point and Forecasted - is when current consumption is forecasted to reach that budget.
  13. Specify an email address to send the Alert to and click Create.
  14. Azure Budget Conditions

Time to clean the dishes - Azure Budget Filters

By default, Scoping the Budget to the Subscription or Resource Group is good enough for 95% of the use cases - but using Budget filters, you can enable a bit more flexibility - for scenarios such as:

  • Product-centric alerts
  • Service centric alerts

While the scope is your level of the Azure hierarchy, your filter is your handrail to stop you from falling, currently, Microsoft offers the following filters:

PropertyWhen to useNotes
Availability zonesBreak down AWS costs by availability zone.Applicable only to AWS scopes and management groups. Azure data doesn't include availability zone and will show as No availability zone.
Billing periodBreak down PAYG costs by the month that they were, or will be, invoiced.Use Billing period to get a precise representation of invoiced PAYG charges. Include two extra days before and after the billing period if filtering down to a custom date range. Limiting to the exact billing period dates won't match the invoice. Will show costs from all invoices in the billing period. Use Invoice ID to filter down to a specific invoice. Applicable only to PAYG subscriptions because EA and MCA are billed by calendar months. EA/MCA accounts can use calendar months in the date picker or monthly granularity to accomplish the same goal.
BillingProfileIdThe ID of the billing profile that is billed for the subscription's charges.Unique identifier of the EA enrollment, pay-as-you-go subscription, MCA billing profile, or AWS consolidated account.
BillingProfileNameName of the EA enrollment, pay-as-you-go subscription, MCA billing profile, or AWS consolidated account.Name of the EA enrollment, pay-as-you-go subscription, MCA billing profile, or AWS consolidated account.
Charge typeBreak down usage, purchase, refund, and unused reservation and savings plan costs.Reservation purchases, savings plan purchases, and refunds are available only when using actual costs and not when using amortized costs. Unused reservation and savings plan costs are available only when looking at amortized costs.
DepartmentBreak down costs by EA department.Available only for EA and management groups. PAYG subscriptions don't have a department and will show as No department or unassigned.
Enrollment accountBreak down costs by EA account owner.Available only for EA billing accounts, departments, and management groups. PAYG subscriptions don't have EA enrollment accounts and will show as No enrollment account or unassigned.
FrequencyBreak down usage-based, one-time, and recurring costs.Indicates whether a charge is expected to repeat. Charges can either happen once OneTime, repeat on a monthly or yearly basis Recurring, or be based on usage UsageBased.
Invoice IDBreak down costs by billed invoice.Unbilled charges don't have an invoice ID yet and EA costs don't include invoice details and will show as No invoice ID.
InvoiceSectionIdUnique identifier for the MCA invoice section.Unique identifier for the EA department or MCA invoice section.
InvoiceSectionNameName of the invoice section.Name of the EA department or MCA invoice section.
LocationBreak down costs by resource location or region.Purchases and Marketplace usage may be shown as unassigned, or No resource location.
MeterBreak down costs by usage meter.Purchases and Marketplace usage will show as unassigned or No meter. Refer to Charge type to identify purchases and Publisher type to identify Marketplace charges.
OperationBreak down AWS costs by operation.Applicable only to AWS scopes and management groups. Azure data doesn't include operation and will show as No operation - use Meter instead.
Pricing modelBreak down costs by on-demand, reservation, or spot usage.Purchases show as OnDemand. If you see Not applicable, group by Reservation to determine whether the usage is reservation or on-demand usage and Charge type to identify purchases.
PartNumberThe identifier used to get specific meter pricing.
ProductName of the product.
ProductOrderIdUnique identifier for the product order
ProductOrderNameUnique name for the product order.
ProviderBreak down costs by the provider type: Azure, Microsoft 365, Dynamics 365, AWS, and so on.Identifier for product and line of business.
Publisher typeBreak down Microsoft, Azure, AWS, and Marketplace costs.Values are Microsoft for MCA accounts and Azure for EA and pay-as-you-go accounts.
ReservationBreak down costs by reservation.Any usage or purchases that aren't associated with a reservation will show as No reservation or No values. Group by Publisher type to identify other Azure, AWS, or Marketplace purchases.
ReservationIdUnique identifier for the purchased reservation instance.In actual costs, use ReservationID to know which reservation the charge is for.
ReservationNameName of the purchased reservation instance.In actual costs, use ReservationName to know which reservation the charge is for.
ResourceBreak down costs by resource.Marketplace purchases show as Other Marketplace purchases and Azure purchases, like Reservations and Support charges, show as Other Azure purchases. Group by or filter on Publisher type to identify other Azure, AWS, or Marketplace purchases.
Resource groupBreak down costs by resource group.Purchases, tenant resources not associated with subscriptions, subscription resources not deployed to a resource group, and classic resources don't have a resource group and will show as Other Marketplace purchases, Other Azure purchases, Other tenant resources, Other subscription resources, $system, or Other charges.
ResourceIdUnique identifier of the Azure Resource Manager resource.
Resource typeBreak down costs by resource type.Type of resource instance. Not all charges come from deployed resources. Charges that don't have a resource type will be shown as null or empty, Others, or Not applicable. For example, purchases and classic services will show as others, classic services, or No resource type.
ServiceFamilyType of Azure service. For example, Compute, Analytics, and Security.
ServiceNameName of the Azure service.Name of the classification category for the meter. For example, Cloud services and Networking.
Service name or Meter categoryBreak down cost by Azure service.Purchases and Marketplace usage will show as No service name or unassigned.
Service tier or Meter subcategoryBreak down cost by Azure usage meter subclassification.Purchases and Marketplace usage will be empty or show as unassigned.
SubscriptionBreak down costs by Azure subscription and AWS linked account.Purchases and tenant resources may show as No subscription.
TagBreak down costs by tag values for a specific tag key.Purchases, tenant resources not associated with subscriptions, subscription resources not deployed to a resource group, and classic resources cannot be tagged and will show as Tags not supported. Services that don't include tags in usage data will show as Tags not available. Any remaining cases where tags aren't specified on a resource will show as Untagged. Learn more about tags support for each resource type.
UnitOfMeasureThe billing unit of measure for the service. For example, compute services are billed per hour.

One or a combination of these filters can be used to create your own meaningful Budgets! You can target specific resources, an example is if you have resources in a Shared Resource Group - for example, Networking, and you have a VPN Gateway, that is used for a Site to Site VPN, for a specific application, that is sitting in another resource group or subscription - you can add the Resource directly into the filter of your Budget for the Azure Gateway, and then include a Tag - that may reference the rest of the application dependencies.

Budgets can be created with all sorts of various tools, from the Azure Portal to:

Relaxing beverage time - Tips & Tricks

Finished the day of cleaning! Now is the time to sit back and enjoy your favourite beverages, and read the labels on the bottles!

  • You can use the Azure Mobile Application to display your Cost and Budgets so keep on top of your consumption on the go!
  • The Microsoft Cost Management team are working on new features all the time, including improvements to Cost Management and Budgets! If you like living on the edge - be sure to check out the Preview portal (and add your feedback)!
  • You can use an Action Group, to trigger a Webhook or Azure Automation runbook - to resize or stop resources. Action Groups are currently only supported for subscription and resource group scopes, so you may need to have one Budget for Monitoring at a higher level and one Budget for running automation at a lower level.

Azure for Students

· 3 min read

Students learn in different ways through many possible avenues and experiences. Microsoft has assets to help students navigate through their journey.

This article aims to help to make access to student resources clearer.

Skilling Journey

MS Student Developer - Skilling Journey

There are 3 phases:

  • Discover
  • Engage
  • Grow

Each phase offers various ways of engage, learn and discover Microsoft Azure services and functionality.

Some of the resources for each phase can be found below:

DiscoverEngageGrow
Microsoft Learn Student HubAzure for StudentsImagine Cup
Microsoft ReactorHackathonsEvents: Student Summit
Microsoft Build
Curriculum on GitHubDev Tools for Teaching
Independent Learner

If you are an independent learner, the resources below can supplement your desire to learn:

  • Learn by Doing: Students can gain the skills to
    apply to everyday situations through hands-on
    personalized training at their own pace or with our
    global network of learning partner
  • Showcase Skills: Help advance their career by
    completing challenges that demonstrate expertise.
    Earn globally recognized and industry-endorsed
    certifications and showcase them to their network.
  • Code Samples: Test out new capabilities in their
    own projects faster and easier with code samples
    that bring Microsoft technology to life.

Build in the cloud-free with Azure for Students ($100 Azure Credit, No credit card required)

Use your university or school email to sign up and renew each year you're a student. Your school or university doesn't need to be enrolled in a program - your university and school email address is used to validate your student status.

Group Learner

If you are a group learner, alongside the same content as the indepedant learning path, check out the Microsoft Student Ambassador program to meet likeminded people, and don't forget to check out for your local Azure meetups!

Education Hub

Located in the Azure Portal, the Education Hub enables easy access to Azure offers and other academic benefits Microsoft provides.

Using the Education Hub, Students can:

  • Download free software provided by their Academic Institution
  • Sign up for academic-specific Azure offers
  • Launch self-guided role-based learning pathways
  • Deploy academic-focused ARM templates

Microsoft Azure Education Hub

FAQS

A bit of a misleading heading! But there is no need to repeat FAQS that have already been answered by Microsoft or community members; the trick is finding where to go!

Most Student FAQs (Frequently Asked Questions) can be answered on the following page:

Frequently asked questions about the Education Hub, but there may be other questions that don't fit in; if that's the case - make sure you check Microsoft Q&A - as most likely, what you need has been answered by Microsoft or a community member.

Make sure you also check out my post on How can I learn how to use Microsoft Azure? and AWESOME-Azure-Architecture list.

Azure Availability Zone Peering

· 8 min read

In most regions (and odds are, if your area doesn't have Avalibility Zones, it's on the roadmap to be set up), Microsoft Azure has Availability Zones.

Each Azure region features data centres deployed within a latency-defined perimeter. At a high level, these zones consist of 3 separate data centres with independent cooling, power, switching etc.

Azure availability zones

Azure availability zones are connected by a high-performance network with a round-trip latency of less than 2ms. They help your data stay synchronized and accessible when things go wrong. Each zone is composed of one or more datacenters equipped with independent power, cooling, and networking infrastructure. Availability zones are designed so that if one zone is affected, regional services, capacity, and high availability are supported by the remaining two zones.

With availability zones, you can design and operate applications and databases that automatically transition between zones without interruption. Azure availability zones are highly available, fault tolerant, and more scalable than traditional single or multiple datacenter infrastructures.

Availability Zone peering

Today we are going to look into Availability Zone peering:

Each data centre is assigned to a physical zone. Physical zones are mapped to logical zones in your Azure subscription. Azure subscriptions are automatically assigned this mapping when a subscription is created.

Physical Zones vs Logical Zones

There are a few things to be aware of here that I will call out:

  • Physical zones are mapped to logical zones in your Azure subscription.
  • Azure subscriptions are automatically assigned this mapping when a subscription is created.

So what does this mean?

We know we have three separate data centres within a single region:

ZoneRegion
1Australia East
2Australia East
3Australia East

We can see these zones in the Azure Portal when we create resources:

Azure Avalibility Zone - Selection

This is great for making your solutions redundant against a single data centre failure and spreading your workloads across different zones; services such as Virtual Networks are zone-redundant by default, allowing access to resources across multiple zones out of the box.

One reason you may have all your resources in a single zone could be latency.

Lets us go back to the paragraphs around physical and logical zones and mapping - what does this mean?

What this means is that each of the three data centres is assigned a physical AND logical mapping, so your Azure datacentres look like this:

Zone (Physical)RegionZone (Logical)
1Australia East3
2Australia East2
3Australia East1

When you deploy a resource into an Azure Avalibility Zone and select Zone 1, you choose the Logical Zone, NOT a physical zone.

This means that FOR EACH Microsoft Azure subscription, whether in the same Microsoft Entra ID tenancy or not, Zone 1 can be a different physical data centre.

So if you have resources deployed across multiple subscriptions, and all your resources are deployed to Zone 1 - they MAY NOT be in the same physical data centre.

Azure SubscriptionsRegionZone (Logical)Zone (Physical)
Sub AAustralia East11
Sub BAustralia East13
Sub BAustralia East11

In an example like the above, you have three separate Azure subscriptions, and you have deployed your Virtual Machines and other resources across all Azure subscriptions into Zone 1, 2 of your subscriptions are using the same physical zone for zone 1, and another subscription is using a separate availability zone altogether.

AzureAvailabilityZones_Logical

One of the reasons the logical and physical zones are different is due to capacity management; out of habit, many people select Zone 1 - this would mean that certain zones become overpopulated while others are underutilized. The logical zones allow Microsoft some ability to spread the load.

It's worth noting that mapping the Logical to Physical Zones of the Avalibility Zones within your region is done when the subscription is created.

Checking your Zone Peers using PowerShell and the Azure API

During normal business use - you don't need to know any of this; select a zone and deploy; if you have resources across subscriptions and run into additional latency - this may be why, although each availability zone is connected through a dedicated regional low-latency network with a round-trip latency of less than 2ms.

But suppose you are curious or want to delve deeper into your Disaster Recovery and resiliency architecture within a single region. In that case, it can be helpful to know the mapping.

This information isn't fed into the Azure Portal. To find the mapping, we need to query the Azure API directly using the Check Zone Peers endpoint.

To do this, I have written a rough PowerShell script that will register the AvailabilityZonePeering Azure feature that you need to enable the lookup and query the API for the mappings.

# Connect to Azure using Get-AzAccount
Connect-AzAccount

# Set the region to 'Australia East'
$region = 'Australia East'

# Get all subscriptions that the account has access to
$subscriptions = Get-AzSubscription | Select-Object -ExpandProperty SubscriptionId

# Get the access token for the authenticated user
$token = (Get-AzAccessToken).Token

# Check if AvailabilityZonePeering feature is enabled and enable it if it's not
$azFeature = Get-AzProviderFeature -ProviderNamespace Microsoft.Resources -FeatureName AvailabilityZonePeering
if (!$azFeature.RegistrationState.Equals("Registered")) {
do {
Register-AzProviderFeature -FeatureName AvailabilityZonePeering -ProviderNamespace Microsoft.Resources
Start-Sleep -Seconds 5
$azFeature = Get-AzProviderFeature -ProviderNamespace Microsoft.Resources -FeatureName AvailabilityZonePeering
} until ($azFeature.RegistrationState.Equals("Registered"))
Write-Host "The AvailabilityZonePeering feature has been enabled."
} else {
Write-Host "The AvailabilityZonePeering feature is already enabled."
}

# Define the request body for the REST API call
$body = @{
subscriptionIds= $subscriptions | ForEach-Object { 'subscriptions/' + $_ }
location = $region
} | ConvertTo-Json

# Define the request parameters for the REST API call
$params = @{
Uri = "https://management.azure.com/subscriptions/" + $subscriptions[0] +
"/providers/Microsoft.Resources/checkZonePeers/?api-version=2020-01-01"
Headers = @{ 'Authorization' = "Bearer $token" }
Method = 'POST'
Body = $body
ContentType = 'application/json'
}

# Invoke the REST API and store the response
$availabilityZonePeers = Invoke-RestMethod @Params

# Initialize an empty array for the output
$output = @()

# Loop through each availability zone and its associated peers and add them to the output array
foreach ($i in $availabilityZonePeers.availabilityZonePeers.availabilityZone) {
foreach ($zone in $availabilityZonePeers.availabilityZonePeers[$i-1].peers ) {
$output += New-Object PSObject -Property @{
Zone = $i
MatchesZone = $zone.availabilityZone
SubscriptionId = $zone.subscriptionId
}
}
$output += ""
}

# Output the results
$output | Format-Table

Once we have connected to Microsoft Azure and run the script, we will get an output like the one below, which I ran across my own three subscriptions:

SubscriptionIdMatchesZoneZone
3bdfd67e-6280-43af-8121-4f04dc84706c21
8df7caa2-95cb-44d1-9ecb-e5220ec6a82511
119bbbb7-3ab5-4eb3-ab21-3c65f562fbef11
3bdfd67e-6280-43af-8121-4f04dc84706c12
8df7caa2-95cb-44d1-9ecb-e5220ec6a82522
119bbbb7-3ab5-4eb3-ab21-3c65f562fbef22
3bdfd67e-6280-43af-8121-4f04dc84706c33
8df7caa2-95cb-44d1-9ecb-e5220ec6a82533
119bbbb7-3ab5-4eb3-ab21-3c65f562fbef33

On the right-hand side, we see the 'Zones' - these are the Physical Zones, so Zone 1 to 3.

For each subscription, we can see the Logical Zone mapping as well.

In this example, my subscription of '3bdfd67e-6280-43af-8121-4f04dc84706c', if I were to deploy to Zone 2 in my Azure Portal, would deploy to the same physical datacenter as Zone 1 of: '8df7caa2-95cb-44d1-9ecb-e5220ec6a825'.

As you can also see, my Zone 3 matches the same Zone 3 logically and physically for all my subscriptions, but there are differences between Zone 2 and 1.

Again, during normal business as usual, you don't need to know this - but it's always good to know how this works. If you want confirmation of the resiliency of your architecture across Availability Zones, this is a great way to confirm whether your resources are physically located together - or not.

Azure Architecture - Solution Requirement Consideration Checklist

· 5 min read

Building a cloud solution on Azure can be an exciting yet daunting task.

The key to a successful implementation is carefully planning and considering solution requirements using the guidance of the Microsoft Cloud Adoption and Well Architecture frameworks.

But knowing what questions to ask and data to capture to give you the bigger picture - to not only consider the solution for the short term and long term, can be difficult. This is where the Azure architecture solution requirements checklist comes in.

Leaning on the great frameworks already in place to assist with the Cloud Adoption and Azure Well Architecture frameworks, the solution requirements checklist is intended to act as a way of asking and capturing the requirements of your solutions. It can be a great reminder to discover some of those requirements (whether functional or non-functional) that you may have forgotten about!

I am using the Azure Review Checklist - as a building block! I created a custom checklist intended to work alongside the review checklists - but aimed more at the discovery and requirements-gathering stage to assist with designing the proper outcomes for the business.

At the time of this article, there are 8 main categories, and various sub categories:

  • AI
  • Business
  • Data
  • Governance
  • Infrastructure
  • Microservices
  • Observability
  • Resiliency

Examples of some questions are:

Main AreaSub AreaChecklist itemDescription (optional)
BusinessGoalsWhy are we moving the solution to Azure?Understand the reasoning behind the decision to move to a cloud platform like Azure. Helps to validate the end result reaches this goal.
BusinessGoalsWhat are the business objectives or quantifiable business goals?What is the business objectives (ie Increased sales revenue, cost reduction, customer satisaction, employee productivity
BusinessGoalsWhat outcomes will you achieve for customer?What is the  objectives for the customer? What do they want to achieve using this solution
BusinessGoalsIs there a timeline for building the solution Azure?Asking about the timeline for building a solution in Azure is important to determine resource allocation, budgeting, prioritization, and setting stakeholder expectations.
BusinessGoalsHow many people will be accessing the solution?Asking about the number of people accessing the solution helps to determine the necessary resources and scalability required to accommodate the expected traffic and usage.
BusinessGoalsIs there a targeted event or date for an announcement about the solution's availability on Azure?Timeline for architecture, deployment, testing can help determine what risks, resource requirements and cost and the delivery of solution.
BusinessGoalsDoes the solution impact a Team, Department or organization?Impact on a team, department, or organization helps determine the scope and potential consequences of the solution, ensuring that all relevant stakeholders are considered and accounted for in the decision-making process.
BusinessCustomersWhat are the customer expecations?Customer expectations helps ensure that the solution meets the needs and desires of the end-users, and make sure business outcomes match customer expectations.
BusinessCustomersIs there a deal or customer opportunity associated with having the solution in Azure?Any associated deals or customer opportunities helps to understand the potential financial benefits, vendor offerings and growth opportunities of using Azure as a platform for the solution.

Azure Architecture - Solution requirement considerations

The Azure Architecture - Solution Requirement Consideration checklist, is intended to be a living resource, I am not an expert in all fields so there may be gaps or questions you feel is relevant or missing! Feel free to open a Pull Request to contribute! This is for you!

You can find the latest version of the checklist on GitHub here: lukemurraynz/Azure_Checklists

Using this is simple!

  1. Download the latest version of the excel spreadsheet from: Azure/review-checklists.
  2. Download the latest version of the Azure Architecture checklist.
  3. Open up the review_checklist excel document and click Import checklist from JSON
  4. Import checklist from file
  5. Import checklist from file
  6. Select the downloaded: azure_architecture_checklist.en.json

Once imported, you can now save the excel document, and start adjusting the Severity, Status and add Comments - to capture the information, to then use to architect your solutions!

Note: I do not create, edit or modify the Excel spreadsheet, created by for the Azure Reviews - I simply use it to run my custom checklist. Make sure to check out the Azure Review Checklists!

There are some settings that you might need to change in your system to run macro-enabled Excel spreadsheets. When initially opening the file you may see the following error, which prevents Excel from loading:

Excel cannot open the file 'review_checklist.xlsm' because the file format or file extension is not valid. Verify that the file has not been corrupted and that the file extension matches the format of the file.

In other cases the file opens with the following message, which prevents you from being able to load the checklist items:

Unblock file or add an exception to Windows Security

  1. You might need to unblock the file from the file properties in the Windows File Explorer, so that you can use the macros required to import the checklist content from github.com:
  2. Additionally, you might want to add the folder where you cloned this repo to the list of exceptions in Windows Security (in the Virus & Threat Protection section):

Azure Deployment history cleanup with Azure DevOps

· 7 min read

Microsoft Azure has a limit of 800 deployments per resource group. This means that a single resource group can only contain 800 historical deployments at most.

A deployment in Azure refers to the process of creating or updating resources in a resource group.

When deploying resources in Azure, it is essential to keep track of the number of historic deployments in a resource group to ensure that the limit is not exceeded. This is because new deployments will fail if the limit is exceeded, and creating or updating resources in that resource group will not be possible.

If you have CI/CD (Continuous Integration and Continuous Deployment) set up to deploy or change your infrastructure or services with code, it can be easy to reach this limit. Azure will attempt to do this automatically when reaching your limit. Still, you may want to pre-empt any problems if you make many deployments and the system hasn't had time to prune automatically, or this is disabed.

This came up in conversations on Microsoft Q&A, so I thought I would dig into it and put together a possible option.

To avoid exceeding the deployment limit, it may be necessary to clean up old deployments.

This can be done by using a script to remove deployments that are no longer needed.

So let's build an Azure DevOps pipeline that runs weekly to connect to our Microsoft Azure environment and clean up historical deployments.

Microsoft Azure Deployment History Cleanup with Azure DevOps

For this article, I will assume you have an Azure DevOps repository setup and the permissions (Owner) to make the necessary privileged actions to the Microsoft Azure environment to do the design.

Note: Scripts and pipeline are "here".

Deploy and Configure

Create Service Prinicipal
  1. Navigate to the Microsoft Azure Portal
  2. Click on Microsoft Entra ID
  3. Click on App Registrations
  4. Click on: + New Registration
  5. Enter the following information:
    • Name (i.e. SPN.AzDeploymentCleanup)
  6. Click Register
  7. Copy the following for later when we add the SPN to Azure DevOps.
    • Application (client) ID
    • Directory (tenant ID)
  8. Click on Certificates & Secrets
  9. Press + New Client Secret
  10. Enter a relevant description and expiry date and click Add
  11. Copy the value of the new secret (this is essentially your password), and you won't be able to see the matter again.
Create Custom Role & Assign permissions

Now that your service principal has been created, it is time to assign permissions because this script targets all subscriptions under a management group; we are going to set the permissions to that management group so that it flows to all subscriptions underneath it - and in the view of least privileged we will create a Custom Role to apply to our Service Principal.

Create Custom Role

For the deployment history to be completed, we will need the following permissions:

  • Microsoft.Resources/deployments/delete
  • Microsoft.Resources/subscriptions/resourceGroups/read
  • Microsoft.Management/managementGroups/read
  • Microsoft.Resources/subscriptions/read
  • Microsoft.Management/managementGroups/descendants/read
  • Microsoft.Management/managementGroups/subscriptions/read
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/operations/read
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/read
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/write
  • Microsoft.Resources/deployments/read
  1. Navigate to the Microsoft Azure Portal
  2. In the search bar above, type in and navigate to Management Groups
  3. Click a management group, click on Access Control (IAM)
  4. Click + Add
  5. Click Add Custom Role
  6. Type in a role name (an example is: AzDeploymentHistoryCleanup)
  7. Check Start from Scratch and next click
  8. Click + Add permissions and the permissions above (you can search for them). Feel free to import the role from a JSON file "here".
  9. Click Next
  10. Add Assignable Scopes (this is the scope you can use to assign a role to - this won't give it to the Service Principal; it will only open it up so we can post it). Make sure you set it at the management group level you are targetting.
  11. Click Review + Create
  12. Click Create
Assign Permissions

Now that the custom role has been created, it is time to assign it to the service principal we made earlier.

  1. Navigate to the Microsoft Azure Portal
  2. In the search bar above, type in and navigate to Management Groups
  3. Click on the management group you want to manage and click on Access Control (IAM)
  4. Click Add
  5. Click Add Role Assignment
  6. Select your custom role (you can toggle the type column, so CustomRoles are first in the list)
  7. Microsoft Azure - add role assignments
  8. Click Members
  9. Make sure 'User, group or service principal' is selected and click + Select Members
  10. Microsoft Azure - Add Roles.
  11. Select your Service Principal created earlier (i.e. SPN.AzDeploymentCleanup)
  12. Click Select
  13. Click Review + assign to assign the role.

Note: Copy the Management Group ID and name, as we will need the information, along with the Service Principal and tenant IDs from earlier, in the next step of setting up Azure DevOps.

Configure Azure DevOps Service Endpoint

Now that the Service Principal and permissions have been assigned in Azure, it's time to create the service connection endpoint that will allow Azure DevOps to connect to Azure.

  1. Navigate to your Azure DevOps organisation.
  2. Create a Project, if you haven't already
  3. Click on Project Settings
  4. Navigate to Service Connection
  5. Click on New service connection
  6. Select Azure Resource Manager
  7. Click Next
  8. Select Service principal (manual)
  9. Click Next
  10. For the scope, choose Group Management
  11. Enter the Management Group ID, the Management Group Name
  12. Time to enter in the Service Principal details copied earlier, for the Service Principal Id paste in the Application ID.
  13. The Service Principal key, enter the secret client value and select the Tenant ID
  14. Click Verify - to verify the connectivity to the azure platform from Azure DevOps
  15. Select Grant access permission to all pipelines and click Verify and save
Configure script and pipeline

Now that we have our:

  • Azure Service Principal
  • Custom role and assignment
  • Service connection

We now need to import the script and pipeline.

If you haven't already done - create a Repo for the AzHistoryCleanup writing.

You can clone (or copy) the files in the AzDeploymentCleanup Repo to your own.

First, we need to copy the name of the Service Principal.

  1. Click Project settings
  2. Click Service Connections
  3. Click on your Service Connection and copy the name (i.e. SC.AzDeploymentCleanup)
  4. Azure DevOps Service Principal
  5. Navigate back to your Repo, and click on AzDeploymentCleanup.yml (this will become your pipeline)
  6. Click Edit
  7. Edit AzDeploymentCleanup YML
  8. Update the variable for ConnectedServiceNameARM to the name of your service connection
  9. Here you can also edit the Script Arguments - for example, in my demo, I am targeting the ManagementGroup named: mg-landing zones and keeping the latest five deployments.
  10. By default, I also have a cron job to schedule this pipeline at 6 AM UTC every Sunday, and you can remove or edit this.
  11. Once your changes are made, click Commit.
  12. Now that your pipeline has been updated, its time to create it - click on Pipelines.
  13. Click New Pipeline
  14. Select Azure Repos Git (YAML)
  15. Select your Repo
  16. Select Azure DevOps repo
  17. Select Existing Azure Pipelines YAML file
  18. Select YAML
  19. Select your Pipeline YAML file and click Continue
  20. Click Save to create the pipeline
  21. Now it's time to run the pipeline! Click Run pipeline
  22. Azure DevOps - Pipeline run
  23. If successful, your script will trigger and clean up the oldest deployment history! This can take several minutes to run if you have a lot of deployments.

Azure Deployments - Cleanup - Comparison 1

Azure Deployments - Cleanup - Comparison 2