Apps on Azure Blog

5 MIN READ

Azure Kubernetes Fleet Manager Demo with Terraform Code

SaswatMohanty

Microsoft

Apr 25, 2025

Introduction

Azure Kubernetes Fleet Manager (Fleet Manager) simplifies the at-scale management of multiple Azure Kubernetes Service (AKS) clusters by treating them as a coordinated “fleet.” One Fleet Manager hub can manage up to 100 AKS clusters in a single Azure AD tenant and region scope, so you can register, organize, and operate a large number of clusters from a single control plane. In this walkthrough, we’ll explore:

The key benefits and considerations of using Fleet Manager
A real-world e-commerce use case
How to deploy a Fleet Manager hub, AKS clusters, and Azure Front Door with Terraform
How everything looks and works in the Azure portal
Along the way, you’ll see screenshots from my demo environment to illustrate each feature.

Why Use Fleet Manager?

Managing dozens or even hundreds of AKS clusters individually quickly become unmanageable. Fleet Manager introduces:

Centralized control plane: Register AKS clusters across subscriptions/regions under one fleet.
Orchestrated upgrades: Define update runs, stages, and groups (ring-based rollouts).
Resource propagation: Declaratively push Kubernetes objects (Namespaces, RBAC, ConfigMaps) from hub → members.
Cross-cluster L4 load balancing (preview): Distribute TCP/UDP traffic across clusters for high availability.
Auto-upgrade profiles: Automatically keep clusters up to date with minimal manual effort.

Portal Walkthrough: Exploring Your Fleet

Once your Fleet Manager hub and member clusters are up, here’s how it looks in the Azure portal.

Member Clusters

The Member clusters blade shows all onboarded clusters, their membership status, update group assignment, and Kubernetes version.

Figure: Four clusters (two dev, two prod) successfully joined to the fleet, all running version 1.32.3.

Multi-Cluster Update

Under multi-cluster update, you can manage both Auto-upgrade profiles and Strategies.

Auto-upgrade profiles let you enable continuous updates by channel (e.g., Stable) and node image:

Strategies define how clusters are grouped and staged during an update run:

Figure: We’ve created development-auto-upgrade and production-auto-upgrade profiles, along with matching strategies.

Fleet Overview

Back on the hub’s Overview blade, you get at-a-glance insights: total member clusters, Kubernetes versions spread, and node image versions.

Figure: The hub reports 4 member clusters (all on 1.32.3), and the node pools all share the same image version.

Azure Front Door Origin Groups

To demonstrate multi-cluster traffic routing, our Terraform deploy includes an Azure Front Door profile with two origin groups (dev & prod). Here’s the Origin groups blade:

And the Front Door Overview, showing the endpoint hostname and associated origin groups:

Figure: Front Door is configured to route /dev/* to the dev clusters and /prod/* to the prod clusters via these origin groups.

Benefits & Considerations

Benefits

One pane of glass for up to 100 AKS clusters.
Ring-based upgrades minimize risk with staged rollouts.
Declarative propagation of configs and policies.
Global traffic distribution at TCP/UDP (L4) level.
Extensible roadmap: Arc support, region failover, Terraform enhancements.

Considerations

Hub is management-only: No user workloads on the hub.
100-cluster limit per fleet.
Regional scope: Hub deployed in one region, though it can manage clusters anywhere.
Private hub networking: Private AKS hub requires VNet/jumpbox connectivity.
Preview features: Multi-cluster L4 load balancing and Terraform support for update groups are still in preview.

Real-World Use Case: Global E-Commerce

A multinational retailer runs dev & prod AKS clusters in North America and Europe. They needed:

Consistent feature flags & RBAC across clusters
Safe, staged upgrades (dev → prod)
High-availability checkout traffic routed to healthy clusters

Solution with Fleet Manager:

Onboard all four clusters into one fleet.
Propagate feature-toggle ConfigMaps and RBAC from hub to members.
Define update strategies for dev and prod, then run upgrades via CLI or portal.
Use Azure Front Door for global routing, failing over between regions.

They cut upgrade windows by 60%, eliminated manual sync tasks, and boosted resilience.

Reference Architecture for Demo:

Deployment with Terraform

All of the above is automated in the aks-fleet-manager GitHub repo. Here’s a quick start:

1. Clone repo

git clone https://github.com/saswatmohanty01/aks-fleet-manager.git
cd aks-fleet-manager/terraform

2. Install CLI tools

chmod +x ../scripts/setup-cli.sh
../scripts/setup-cli.sh

3. Authenticate & select subscription

az login
az account set -s <subscription-id>

4. Initialize Terraform

terraform init

5. Configure variables (terraform.tfvars):

primary_region = "eastus"
secondary_region = "westeurope"
resource_prefix = "mycompany"
dev_node_count = 2
prod_node_count = 3

6. Plan & apply

terraform plan -out=tfplan
terraform apply tfplan

7. Create update groups (post-deploy)

cd ../scripts
chmod +x create-update-groups.sh
./create-update-groups.sh

Once complete (about 10–15 minutes), you’ll have:

4 AKS clusters (dev/prod in two regions)
A Fleet Manager hub with 4 member clusters
Auto-upgrade profiles and strategies
An Azure Front Door endpoint routing /dev/ and /prod/

Known Issue. Manual Step in Azure Front Door

Refer GitHub: README.md

Get the terraform output for all four AKS clusters service endpoint IP addresses. You can get it from step 3 using kubectl get svc for all four clusters.
There is a bug, which does not allow to update the service IP addresses for each AKS cluster in Azure Frontdoor->Origin Groups
Manually update the IP addresses for Dev and Prod AKS cluster service IP addresses. Go to Azure portal->Azure Front door->Settings->Origin Groups->dev-origin-group
Manually update the IP addresses for Dev and Prod AKS cluster service IP addresses. Go to Azure portal->Azure Front door->Settings->Origin Groups->prod-origin-group

VS Code Experience

Follow the VsCode Deployment Guide from GitHub Repo

Conclusion & Next Steps

Azure Kubernetes Fleet Manager reduces the pain of managing multi-cluster AKS environments by centralizing control, orchestrating upgrades, and enabling global traffic patterns. To go further:

Experiment with auto-upgrade profiles to automate patch deployments.
Integrate Fleet operations into CI/CD pipelines with az fleet CLI or Terraform (as features mature).
Explore GitOps workflows (Flux/Argo CD) for multi-cluster app deployments.

Fleet Manager is evolving rapidly—keep an eye on the preview features and Terraform provider updates. With Fleet Manager, managing up to 100 AKS clusters doesn’t have to be a headache. Give it a try and share your experiences!

References