Azure Optimizer and Finops Toolkit

[jamelachahbar]

As cloud adoption accelerates, managing and optimizing cloud costs has become a critical challenge for organizations. I am currently developing a tool to handle one aspect of these challenges, reducing waste.
I would like to introduce the idea of an Azure Cost Optimizer Tool, that I am currently developing and testing.

What is it?

Azure Optimizer is a Python-based tool used to optimize Azure resource costs by applying various policies to resources across multiple subscriptions. It identifies resources that meet specific criteria and applies actions such as scaling, stopping, or deleting them to reduce costs.
The main goal is to allow the FinOps team or Engineering teams to clean up waste semi-automatically, leveraging this tool and already existing processes.

In this discussion, I would like to explore the concept of this Azure Optimizer Tool and discuss how it could potentially be leveraged and integrate with FinOps hubs and be part of the toolkit to enhance cloud financial management.

Its current Features

• Apply Policies: Apply predefined policies to resources, such as stopping unused VMs, deleting unattached disks, scaling SQL databases, etc.

  • Stop unused virtual machines based on CPU metric.
  • Delete unattached disks.
  • Delete all resources in a resource group based on tags.
  • Delete idle Application Gateway.
  • Delete unattached Public IP Addresses.
  • Delete unattached Network Interfaces.
  • SQL DTU Scaling: Dynamically scale Azure SQL databases based on defined policies and schedules.
  • Storage Account SKU change: Update Storage Account SKU to Standard_LRS for specific SKUs and tags
  • filtering based on tags: excluding resources based on tags is also supported
  • Can be combined with Azure policy or any other existing tool

• lightweight: no deployment to Azure needed and can run on github using github runners and workflows. You will need a service principle with the right permissions and github pat token to run the workflow.

• Possibility to make it enterprise grade and run it using Azure Functions or Containers

• Analyze cost data for trends and anomalies.

• Generate summary reports.

• Multi-Subscription Support: Process multiple subscriptions within a tenant.

[helderpinto]

Thanks for sharing this idea, @jamelachahbar. A small part of what you're suggesting is supported by the Azure Optimization Engine remediation runbooks, namely automating VM right-sizing, orphaned disks deletion/downgrade or downgrading disks of deallocated VMs. As we are planning the next version of the engine (see discussion), it would be great if we could share synergies.

[jamelachahbar]

That sounds great. I'll explore this further and it could be valuable to have a discussion and exchange ideas. I'll proceed with deploying your solution to get a better understanding of how it all functions.

[cheenashe]

While deploying the tool make sure you cover these following points/parameters as well: Azure Cost Optimizer Tool, consider focusing on the following key areas:

1. Integration with Existing Systems:
   Current Infrastructure: Discuss how the tool will integrate with the existing cloud infrastructure management and financial management systems. Address any potential challenges or additional requirements for seamless integration.
   FinOps Hub Integration: Explore how the tool can be incorporated into your FinOps hub and align with current processes and workflows. Highlight any APIs or interfaces needed for this integration.
2. Customization and Policy Configuration:
   Policy Customization: Detail how policies can be customized to fit specific organizational needs. Discuss any built-in flexibility or configurations available for different use cases.
   Tagging and Resource Management: Explore how tagging strategies can be applied for efficient resource management and cost optimization. Discuss how tags can be leveraged in policy configurations.
3. Security and Compliance:
   Data Security: Address how the tool ensures data security, particularly when dealing with sensitive financial and resource data.
   Compliance Requirements: Discuss how the tool aligns with compliance and regulatory requirements, including data handling and audit trails.
4. Performance and Scalability:
   Performance Metrics: Evaluate the tool’s performance in various scenarios and discuss any benchmarking or performance metrics available.
   Scalability: Explore how the tool scales with increasing resource volumes and multiple subscriptions. Discuss any potential limitations and how they can be addressed.

[jamelachahbar]

Thanks, @cheenashe appreciate your feedback and contributing ideas