What will happen when upgrading a "public" hub that has been manually converted to "private" after initial deployment?

[philipstreet]

My company are considering deploying a FinOps Hub and prefer the "private access" option. Like most enterprises we have tight security controls on what is deployed, how and by "who". Ideally we would create all the required networking resources ourselves and then use the toolkit to deploy into those, but that is not an option currently; the toolkit deploys the vnet, private dns zones etc.

I want to ask what would happen if we did the following:

1. Deploy a virtual network peered into our hub, plus any required Private DNS Zones.
2. Deploy a FinOps Hub in public access mode with version 0.x of the toolkit, from an Azure DevOps pipeline using self-hosted build agents.
3. Use IaC (we use Terraform) to convert the Hub into private access mode; create Private Endpoints etc.
4. Upgrade the FinOps Hub with version 0.y of the toolkit.

Will the upgrade (#4) remove any of the changes made after the initial deployment (#3)?

Thanks.

[MSBrett]

Upgrades will remove any changes you make and reset the solution to using public connections.

Assuming you're using something like landing zones where policy prevents creation of network resources in prod your release management process should:

1. Create a new sub and put in the sandbox/staging management group where policy won't block deployments.
2. Deploy the solution.
3. Move the subscription into one of your prod management groups.
4. Let policy audit and the apply exceptions for the DNS zones, etc.
5. Connect the FTK to your internal network via peering or private endpoints for ADLS and ADX.
6. Create the required DNS entries in your centralized DNS for the FTK data lake and kusto cluster.

Private endpoints are simpler in many ways because you don't need to set up routing. Your network team can control the private endpoints and associated DNS records and the FTK can maintain it's own network and DNS without taking a dependency on your infrastructure. It's unlikely that we'll support BYO vNet and BYO DNS any time soon because of the number of support tickets misconfigured networking generates.

[philipstreet]

Thanks for the suggestion @MSBrett.

Doesn't the toolkit use Bicep to deploy the resources, which is idempotent so it shouldn't remove the additions I've made, such as private endpoints? I cannot see the use of "--mode Complete" in the scripts, which could remove changes.

For example, if I were to deploy the private access version and then add the following:

• Peer the vnet to our hub and add a RT on the subnet, OR
• Create Private Endpoint(s) into a separate existing vnet that is peered into our Hub, and add the required DNS A records in our central Private DNS Zones

Would your toolkit scripts really remove the above changes? Thanks.

[MSBrett]

No, the toolkit won't remove those additional resources. But, in public mode Azure Data Factory is configured to use a public integration runtime for the pipelines, and that public IR needs access to the public endpoints of ADLS and ADX. In private mode it creates a managed vNet + IR and managed private endpoints to get to the data and then adjusts a bunch of concurrency settings to compensate for the lower performance. Each time you upgrade FinOps Toolkit in public mode the FTK it will reset the firewall on storage and kusto to public, reconfigure ADF to use the public runtime.

[philipstreet]

Ahhh, ok that makes sense. Does the documentation mention that's what happens when you select private access? I have found the docs lacking in detail about resources are deployed and why.

[flanakin]

@philipstreet here's the private networking doc: https://learn.microsoft.com/cloud-computing/finops/toolkit/hubs/private-networking. Create an issue if you run into anything that is missing or confusing and we'll make sure to get it updated.

[philipstreet]

Ah yes, you're right. I really need to read ALL the detail, rather than skimming over it. 🤦 Thanks.