Azure Pipeline auth broken since CLI 3.104.0: use a dedicated service principal

What was reported: Issue pulumi/pulumi#15786 (closed, 31 comments): Since the CLI version 3.104.0 (3.103.1 works fine), we are unable to run in our Azure pipeline. Error: secrets (code=Unknown): DefaultAzureCredential authentication failed. GET the cloud provider's instance metadata endpoint returned 400 Bad Request.

What works: If pulumi fails in Azure Pipelines after upgrading past CLI 3.103.1, with DefaultAzureCredential errors against the instance metadata endpoint (the cloud provider's instance metadata endpoint), upstream go-cloud changed the credential behavior and will not revert it. The fix that worked in the thread: stop using the Azure DevOps ServiceConnection for Pulumi auth and create a dedicated service principal for Pulumi instead.

In the GitHub issue, the reporter asked: Azure Pipeline auth broken since CLI 3.104.0: use a dedicated service principal. The working answer that came out of the thread: If pulumi fails in Azure Pipelines after upgrading past CLI 3.103.1, with DefaultAzureCredential errors against the instance metadata endpoint (the cloud provider's instance metadata endpoint), upstream go-cloud changed the credential behavior and will not revert it. The fix that worked in the thread: stop using the Azure DevOps ServiceConnection for Pulumi auth and create a dedicated service principal for Pulumi instead. Source: https://github.com/pulumi/pulumi/issues/15786