Skip to content

Pulumi Cloud OIDC from Azure DevOps

How Azure DevOps pipelines authenticate to Pulumi Cloud and to Azure without a long-lived PULUMI_ACCESS_TOKEN or client secret anywhere, and how the whole trust is created and managed as code from this repo.

Status: proven and self-managed

The handshake was proven end-to-end by a standalone CI-persona smoke-test pipeline (build 180769: an Azure DevOps pipeline authenticates to Pulumi Cloud and lists ESC environments with no Pulumi access token anywhere), then hardened into a least-privilege, two-persona model (builds 180845–181029, see Narrowing the identity).

Both sides of the trust are now created by infra/pulumi-config itself — the Entra app registrations, the Azure DevOps service connections, the Pulumi Cloud teams/roles, and the OIDC issuer's auth policies. Nothing here depends on a Terraform PR in another repo.

Why

Storing a Pulumi access token — or an Azure client secret — in a variable group means a long-lived credential that never expires on its own, is copyable, and has to be rotated by hand. Azure DevOps already mints short-lived OIDC id_tokens for Workload Identity Federation service connections; both Entra ID and Pulumi Cloud can be registered as relying parties for them. Every pipeline run gets a fresh, scoped token instead of a standing secret.

How it's built

Everything is created by one Pulumi program, infra/pulumi-config (Pulumi project cloud-foundations, stack pulumi-config). There is no external repo in the loop and no manual Azure DevOps or Entra portal step for the steady-state path — only the original, long-since-completed registration of the Pulumi Cloud OIDC issuer (see The issuer).

For each Azure environment in Pulumi.pulumi-config.yaml (development-sandbox, shared-services, platformdev, test, qa, uat, prod), the EnvironmentIdentity component (infra/pulumi-config/Environments/EnvironmentIdentity.cs) creates:

  • an Entra app registration iac-<env> and its service principal, granted Owner on that environment's subscription
  • a Pulumi ESC environment azure/<env> that logs the app in via OIDC (fn::open::azure-login) — no client secret, no PULUMI_ACCESS_TOKEN
  • a federated credential on the app trusting Pulumi ESC's own OIDC issuer, scoped to that one environment (pulumi:environments:org:saif-corp:env:azure/<env>) — this is what lets the ESC environment mint Azure tokens for the app in the first place
  • one Azure DevOps service connection per project, named iac-<env>, in every project listed under cloud-foundations:azureDevOps.projects — an AzureRM connection using WorkloadIdentityFederation, backed by the same app
  • an all-pipelines azuredevops.PipelineAuthorization on each of those connections, so no pipeline needs the manual, click-through authorization Azure DevOps otherwise demands the first time it references the connection
  • a matching federated credential on the app for each of those connections, with its issuer/subject read back from the connection resource's own exported workload_identity_federation_issuer / _subject attributes rather than hand-typed, so they cannot drift from what Azure DevOps actually presents

Those resources are grouped into three child components, split by blast radius rather than by resource type:

Component Holds
WorkloadIdentity the Entra app and principal, Owner on the subscription, two directory roles, two Microsoft Graph permissions
EscEnvironmentAccess the credential trusting Pulumi Cloud, the azure/<env> ESC environment, and the two team grants
AzureDevOpsAccess one service connection per Azure DevOps project, each with the credential that trusts it and an all-pipelines authorization

The two access components are independent ways of obtaining a token for the same identity, so revoking one leaves the other intact.

Tenant and organization singletons live under Organization/ instead, because they are shared by every environment and so belong to none of them:

Unit Holds Shape
MachineTeam ×2 a Pulumi Cloud team, its custom organization role, and the assignment binding them — plus any standing grants with no local resource to sit beside component
AzureBaseEnvironment the shared azure/base ESC environment and both personas' open grants on it component
AdminAccess the iac-platform-root service connection and the credential that trusts it component
DirectoryRoles the two tenant-singleton directory role activations loose resources
OidcTrust the OIDC issuer and its two auth policies loose resource

The last two are deliberately not components. The rule this repo applies is that a component earns its node when its resources share an invariant — remove one and the rest are broken or meaningless. A Team without its TeamRoleAssignment has no organization access level, so every pipeline token fails at exchange rather than at deploy; that is an invariant. DirectoryRoles' two activations need nothing from each other, and OidcTrust declares a single resource whose policies are inline properties, so neither has one. Grouping them anyway would be filing by "declared in the same place", which is what a component is not for.

Note also that grouping by provider would be the wrong axis here, however tempting the Organization/ folder's provider mix makes it look. This program's whole job is federation between providers: AzureDevOpsAccess reads a service connection's exported issuer and subject straight into an Entra credential, and EscEnvironmentAccess derives a credential's subject from an ESC environment's path. Splitting either by provider severs the dependency the pair exists for. The components are named for what trust they establish — which is why a future OktaAccess would slot in beside EscEnvironmentAccess as a fourth child of EnvironmentIdentity, not into an Okta/ folder.

Separately, the AdminAccess component (infra/pulumi-config/Organization/AdminAccess.cs) creates one more connection, iac-platform-root — the identity that deploys infra/pulumi-config itself. It reuses the pre-existing iac-platform-root app (looked up by display name, not created here) rather than minting a new one — which is why it is named for the access path rather than the identity, exactly parallel to AzureDevOpsAccess.

\"platform\" means the platform, not the Platform project

The name comes from cloud-foundations:admin.identityName and refers to the platform as a whole — it is the root IaC identity for the estate. It is not derived from the Azure DevOps project that happens to share the name. Which project hosts the connection is decided independently, by the pulumi-config-admin tag in azureDevOps.projects (today: Platform).

Those two were briefly conflated: the name used to be built as iac-{adoProjectName}-root. That encoded a dependency that does not exist — the Entra app is manually maintained and always called iac-platform-root — so moving the admin tag to another project would have renamed the lookup and failed the deploy outright. Move the tag freely; the identity keeps its name.

flowchart LR
    subgraph "Per environment × per ADO project"
        ADO[ADO pipeline<br/>AzureCLI@2]
        SC["Service connection<br/>iac-&lt;env&gt;"]
    end
    ADMIN["Service connection<br/>iac-platform-root<br/>(Platform project only)"]
    EID[Entra ID<br/>login.microsoftonline.com]
    PC[Pulumi Cloud<br/>org saif-corp]

    ADO -->|requests id_token| SC
    SC -->|federated cred| EID
    EID -->|Entra-issued id_token| ADO
    ADO -->|az login / az account set| ADO
    ADO -->|pulumi login --oidc-token| PC
    PC -->|short-lived Pulumi token| ADO
    ADMIN -->|deploys pulumi-config itself| PC

AzureCLI@2 also performs a real az login / az account set before running your script. If the backing service principal cannot see the connection's subscription, the task fails there — before any Pulumi command runs. infra/shared-services/permissions.tf documents this constraint for other pipelines.

The mechanism is addSpnToEnvironment: true, which exposes the OIDC id_token to the inline script as $env:idToken. That token is then handed to pulumi login.

The config file is the source of truth

infra/pulumi-config/Pulumi.pulumi-config.yaml drives the whole fan-out. Four stack-config sections:

config:
  cloud-foundations:admin:
    identityName: iac-platform-root
    environmentName: shared-services
  cloud-foundations:azure:
    tenantId: <Entra tenant guid>
    environments:
      - name: development-sandbox
        subscriptionId: <guid>
      - name: shared-services
        subscriptionId: <guid>
        displayName: Shared Services
      - name: platformdev
        subscriptionId: <guid>
        displayName: Platform Dev
      # test / qa / uat / prod likewise
  cloud-foundations:azureDevOps:
    organizationId: <Azure DevOps org guid>
    organizationName: <Azure DevOps org display name>
    projects:
      # Every project lists azure-devops-ci, which is what gets it the per-environment
      # iac-<env> connections. Exactly one project also lists pulumi-config-admin, which
      # is what gets it the one-off iac-platform-root connection.
      - name: Platform
        pulumiTeams: [azure-devops-ci, pulumi-config-admin]
      - name: Architecture
        pulumiTeams: [azure-devops-ci]
      # Case-Management / Claims / Corporate / Customer / Documents / Policy /
      # Regulatory / SAIF likewise
  cloud-foundations:pulumiOidc:
    displayName: pulumi-esc-oidc
    issuer: https://api.pulumi.com/oidc
    audience: azure:saif-corp
    subjectPrefix: "pulumi:environments:org:"

Adding an environment (a new subscription) or an Azure DevOps project is a YAML edit and a pulumi up — no external PR, no manual portal step, no new hardcoded array. Every environment gets the identical treatment above — including development-sandbox (ad hoc manual testing) and shared-services (shared resources other pipelines reach into) — since all of them create service connections and none is special-cased.

Projects are cheap, but not unlimited — 9 slots remain

Entra caps an application at 20 federated identity credentials, and the iac-<env> applications are per environment, not per project. So each project added to projects: costs one credential on every application at once, not one in total.

Today that is 10 projects plus the Pulumi ESC credential — 11 of 20 on each of the seven applications. Nine more projects fit. The tenth fails during pulumi up, on whichever environment Pulumi happens to reach first, so the error will name an arbitrary environment rather than the project you just added.

Should that ceiling ever bind, the fix is to stop federating every project against the same per-environment application — either an application per project/environment pair, or a single shared connection projects are granted access to.

Only one ESC import remains: bootstrap/azure

Earlier revisions of this stack imported azure/<env> back into itself to read {env}:subscriptionId — circular, since this stack is what creates those environments. Subscription IDs now live as plain stack config instead (above), and this stack is their source of truth, not a consumer of it. The stack also no longer imports platform/all (which existed only to pull in platform/pulumi's long-lived PULUMI_ACCESS_TOKEN) — Pulumi Cloud auth for deploying this very stack now comes from the iac-platform-root service connection's OIDC exchange, via a federated credential scoped to pulumi:environments:org:saif-corp:env:bootstrap/azure.

This environment is manually maintained and deliberately not managed by this stack — it is what the stack authenticates with, so it cannot be something the stack creates. It was previously platform/iac-platform-root, then bootstrap/pulumi-config. The bootstrap/ project namespace is reserved for exactly one category: credentials a human enters by hand because no stack can create them. Third-party vendor credentials belong here too, one environment per vendor and tier — bootstrap/okta-nonprod, bootstrap/okta-prod, bootstrap/dynatrace — so each keeps its own grant boundary while Pulumi Cloud's project grouping still presents them as a single place to look.

Naming environments after what the credential is for, not what consumes it, is what makes that namespace scan consistently: bootstrap/azure sits beside bootstrap/okta-prod and reads the same way. It also means the environment holds only credentials. Non-secret settings that merely happened to be convenient there belong in stack config instead — the Azure DevOps organization URL moved out for exactly this reason, and is now derived from cloud-foundations:azureDevOps.organizationName.

bootstrap/azure is the one asymmetric member of the namespace: its path is baked into a federated credential subject, so it cannot be renamed or deleted casually (see below). The vendor environments carry no such constraint.

Consume vendor credentials with implicit imports, not imports:

An explicit imports: entry merges the entire imported environment into the consumer's resolved output, so every value in bootstrap/okta-prod would be readable by anyone who can open the importing environment. An implicit import exposes only the value actually referenced:

# in a consuming environment — only clientSecret crosses the boundary
values:
  okta:
    clientSecret: ${environments.bootstrap.okta-prod.okta.clientSecret}

Never import vendor credentials into azure/base: AzureBaseEnvironment gives both personas an open grant on it, so a secret placed there is readable by every CI pipeline in every Azure DevOps project whether or not anything imports it.

Nothing imports azure/base today, so it propagates nothing on its own — the exposure above is the grants, not inheritance. Whether to delete it or wire the azure/<env> environments to genuinely import it is tracked in cloud-foundations#104.

Whether the reader additionally needs an open grant on the imported environment is not documented and has not been verified here. Until it is, assume it does not and rely on implicit imports to limit exposure. A granted/ungranted assertion pair is the natural way to prove it; the retired CI smoke test carried one and is archived under docs/_internal/ if you want the shape of it.

Renaming this environment is a two-step change, and cleanup is part of it

The environment path is load-bearing: it is half of the federated credential subject (pulumi:environments:org:saif-corp:env:<project>/<environment>) that lets ESC mint the iac-platform-root app's Azure tokens. Add the new subject as an additional federated credential on that app before switching the environment: import, and only remove the old one after a real pipeline run proves the new path. Switching the import first leaves the stack unable to authenticate — including unable to deploy the fix.

Verify the new credential in isolation before switching anything:

pulumi env open saif-corp/bootstrap/azure environmentVariables.ARM_CLIENT_ID

That forces the azure-login exchange to actually execute while printing only a non-secret client id. Expect a OIDC token validation failed with Azure error for the first minute or so — Entra takes a short while to propagate a new credential, so retry before concluding the subject is wrong.

Then delete the old environment and its now-orphaned federated credential. A credential whose environment no longer exists is not inert: its subject is just a string, so anyone able to create an ESC environment at that exact path can mint tokens for the app — which here is Owner on seven subscriptions. Two such orphans accumulated in this tenant from earlier renames (platform/iac-platform-root and platform/all, both 404) before anyone noticed. Audit with:

az ad app federated-credential list --id <appId> --query "[].{name:name,subject:subject}" -o table
# then, for each pulumi:environments subject, confirm the environment still exists:
pulumi env get saif-corp/<project>/<environment> --definition

Entra also caps an application at 20 federated credentials, so orphans consume a finite budget as well as widening the trust surface.

Deleting a team or an ESC environment in the UI silently orphans its grants

TeamEnvironmentPermission, TeamStackPermission and TeamRoleAssignment are children of the team and environment they bind. Deleting either side in the Pulumi Cloud UI removes the grants with it — but Pulumi's state file still records them as live, and pulumi preview compares desired state against state, not against the server. The stack therefore reports itself clean while the CI identity has silently lost access.

This has already happened once here: 13 environment grants, one role assignment and one stack permission were destroyed out of band and went unnoticed across four consecutive clean previews. The only thing that surfaces it is pulumi refresh, which reconciles state against the provider:

pulumi refresh --stack pulumi-config   # detects the vanished grants
pulumi up --stack pulumi-config        # recreates them

Verify the repair against the API rather than trusting the resulting state — that is the failure mode being diagnosed:

curl -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
  https://api.pulumi.com/api/orgs/saif-corp/teams/azure-devops-ci

Prefer deleting environments and teams through this stack. When something must be removed in the UI, run a refresh immediately afterwards.

Reference values

Item Value
Azure DevOps organization SAIFCorporation
Azure DevOps org GUID 7f80b586-0eee-43fa-8743-8d4025b72800
Entra tenant a86cb8ed-369b-4df5-ace5-43811f6e08cf
Issuer (iss) https://login.microsoftonline.com/a86cb8ed-369b-4df5-ace5-43811f6e08cf/v2.0
Audience (aud) fb60f99c-7a34-4190-8149-302f77469936 — see below
Service connections iac-<env>, one per environment × Azure DevOps project (config-driven, see above); iac-platform-root in Platform only
App registrations iac-<env>, one per environment; iac-platform-root (admin)
Pulumi organization saif-corp

The registered audience and the emitted aud claim are different strings

Every federated credential reports its audience as api://AzureADTokenExchange:

$ az ad app federated-credential list --id <app-id> --query "[].audiences"
api://AzureADTokenExchange

But the token actually issued carries the magic GUID form:

aud: fb60f99c-7a34-4190-8149-302f77469936

Confirmed in build 180769. Pulumi matches the claim in the token, not the value in the credential, so the policy rule must be the GUID. Using api://AzureADTokenExchange fails with an audience mismatch. OidcTrust.AadTokenExchangeAudience is set to the GUID for exactly this reason.

The subject is a canonical Entra identifier, not a readable path. Its grammar is:

/eid1/c/pub/t/{tenant}/a/{app}/sc/{adoOrg}/{serviceEndpointId}
Segment Encoding Value here Varies?
eid1 literal Entra subject scheme, v1 no
c/pub literal cloud = public no
t/{tenant} base64url GUID 7bhsqJs29U2s5UOBH24Izwa86cb8ed-369b-4df5-ace5-43811f6e08cf no
a/{app} base64url GUID rISbSSETf0KqFyZ8ppdXmA499b84ac-1321-427f-aa17-267ca6975798 no
sc literal subject kind = service connection no
{adoOrg} plain GUID 7f80b586-0eee-43fa-8743-8d4025b72800 no
{serviceEndpointId} plain GUID one per service connection yes

Note the mixed encoding: the tenant and app segments are base64url-encoded GUIDs, while the org and endpoint segments are plain dashed GUIDs.

{app} is Azure DevOps, not one of our app registrations

499b84ac-… is the well-known Azure DevOps first-party application ID — the same value you pass to az account get-access-token --resource. It identifies Azure DevOps as a whole, not any individual iac-<env> app.

Because this segment (and everything before it) never varies, the subject says nothing about which app backs a connection — only that a token came from some Azure DevOps Entra-federated service connection in this tenant, in this org, from this specific endpoint. That is exactly what the subject prefix OidcTrust composes pins, and it is why the CI policy's wildcard (see Choosing the subject rule) covers every iac-<env> connection this stack creates without listing any of them.

To decode a segment yourself:

$s='rISbSSETf0KqFyZ8ppdXmA'; $p=$s.Replace('-','+').Replace('_','/')
switch($p.Length % 4){ 2 {$p+='=='} 3 {$p+='='} }
[guid]::new([Convert]::FromBase64String($p)).Guid

Verify the Entra discovery document if something looks wrong (confirmed reachable):

curl -s https://login.microsoftonline.com/a86cb8ed-369b-4df5-ace5-43811f6e08cf/v2.0/.well-known/openid-configuration

Why iss and aud are not enough

A reference article this work started from suggested iss + aud "will most likely be sufficient", treating sub as optional. That does not hold here.

Every federated credential registered against this tenant carries the same audience — api://AzureADTokenExchange as registered, emitted as fb60f99c-7a34-4190-8149-302f77469936. It is the fixed audience for Entra workload identity federation, so as a rule it excludes nothing. iss narrows only to the Entra tenant — the whole tenant, not just Azure DevOps or just this Pulumi program's connections. So an iss + aud policy would accept any Entra workload identity in the tenant that can obtain this audience, and sub is the only rule doing real access control.

Both issuer forms exist in this tenant

Some older, unrelated app registrations still present vstoken.dev.azure.com/<orgId> tokens. Every connection created by this repo uses the Entra form above. Don't assume from a neighbouring connection which form applies — read the actual federated credential.

Choosing the subject rule

Pulumi policy claim values support wildcards: * matches zero or more characters, ? matches zero or one, and . matches exactly one character — not a literal dot. Don't read these rules as plain string equality.

The CI rule wildcards only the final segment:

/eid1/c/pub/t/7bhsqJs29U2s5UOBH24Izw/a/rISbSSETf0KqFyZ8ppdXmA/sc/7f80b586-…/*

That still pins the Entra tenant, that the token was issued via Azure DevOps rather than some other workload identity, that it came from the SAIFCorporation organization, and that the subject is a service connection. It is much tighter than omitting sub. What it does not pin is which service connection — so any pipeline in any of the projects listed under cloud-foundations:azureDevOps.projects, using any iac-<env> connection, can mint the CI token.

Why not pin the endpoint GUID?

Considered and rejected, for the same reason it was rejected before this stack owned the connections: Azure DevOps is the deployment mechanism for these environments, so a pipeline that can deploy through one iac-<env> connection can deploy through any of them in that project — pinning a GUID defends against nothing an attacker couldn't reach another way. It would also multiply the maintenance cost: one rule per environment × project, forever, growing every time a project or environment is added, versus the single wildcard rule today that covers new connections automatically.

The one place a GUID is pinned is the elevated (admin) policy — see Two personas, one issuer — because that one connection genuinely should not be reachable from anywhere else.

The tripwire

The wildcard's residual exposure is a hand-made Entra-WIF service connection somewhere in the tenant that isn't one this stack created. Such a connection would not automatically gain anything here, since the CI policy's teamName requires the pipeline to explicitly request azure-devops-ci — but it is worth periodically confirming that every Entra-WIF Azure DevOps connection in the tenant is one this stack (or a known predecessor) created.

Remember deny always beats allow regardless of rule order or specificity — a deny policy is the reliable way to carve out an exception if a specific connection is ever compromised.

Verifying the handshake

The test pipelines were retired

Two standalone pipelines proved this handshake and exercised the two personas — pulumi-oidc.yml (CI persona, definitions 5595 and 5597) and pulumi-config-admin.yml (elevated persona, definition 5608). Both lived under .azdo/experiments/ and have been deleted, since the shared v2/deployments/pulumi-pulumicloud/ service in SAIF/pipeline-templates supersedes them.

Their findings, caveats and full source are archived at docs/_internal/pulumi-oidc-experiment-pipelines.md in this repository. That page is excluded from this site, so read it in the repo.

Until the shared deployment service ships, nothing re-verifies this chain on a pull request. A break in infra/pulumi-config would not be caught automatically. To re-verify by hand, restore the archived admin pipeline to a branch and run it.

What the elevated pipeline established, and what any replacement should still cover: minting a pulumi-config-admin token through iac-platform-root and running pulumi preview on infra/pulumi-config itself. That single command exercises the whole chain — stack read, open on bootstrap/azure, ESC's federated credential minting Azure tokens, and the azure/azuread/azuredevops providers authenticating with them — so any broken link fails the run.

Why the pipeline asserted the token subject

Until build 181564 the issuer carried two rules allowing the admin team: one for iac-platform-root and one for the old externally-managed connection. A successful login therefore did not by itself prove the new connection worked — the old one may have satisfied the policy. The pipeline's expectedEndpointId parameter closed that hole by asserting the token's sub claim ends with the expected endpoint GUID, and failed loudly when it did not.

That assertion produced the evidence to retire the legacy rule. Carry it forward: it is also the tripwire that would catch the admin persona silently authenticating through some other connection.

Two behaviours worth carrying into any replacement:

  • Plain pulumi preview compares against the state file, so grants deleted in the Pulumi Cloud UI stay invisible to it. Only --refresh reads the server, and it is the only way to catch that class of drift.
  • pulumi up has never been run from a pipeline here. Every green run is preview only, so the apply path is unproven.

Generated ESC YAML must be newline-pinned

Running this stack from a Linux agent for the first time (build 181555) previewed 8 to update where the same commit previewed 253 unchanged on a Windows workstation.

YamlDotNet writes line breaks through TextWriter.WriteLine(), which emits Environment.NewLine — CRLF on Windows, LF on Linux. That rendered YAML is the pulumiservice.Environment resource's input, so every azure/* environment's content was platform-dependent: a local pulumi up and a build agent would each rewrite all eight back to their own line endings, indefinitely, churning the very environments the OIDC login path depends on.

All ESC YAML is therefore serialized through EscYaml, which pins .WithNewLine("\n"). Use it for any new call site rather than constructing a SerializerBuilder inline.

Don't measure line endings through the PowerShell pipeline

pulumi env get … --definition | Out-String reports CRLF whatever is actually stored, because PowerShell normalises line endings. Read raw bytes instead:

cmd /c "pulumi env get saif-corp/azure/base --definition > $env:TEMP\d.txt 2>nul"
$b = [IO.File]::ReadAllBytes("$env:TEMP\d.txt")
"CR=$(($b | ? { $_ -eq 13 }).Count)  LF=$(($b | ? { $_ -eq 10 }).Count)"

The reliable signal is simpler: if a preview is clean on one OS and dirty on another, something in the program is platform-dependent.

What a successful run looks like

--- token claims (configure Pulumi Cloud from these) ---
iss: ***
aud: fb60f99c-7a34-4190-8149-302f77469936
sub: ***
-------------------------------------------------------
Logging in to Pulumi Cloud as org 'saif-corp'
Logged in to pulumi.com as service-account:991b80a7-…
--- pulumi whoami ---
User:         service-account:991b80a7-…
Organizations: saif-corp
Token type:   team: azure-devops-ci
Token name:   Azure DevOps – SAIFCo-1785448625627678

The identity is an ephemeral Pulumi service account, minted per run and named after the OIDC issuer. Nothing is stored.

Environment names are visible to any authorized token

pulumi env ls returns every ESC environment in the organization regardless of how narrowly the token is scoped — TeamEnvironmentPermission gates opening, not listing. Contents stay protected; names and structure do not. See Listing is not permission-gated.

Troubleshooting

Start from the printed claims, but expect masking

The smoke test decodes and prints iss, aud and sub. Azure DevOps redacts iss and sub as *** because they derive from the service connection; aud comes through. For the masked values, read Graph or the connection's federation details panel.

Symptom Likely cause
unsupported protocol scheme "" The backend URL was omitted. Use pulumi login https://api.pulumi.com --oidc-token ..., not pulumi login --oidc-token ....
No OIDC token was returned (our guard) The connection is not using Workload Identity Federation, or addSpnToEnvironment: true is missing.
idToken is not a JWT (our guard) The task exposed something other than a JWT. Check the service connection's authentication scheme.
Invalid issuer / issuer is not registered: … for org saif-corp The issuer registered in Pulumi Cloud does not exactly match the iss claim — it should be https://login.microsoftonline.com/a86cb8ed-…/v2.0, not a vstoken.dev.azure.com URL. Issuer URLs are immutable, so this usually means re-registering.
Audience mismatch The aud rule is almost certainly set to api://AzureADTokenExchange. The emitted claim is the GUID fb60f99c-7a34-4190-8149-302f77469936 — use that.
Access denied / no matching policy Either the sub rule does not match the organization GUID 7f80b586-…, the requested --oidc-team doesn't match any policy's teamName, or a deny policy is winning.
OIDC token exchange failed: access_denied when requesting the admin team Expected from anything other than the iac-platform-root connection. The elevated policy is pinned to that one endpoint GUID — see Two personas, one issuer.
Requested token type unavailable The Pulumi organization's edition does not grant that token type. saif-corp is Business Critical — personal, organization, team and deployment-runner are all available.
Fails at az account set, before any Pulumi output An Entra problem, not a Pulumi one. The service principal cannot see the connection's subscription.

Narrowing the identity

A broad organization / Member Allow policy is what proved the handshake first (build 180769) — it grants more than any deployment pipeline needs. infra/pulumi-config now models a narrower identity in code — teams and roles in Organization/MachineTeam.cs, the issuer's policies in Organization/OidcTrust.cs — so the model is iterated by editing C#, not by clicking.

Two personas, one issuer

azure-devops-ci pulumi-config-admin
Deploys app / infra pipelines via iac-<env> connections infra/pulumi-config itself, via iac-platform-root
Does pulumi env run … -- terraform apply, pulumi up on its own stacks creates Entra apps, ESC environments, Azure Owner assignments, the OIDC issuer's own policies
ESC Open on azure/<env> Open on every environment it manages (transitive closure — see below)
Stacks creates and owns its own (stack:create + creator grant) Edit on cloud-foundations/pulumi-config
Org access stack:create only stack:create, environment:create, team/role/OIDC-issuer/auth-policy management verbs

The only part of sub that varies between connections is the endpoint GUID, and that alone separates the two identities via a second policy on the same issuer:

Policy sub rule Mints
Elevated pinned to the iac-platform-root connection's endpoint GUID (an Output, not a hardcoded constant) team pulumi-config-admin
CI …/sc/7f80b586-…/* — any project, any iac-<env> connection team azure-devops-ci

These do not race each other. The pipeline requests a team (pulumi login --oidc-team azure-devops-ci), so each policy is an authorization check on a (sub, tokenType, teamName) triple — proven in build 180845 (Token type: team: azure-devops-ci) and build 180854 (a non-Platform connection's admin request refused at the exchange, while Platform's succeeds in build 180855). The two-persona boundary holds in both directions.

The cutover rule is gone

OidcTrust.Create declares exactly two policies: the iac-platform-root-pinned elevated rule above and the CI wildcard. A third rule pinned to the old externally-managed Platform Azure-PulumiCloud endpoint (3ac7d649-…) existed during cutover so the identity that deploys this stack couldn't lock itself out. Builds 181555 and 181560 proved the new connection mints the admin token, so it was removed and build 181564 confirmed the admin persona still authenticates without it.

Where the grants live

Environment access for the CI persona is granted next to the environment it grants, inside the EscEnvironmentAccess component — every azure/<env> this stack creates gets its CI TeamEnvironmentPermission in the same commit, so an environment and its pipeline access cannot drift apart. azure/base is granted once separately (a grant per azure/<env> would collide on the same shared environment).

Neither team has a human member, which makes their grants a ceiling for machine tokens rather than something a person inherits.

Pulumi Cloud adds whoever first applies the stack as a team admin

Team membership is populated at create time and cannot be declined: whichever identity runs the first pulumi up is added to both teams as an admin. That is normally the organization service account, but a human running the initial apply is added instead — as happened when this stack was rebuilt from empty. Remove them by hand afterwards:

curl -X PATCH -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
  -d '{"memberAction":"remove","member":"<github-login>"}' \
  https://api.pulumi.com/api/orgs/<org>/teams/<team>

Removing the member produces no Pulumi diff, because the provider only acts on changes to the declared list — see below.

Check membership rather than assuming it, since GET /teams reports members: 0 for every team; fetch each team individually. When a team genuinely has no members the response omits the members key altogether rather than returning an empty array — so verify the key's presence before counting, or a language that wraps null into a one-element array (as PowerShell's @($x) does) will report one phantom member and send you chasing a removal that already worked.

Declaring Members empty is intent, not enforcement — and IgnoreChanges makes it worse

An empty Members list does not remove whoever Pulumi Cloud added: the provider acts only on changes to the declared list, so a member it never saw declared is left alone indefinitely. Declaring it empty records intent and nothing more.

IgnoreChanges on Members is actively harmful here, and the reason is the reverse of the intuition. It substitutes the live value into the desired inputs, so a declared [] and a cloud-populated ['someone'] are guaranteed to disagree on every run — the option causes a permanent ~members diff rather than suppressing one. It was removed; the preview went clean at 274 unchanged with no update even attempted.

Two scope vocabularies exist and are easy to confuse. stack:read / environment:open are entity scopes, granted per entity by TeamStackPermission / TeamEnvironmentPermission. A global-resource-type OrganizationRole carries only the org-wide verbs:

Operation Needs Granted by
pulumi preview / up on an existing stack stack:read / stack:write TeamStackPermission, or the creator grant (see below)
Create a new stack stack:create OrganizationRole (global)
pulumi env run / pulumi env open environment:open TeamEnvironmentPermission
…on everything it imports, transitively environment:open TeamEnvironmentPermission
Create a new ESC environment environment:create OrganizationRole (global)

The authoritative scope catalogue is the getOrganizationRoleScopes data source, or GET /api/orgs/{org}/roles/scopes.

Pulumi Cloud silently drops team:list

It's in the catalogue but discarded on write — ask for it and read the role back, and it is simply absent, no error. Worse, Pulumi's state keeps what you declared, so pulumi preview stays clean and the gap never surfaces from the CLI. AdminGlobalScopes leaves it out so the code matches what the server actually stores; confirm any future scope change with GET /api/orgs/{org}/roles/{roleId}, not pulumi preview.

Create-if-absent, and what it costs

Pipelines create their own stacks on first deploy — stack:create is the only stack-level scope either team's role carries, and no TeamStackPermission is provisioned ahead of a pipeline's first run. This is a deliberate departure from the Terraform estate, which splits bootstrap from deploy purely as a workaround for Terraform Cloud variable-set mapping; that split is not being carried over.

What makes it work is Pulumi Cloud's creator grant: whoever creates a stack (or an ESC environment) automatically gets full control over it, with no corresponding TeamStackPermission needed. Build 181027 proved it for CI — holding stack:create and zero stack grants:

create                            HTTP 200
read own                          HTTP 200
export own state                  HTTP 200
delete own                        HTTP 204

The same mechanism explains why the admin persona opens every azure/<env> environment (and, during cutover, every auth/azure-<env> this stack used to own) while holding no TeamEnvironmentPermission on most of them: it created them.

The creator grant is full control, including delete

The team above deleted the stack it had just created, despite the role carrying no stack:delete and the team holding Stack Admin nowhere. Under create-if-absent, CI can delete its own production stacks, and no role in MachineTeam.cs can restrain that — the grant is issued by Pulumi Cloud at creation time, not by anything declared here. Mitigation is out of band: the org's Allow stack admins to delete stacks toggle, or accepting it as the price of having no bootstrap step.

The elevated persona's environment grants are a transitive closure, plus creator access

A TeamStackPermission says nothing about environments, so the persona needs open grants of its own — and each one must be the transitive import closure of what it consumes, not just the environments it names, because Pulumi checks permission at every link of an import chain. Missing a link fails misleadingly: a refused import resolves to nothing rather than raising, so it surfaces as unknown property "subscriptionId" rather than a permission error pointing at the actual missing grant.

Those grants live beside the resource they grant on, so most of them are not in MachineTeam.cs at all: the per-environment grants sit in EscEnvironmentAccess and the azure/base grants in AzureBaseEnvironment, next to the environments they open. What remains on MachineTeam.CreateAdmin as ExternalEnvironments is only the closure with no local resource to sit beside — today just bootstrap/azure, which this stack consumes but does not create.

Do not read this persona's blast radius off its grant list

pulumi-config-admin opens every environment infra/pulumi-config manages — via the creator grant, not its declared grants — while correctly being refused environments the stack does not manage. When writing a negative assertion for this persona, target an environment the stack doesn't own, or it will always open.

Effective permissions are a union, so the org-wide defaults matter more than the roles

Pulumi Cloud computes effective permissions as the union of a principal's own organization role and every role assigned to teams it belongs to — team roles add to the baseline, they don't replace it. Both teams here resolve to the org service account, which carries the built-in Member role, whose reach for stacks/environments is set org-wide by two UI-only dropdowns under Settings → Access management → Roles → organization-wide role settings (no REST route, no provider resource).

Build 181022 showed what that costs today: the CI team, holding zero stack grants, read an unrelated project's stack in full (metadata, full state export, latest update — all HTTP 200). Environments were correctly refused, pinning the asymmetry to Stack permissions ≥ Read, Environment permissions = None.

Stack scoping is decorative until that dropdown is None

No experiment run against this org can currently prove a stack-level scope is honoured, because the Member baseline already supplies read regardless of what any role here declares. Setting Stack permissions to None is a pending, org-level, UI-only change — tracked separately, not blocking anything documented above.

A related, previously-wrong assumption: a global-resource-type role was thought to reject entity scopes like stack:read / environment:open. It does not — POSTing one with all four returns 200 and reads them back; only team:list is dropped, per the warning above. So a global stack:read (or a tag-based ABAC rule) is a real option for granting StackReference access at scale, once the dropdown above makes it testable.

The issuer

The Pulumi Cloud OIDC issuer is an ordinary managed resource in this stack — created by pulumi up, not registered by hand. It was originally registered in the UI and adopted with a one-time ImportId fed by a cloud-foundations:oidcIssuerId config value; both have since been removed, and a later teardown/rebuild created the issuer from empty, which proved the create path that adoption had until then kept untested.

Removing the config value retired a genuine footgun. It was load-bearing in a way its name did not suggest: clearing it made Create return early, which dropped the issuer from the program, and pulumi up would then delete the registered issuer along with both auth policies — locking out every persona, including the one that deploys this stack. There is now no config value to clear.

Never pin the issuer thumbprint

OidcIssuer accepts a Thumbprints list, and reading the live value back into the source as a constant looks like sound practice. It is a trap: the thumbprint belongs to Microsoft's TLS certificate, not to this configuration, so pinning it makes the resource fail the moment that certificate rotates:

error creating oidc issuer: 400 Bad Request:
Calculated thumbprint 5cc383ad… does not match any of the provided.

Adoption hides this — an imported issuer never re-validates — so the failure surfaces only on the first real create, potentially long after the pin was introduced. The rebuild demonstrated the volatility directly: the value Pulumi calculated during the failure and the value it stored minutes later on success were different again. Leave Thumbprints unset, which is what registering through the UI does.

The issuer URL is still replace-on-change

Url and Organization are replace-on-change and the URL is immutable server-side, so changing the tenant id re-registers the issuer and takes both policies with it — a lockout reached a different way. A genuinely new stack creates its own issuer, which is the intended behaviour; this stack must keep pointing at the tenant it already trusts.

To stop managing the issuer without destroying it, remove it from state with pulumi state delete rather than deleting the resource from the program.

Policies aren't readable over REST

Every /oidc/issuers/{id}/… policy route 404s, so the live policy list can't be fetched directly. Read them from pulumi stack export (they are stored as properties of the OidcIssuer resource) or from pulumi preview --diff.

The provider's own permissions helper is unusable on the current .NET SDK

OrganizationRole.Permissions is a descriptor tree keyed by a __type discriminator, and the provider ships BuildAllowPermissions to build it. The .NET SDK — not the helper — strips __-prefixed keys while deserializing invoke results, in Deserializer.DeserializeStruct. BuildAllowPermissionsResult.Permissions is an untyped ImmutableDictionary<string, object>, so the discriminator is just a key inside it and gets filtered out with the rest. The descriptor reaches the provider without it (failed to parse permission descriptor: type '' not recognized). MachineTeam.cs hand-authors the literal InputMap<object> instead, which works because the serializer has no matching filter on the input path — __type is legal outbound and only unreadable inbound.

Still present in Pulumi 3.110.0, in 3.110.1-alpha, and on main; reported upstream as pulumi/pulumi-dotnet#1102. pulumi/pulumi#22738 is the Python twin, accepted and fixed there. Provider 1.3.0 renamed the discriminator to kind at the SDK boundary (pulumi-pulumiservice#778), so the helper may round-trip cleanly now — confirm with a real invoke before dropping the workaround.

Resource discovery with Pulumi Insights

Each environment also gets a Pulumi Insights account that scans its subscription and indexes what it finds, so the estate can be searched for resources that no Pulumi stack manages. Environments/InsightsDiscovery.cs creates one InsightsAccount per environment.

It adds no new credential path, which is the whole reason it is cheap. Pulumi's Azure Insights recipe asks for an OIDC federated credential with audience azure:<org> and subject pulumi:environments:org:<org>:env:<project>/<env>, plus an ESC environment exposing fn::open::azure-login with oidc: true. The azure/<env> environments this stack already creates satisfy that exactly — so the account selects an existing environment by name rather than provisioning a parallel identity:

Environment = $"{EscEnvironmentAccess.EscProject}/{args.Environment}",

That is the console wizard's "Connect using existing ESC credentials" option, expressed in code.

The scanner inherits Owner

Reusing the ESC environment means the scanner authenticates as iac-<env>, which holds Owner on the subscription. Insights needs only read. This is not a regression — Pulumi Cloud could already open azure/<env> and mint that token, so attaching Insights widens what Pulumi Cloud does with a credential it could already obtain, not what it can obtain. It is still more privilege than a read-only scan warrants, and a natural place to narrow later.

Two knobs, because a playground is not an inventory

The scan schedule has an estate-wide default and a per-environment override:

cloud-foundations:insights:
  scanSchedule: daily          # optional; daily when absent
cloud-foundations:azure:
  environments:
    - name: development-sandbox
      insightsScanSchedule: none   # a playground produces churn, not an inventory

Accepted values are none, 12h and daily. none genuinely disables scanning rather than merely recording an intention — verified by reading scheduledScanEnabled back from the API, which is false for the sandbox and true for the other six.

Accounts are named azure-<env>, with a hyphen

Insights reads / in an account name as hierarchy, where a scan or a delete on a parent cascades to its children. The seven subscriptions are peers, not a tree, so azure/<env> would misdescribe the relationship and make a delete far more dangerous than it looks.

DependsOn here is load-bearing

Environment is passed as a plain string, so Pulumi sees no data dependency on the ESC environment. Account creation validates the credential immediately and 404s if the environment does not exist yet, so InsightsDiscovery takes an explicit DependsOn on it.

Verifying a scan actually ran

scanStatus on the account is always empty — do not read it

GET …/accounts/{name} returns a scanStatus object that stays blank ({id: "", status: "", startedAt: "0001-01-01…"}) no matter how many scans have succeeded. Reading it suggests scanning is broken when it is working perfectly. The real history is a separate sub-resource:

GET https://api.pulumi.com/api/preview/insights/{org}/accounts/{name}/scans

which returns scanStatuses[] with status, startedAt and finishedAt. All seven accounts report succeeded, taking roughly two minutes each.

Discovered resources are not under the account — …/accounts/{name}/resources 404s. They are in the org-wide resource search:

GET https://api.pulumi.com/api/orgs/{org}/search/resources?query=account:azure-prod

Finding undocumented Pulumi Cloud routes

Note the shape: preview leads the path and there is no orgs segment, so none of these routes are guessable. Every attempt to infer them returned 404. They were recovered by scanning the provider plugin binary's string table for route templates:

$exe = "$env:USERPROFILE\.pulumi\plugins\resource-pulumiservice-v1.3.0\pulumi-resource-pulumiservice.exe"
[Text.Encoding]::ASCII.GetString([IO.File]::ReadAllBytes($exe)) |
    Select-String -AllMatches 'api/[a-z/{}]*insight[a-z/{}]*' |
        ForEach-Object { $_.Matches.Value } | Sort-Object -Unique

The technique generalises to any Pulumi Cloud endpoint the docs do not cover: the provider must contain the literal it calls.