<# .SYNOPSIS Provisions and VERIFIES the Exchange Online side of TATER Departure Watch: a scoping group, a management scope bound to that group, and a read-only Exchange role assignment for an EXISTING app registration. GRAPH-GRANT CONTRACT (ADO #1859/#2199 — supersedes older text below): Departure Watch runs on a DEDICATED app registration whose Graph grant is envelope-only Mail.ReadBasic.All and NOTHING broader. The collector verifies the token's own roles claim on every tick and REFUSES a token carrying Mail.Read, Mail.ReadWrite or Mail.ReadWrite.All (TOKEN_ROLES_TOO_BROAD). The mailbox boundary is an ApplicationAccessPolicy fencing that dedicated app to the watch group — MEASURED to bind (2026-08-17: in-scope 200, out-of-scope 403), where the Exchange management scope this script also provisions was MEASURED NOT to bind an app-only Graph read (2026-08-08). Passages below that describe Mail.Read as "the required grant" predate this contract; the audit at the end of this script enforces the current one. .DESCRIPTION Departure Watch reads message METADATA ONLY from a departed employee's mailbox. Before TATER can read anything, Exchange Online must be told which mailboxes the application is allowed to touch — otherwise a tenant running "RBAC for Applications" in default-DENY mode blocks every read with "[RAOP] : Blocked by tenant configured AppOnly AccessPolicy settings." WHAT THIS SCRIPT DOES - Resolves the ENTERPRISE APPLICATION (service principal) object id for an AppId you supply. - Enables organization customization if the Exchange org is dehydrated. - Finds or creates a mail-enabled security group (the watch group). - Optionally adds one mailbox to that group. - Finds or creates a management scope filtered on that group's DISTINGUISHED NAME, and verifies a pre-existing scope actually points at that group. - Registers the Exchange service principal and assigns the READ-ONLY role "Application Mail.Read", scoped to that management scope. - Enumerates EVERY Exchange management role assignment held by this app and fails if a mailbox-data role is unscoped or scoped elsewhere. - Runs a POSITIVE and a NEGATIVE authorization test. - Audits the app's Entra application-permission grants on BOTH resources that can reach mailbox data — Microsoft Graph AND Office 365 Exchange Online — confirming the REQUIRED Graph mail-read grant is present and flagging any grant wider than this read-only feature needs. HOW THE TWO GRANTS RELATE (the part that is easy to get backwards) The Entra application permission and the Exchange management scope do DIFFERENT jobs, and BOTH are required: Graph Mail.Read (application) decides whether the client-credentials token carries mail scope AT ALL. Without it Graph rejects the call before Exchange RBAC is ever consulted, so NO mailbox can be read — see the 'forbidden' branch of api/src/lib/departureWatchGraph.ts. Exchange management scope decides WHICH mailboxes that token may reach — on a tenant that enforces RBAC for Applications (RAOP). So the PRESENCE of a Graph mail grant is NOT a finding. It is a prerequisite, and its ABSENCE is the defect. Setup-TATEREmailIntake.ps1 has shipped exactly this combination — tenant-wide Entra Graph mail grants alongside a single-mailbox Exchange management scope — since 2026-06-05 on a RAOP-enforcing tenant, correctly scoped to one mailbox the whole time. The [RAOP] 403 that script exists to remediate is itself the proof: the tenant-wide Graph grant was already consented and every read still failed until the SCOPED Exchange role assignment landed. A Graph grant that defeated the Exchange scope would have made that failure impossible. WHAT ACTUALLY DISTINGUISHES "BOUNDED" FROM "TENANT-WIDE" Only an EMPIRICAL NEGATIVE READ — an app-only token for THIS application asking Graph for a mailbox that is NOT in the watch group: 403 carrying the [RAOP] fingerprint -> bounded 200 -> UNBOUNDED — the real danger no Graph mail grant at all -> nothing is readable by anyone This script CANNOT perform that read: it holds an administrator's delegated sign-in, not the application's own credentials, and it will not handle a client secret. It therefore reports boundedness as NOT RUN with the exact commands to run, and NEVER as PASS. See EXIT CODES below. ############################################################################ # MEASURED 2026-08-08 AT CARON BLETZER: THE READ RETURNED 200. NOT BOUNDED. # # This is no longer a theoretical caveat. The empirical read was performed # for the first time and the application read a mailbox OUTSIDE the watch # group. Everything this script provisions was present and correct: # # * one mail-reaching Graph permission (Mail.Read) and no other # * one Exchange role assignment, Application Mail.Read, scoped # * the scope well-formed, IsValid True, filtering on the group DN # * the group containing exactly the one intended mailbox # * NO full_access_as_app; only Exchange.ManageAsApp # # There was nothing to fix. The premise is wrong: an Exchange management # scope did not constrain an app-only Graph mail read in that tenant. # # It also inverts the reasoning in the block above. The email-intake [RAOP] # 403 was read as proof that scoping works. The consistent explanation is # that RBAC for Applications denies an app with NO role assignment, and once # ANY assignment exists the Graph permission's tenant-wide reach applies — # so adding a scoped assignment did not scope that feature, it unblocked it. # # THEREFORE: a PASS from every check in this script still does not mean the # application is confined to the watch group. Run the empirical read, and do # not enable monitoring until it returns 403 [RAOP]. Where it returns 200, # the cheapest containment is removing the mail permission from the app # rather than an ApplicationAccessPolicy, which scopes the WHOLE app and has # previously broken unrelated features in this tenant. ############################################################################ WHAT THIS SCRIPT DOES NOT DO - It does NOT create an app registration. It reuses the org's existing per-tenant TenantCredentials application. Pass its AppId. - It does NOT create a service principal / enterprise application. If one does not exist the script stops and tells you how to create it. - It does NOT create, print, or rotate any client secret. - It does NOT remove any Entra permission grant. The audit reports and explains; a human decides. - It does NOT turn on monitoring. It provisions an ACCESS BOUNDARY. A provisioned mailbox is not a monitored one. See "TATER-SIDE STATUS". TATER-SIDE STATUS (read this before you tell anyone monitoring is on) BUILT: the REST API (attestation, per-case authorization, case create, activate, deactivate, gate status), collection every 15 minutes, a daily briefing, retention enforcement, and delivery to the case's recorded recipient list. NOT BUILT: any user interface, and any MCP tool. The authorization and activation paths are deliberately excluded from MCP — an agent must not be able to authorize surveillance of a person. STILL REQUIRED before any message is read, none of which this script does: 1. DEPARTURE_WATCH_ENABLED=1 on the API (TATER Security sets this; it is a platform kill switch and it is currently OFF). 2. An organization policy attestation, which expires annually — expiry STOPS collection rather than warning about it. 3. A named human's per-case authorization with a written justification. 4. Activation, which re-verifies this Exchange boundary and refuses if it is absent. This status changes as the feature ships. Check it against your release rather than trusting a copy of this text. Required PowerShell modules: - ExchangeOnlineManagement - Microsoft.Graph.Applications Required roles: - Exchange Administrator (management scope + role assignment) - Global Reader or Application Administrator (read the app's Entra permission grants for the audit — READ-ONLY Graph scopes are requested) - OR Global Administrator (has both) Idempotent — safe to re-run. Re-running changes nothing that already matches, and re-runs every verification check. Help doc: https://www.tatersecurity.com/Docs/Help/departure-watch.html .PARAMETER AppId The application (client) id of the org's EXISTING TenantCredentials app — the same app already used for Graph compliance scanning. Not an app registration object id, not a service principal object id. .PARAMETER Mailbox Optional. A mailbox to add to the watch group. Omit to provision the scope and role assignment without putting any mailbox in scope yet. .PARAMETER GroupName Name of the security group that holds the watched mailboxes. Default: 'sg_tater_departure_watch'. .PARAMETER ScopeName Name of the Exchange management scope. Default: 'TATER-DepartureWatch'. .PARAMETER NegativeTestMailbox A mailbox that is NOT in the watch group. It supplies the Exchange-side evidence of boundedness: Exchange RBAC does not place that mailbox in this application's scope. Strongly encouraged — without it the script can only show that access works, never that it is limited. Note what it is and is not. Test-ServicePrincipalAuthorization reports the STORED RBAC CONFIGURATION, not enforced state, so a passing negative test is strong evidence and not proof. The empirical negative read described above is the proof, and this script cannot perform it. .PARAMETER GroupManagedBy Optional. Owner(s) of the watch group when the group has to be created. Adding a mailbox to this group grants an application read access to that person's mail, so who may do that is part of the boundary. Omitted, Exchange defaults ManagedBy to the account running this script. .PARAMETER VerifyOnly Run every check and change nothing. Missing objects are reported as failures rather than created. .PARAMETER WhatIf Preview every change without making it. This ALSO suppresses the PSGallery module bootstrap in Step 1 — a dry run installs nothing at all. If a required module is missing under -WhatIf the script stops and prints the exact Install-Module command for you to run. .EXAMPLE # Provision + verify, adding one mailbox and proving boundedness .\Setup-TATERDepartureWatch.ps1 -AppId 11111111-2222-3333-4444-555555555555 ` -Mailbox departing.manager@contoso.com ` -NegativeTestMailbox unrelated.person@contoso.com .EXAMPLE # Reusing the org's existing compliance-scanning app with a distinct scope # name, because a scope called TATER-DepartureWatch already exists .\Setup-TATERDepartureWatch.ps1 -AppId 11111111-2222-3333-4444-555555555555 ` -GroupName 'sg_tater_departure_watch_2' ` -ScopeName 'TATER-DepartureWatch-2' ` -Mailbox departing.manager@contoso.com ` -NegativeTestMailbox someone.else@contoso.com .EXAMPLE # Show what would change, without changing anything .\Setup-TATERDepartureWatch.ps1 -AppId 11111111-2222-3333-4444-555555555555 ` -Mailbox departing.manager@contoso.com -WhatIf .EXAMPLE # Re-verify an existing deployment (no changes at all) .\Setup-TATERDepartureWatch.ps1 -AppId 11111111-2222-3333-4444-555555555555 ` -NegativeTestMailbox someone.else@contoso.com -VerifyOnly .NOTES EXIT CODES — this script is safe to gate a pipeline, runner or && chain on. 0 Every check PASSED. Nothing failed and nothing was left unproven. 1 At least one check is NOT RUN and none FAILED. Something the script exists to prove was not proved — treat it as unverified, not as safe. 2 At least one check FAILED. Two different kinds of bad arrive here and the table tells them apart: ACTIVE EXPOSURE — "NEGATIVE TEST FAILED", a mailbox-data role assignment outside the scope, or an Entra grant wider than a read-only feature can justify. BROKEN FEATURE — no Graph mail-read grant, so nothing can read any mailbox at all. Not an exposure; still a failure. EXIT 0 IS NOT REACHABLE FROM A SCRIPT-ONLY RUN, deliberately. The boundedness check needs an app-only read this script cannot perform (see "WHAT ACTUALLY DISTINGUISHES..." above), so that row is always NOT RUN and the best a clean run produces is 1. This is not a red gate to route around: 1 here means "provisioned and configured correctly, boundedness not yet demonstrated empirically". Gate on `-le 1` if you must automate, and run the empirical read as its own step. A script that could flip that row to PASS on its own say-so would be asserting the single thing it has no evidence for. An early abort (a throw, e.g. Graph or Exchange sign-in failure) terminates with the host's own non-zero status before any check runs. #> # SupportsShouldProcess is a DELIBERATE departure from Setup-TATEREmailIntake.ps1, # which has no -WhatIf. Departure Watch grants an application read access to a # person's mailbox; an operator must be able to see exactly what would change # before it changes. [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'Medium')] param( # Application (client) id of the org's EXISTING TenantCredentials app. # This script never creates an app registration — Departure Watch reuses the # per-tenant app already used for Graph compliance scanning, so that one # consent decision covers one identity instead of two. [Parameter(Mandatory = $true)] [string]$AppId, # Optional mailbox to add to the watch group. Provisioning the scope with no # mailbox in it is a legitimate first step — the fence exists before anyone # is inside it, and the positive test will then report NOT RUN rather than # inventing a subject to test with. [string]$Mailbox, # Security group holding the watched mailboxes. The management scope filter # matches DIRECT members of this group only — nested groups do NOT expand. # Default follows the sg_/dg_/ag_ group naming standard. [string]$GroupName = "sg_tater_departure_watch", # Primary SMTP address for the group when it has to be created. Left blank, # it is derived from GroupName plus the tenant's DEFAULT accepted domain. # Pass it explicitly in a multi-vanity-domain tenant where the default # accepted domain is not the one you want on this object. [string]$GroupPrimarySmtpAddress, # Exchange management scope name. One scope serves the whole group, so this # rarely needs changing — but pass a distinct name if a scope by the default # name already exists for some unrelated purpose. [string]$ScopeName = "TATER-DepartureWatch", # A mailbox that must NOT be readable. This is the Exchange-side evidence of # boundedness. It is strong but it is not proof: the cmdlet behind it reports # the STORED RBAC configuration, not enforced state — only the empirical # app-only read in Step 10c can settle that, and this script cannot perform # it. Omitting this is allowed but is reported as a NOT RUN check, never as # a pass. [string]$NegativeTestMailbox, # Owner(s) of the watch group, applied only when this script CREATES it. # Membership of this group is the gate to the surveillance scope, so who may # change it is part of the boundary and is reported in the results table. [string[]]$GroupManagedBy, # Enterprise-application (service principal) object id, if you already know # it and cannot sign in to Graph from this session. Supplying it lets the # provisioning proceed, but the Entra grant audit will then report NOT RUN. [string]$ServicePrincipalObjectId, # Run every check, change nothing. Anything missing is a FAIL, not a create. [switch]$VerifyOnly, # Use interactive browser auth for Graph instead of device code (default). # Device code sidesteps the InteractiveBrowserCredential vs cached # Microsoft.Identity.Client mismatch that older PS sessions hit. [switch]$UseBrowserAuth ) $ErrorActionPreference = "Stop" function Write-Step($msg) { Write-Host "▸ $msg" -ForegroundColor Cyan } function Write-OK($msg) { Write-Host "✓ $msg" -ForegroundColor Green } function Write-Warn2($msg){ Write-Host "⚠ $msg" -ForegroundColor Yellow } # Write-Fail is an addition to the sibling script's helper set. This script # exists to verify a security boundary; a failed check must not render in the # same colour as an advisory note. function Write-Fail($msg) { Write-Host "✗ $msg" -ForegroundColor Red } # Every check records exactly one of PASS / FAIL / NOT RUN. A check that could # not run must never render as a pass — that is the failure mode this whole # script is written against. $script:checks = @() function Add-Check($name, $status, $detail) { # A second status for the same check is a defect in THIS script, and it # would quietly corrupt the PASS / FAIL / NOT RUN counts that the exit code # is computed from. Surface it instead of letting one status mask the other. if (@($script:checks | Where-Object { $_.Name -eq $name }).Count -gt 0) { Write-Warn2 "INTERNAL: check '$name' was recorded more than once — reporting both records." $name = "$name (duplicate record)" } $script:checks += [pscustomobject]@{ Name = $name; Status = $status; Detail = $detail } } # Read a recorded status back by name. A check that was never recorded reports # NOT RUN — the safe reading — so a downstream claim can never be made on the # strength of a check that silently vanished. function Get-CheckStatus($name) { $c = @($script:checks | Where-Object { $_.Name -eq $name }) | Select-Object -First 1 if ($c) { return $c.Status } return "NOT RUN" } # Check names referenced later (summary, boundedness claim, exit status). Held # in variables so a rename cannot silently break a lookup and turn a real PASS # or FAIL into a phantom NOT RUN. $checkNamePositive = "POSITIVE test (access works)" $checkNameNegative = "NEGATIVE test (cmdlet says out-of-scope mailbox is denied)" $checkNameOtherRoles = "App's other Exchange role assignments" $checkNameGroupGate = "Watch group membership is gated" # The single "Entra mailbox-access grant audit" check was split into three, # because it was answering three different questions with one verdict — and it # answered the first one backwards, failing on the presence of the very grant # the feature requires. $checkNameGraphGrant = "Graph mail-read grant present (REQUIRED)" $checkNameLeastPriv = "Entra grants are least privilege" $checkNameBounded = "Entra grant boundedness (empirical read)" # Normalise an OPATH recipient filter for WHOLE-STRING comparison. Substring # matching is not safe here: a filter that CONTAINS the expected clause can also # contain "-or MemberOfGroup -eq ''", which is a strictly WIDER # fence that a substring test happily accepts. function ConvertTo-NormalisedFilter($f) { if ($null -eq $f) { return "" } $n = [string]$f $n = $n -replace '"', "'" # Exchange may re-quote; unify the quote char $n = $n -replace '\s+', ' ' # collapse all whitespace runs $n = $n.Trim() $n = $n -replace '^\((.*)\)$', '$1' # drop one layer of wrapping parentheses return $n.Trim().ToLowerInvariant() } # Tracks whether this run actually changed anything that could affect an # authorization test. It is what lets the verification section distinguish # "not propagated yet" from "misconfigured" instead of guessing. $provisioningChangedNow = $false $noChanges = $VerifyOnly.IsPresent # Mailbox data can be reached from TWO different resources, and the Step 10 # audit must inspect BOTH — but they play OPPOSITE roles for this feature. # # Microsoft Graph — where the REQUIRED grant lives. Departure Watch # reads mail through Graph, so a Graph mail-read # application permission is what makes the # client-credentials token carry mail scope. With # no such grant, Graph rejects the call before # Exchange RBAC is consulted and nothing is # readable at all. # Office 365 Exchange Online — where NOTHING this feature needs lives. The # reader never uses EWS, IMAP or POP, so every # mailbox-reaching role on this resource is # surplus by construction — and it is where # full_access_as_app lives, the single most # complete mailbox permission there is. # # The Exchange Online resource matters SPECIFICALLY on this app. Departure Watch # reuses the org's existing per-tenant compliance-scanning registration, and that # app already holds Exchange.ManageAsApp on the Office 365 Exchange Online # service principal (it is how cloud scanning runs Exchange cmdlets app-only). # So the Exchange resource is the one this app demonstrably already has grants # on. An audit that inspects only Graph misses that pile entirely while printing # a clean result. $graphResourceAppId = "00000003-0000-0000-c000-000000000000" $exchangeResourceAppId = "00000002-0000-0ff1-ce00-000000000000" $helpDocUrl = "https://www.tatersecurity.com/Docs/Help/departure-watch.html" Write-Host "" Write-Host "════════════════════════════════════════════════════════════════════════════" -ForegroundColor Magenta Write-Host " TATER Departure Watch — Exchange provisioning + verification" -ForegroundColor Magenta Write-Host "════════════════════════════════════════════════════════════════════════════" -ForegroundColor Magenta Write-Host "" if ($noChanges) { Write-Warn2 "-VerifyOnly: nothing will be created or modified. Missing objects are reported as FAIL." Write-Host "" } # ────────────────────────────────────────────────────────────────────────── # Step 1 — Verify / install required modules # ────────────────────────────────────────────────────────────────────────── $requiredModules = @( 'ExchangeOnlineManagement', 'Microsoft.Graph.Applications' ) $missingModules = @($requiredModules | Where-Object { -not (Get-Module -ListAvailable -Name $_) }) if ($missingModules.Count -gt 0) { if ($WhatIfPreference) { # -WhatIf must not fetch and execute code on the admin's workstation. A # documented "preview it first" path that installs modules from PSGallery # unattended is not a preview, so this stops instead. Write-Host "" Write-Fail "Missing required module(s): $($missingModules -join ', ')" Write-Warn2 "-WhatIf: nothing is installed, including modules. Install them yourself and re-run:" foreach ($m in $missingModules) { Write-Host " Install-Module $m -Scope CurrentUser -Repository PSGallery" } Write-Host "" throw "Aborted — required module(s) missing and -WhatIf installs nothing." } foreach ($m in $missingModules) { Write-Step "Installing module $m (current user)…" Write-Warn2 "This DOWNLOADS AND INSTALLS code from PSGallery. -Force also suppresses the" Write-Warn2 "untrusted-repository confirmation prompt. Ctrl-C now if that is not wanted." # -WhatIf:$false is belt-and-braces: the -WhatIf case already returned above. Install-Module $m -Scope CurrentUser -Force -AllowClobber -Repository PSGallery -WhatIf:$false } } Import-Module ExchangeOnlineManagement -ErrorAction Stop Import-Module Microsoft.Graph.Applications -ErrorAction Stop # ────────────────────────────────────────────────────────────────────────── # Step 2 — Connect to Microsoft Graph (READ-ONLY scopes) # ────────────────────────────────────────────────────────────────────────── # Graph is used for exactly two reads: resolving the enterprise application's # object id, and enumerating the app's application-permission grants for the # Step 9 audit. Nothing here writes to Entra, so no write scope is requested — # this script cannot remove a grant even if you want it to. $graphConnected = $false $graphContext = $null $tenantId = "(unknown — Graph not connected)" $authMode = "device code" if ($UseBrowserAuth) { $authMode = "interactive browser" } Write-Step "Connecting to Microsoft Graph ($authMode sign-in, read-only scopes)…" $graphScopes = @( 'Application.Read.All', # read the app registration + service principal 'Directory.Read.All' # read appRoleAssignments for the Step 10 grant audit ) # Re-running this script is the NORMAL path, not the exception: the membership # cache means a second -VerifyOnly pass ~2h later is expected, and the operator # has usually just been in both consoles by hand. Burning a fresh device code on # every run is pure friction, so reuse a live session that already carries the # scopes we need. Scopes are CHECKED, not assumed — a session connected for some # other purpose can be missing one, and that would fail later, further from the cause. $existingGraph = $null try { $existingGraph = Get-MgContext -ErrorAction SilentlyContinue } catch { $existingGraph = $null } $graphMissingScopes = @() if ($existingGraph) { $graphMissingScopes = @($graphScopes | Where-Object { $existingGraph.Scopes -notcontains $_ }) } $graphPreexisting = $false try { # NO `| Out-Null` here — `-UseDeviceCode` prints the code + verification # URL to the host, and suppressing the output stream hides the prompt. if ($existingGraph -and $graphMissingScopes.Count -eq 0) { $graphPreexisting = $true Write-OK "Reusing the existing Microsoft Graph session ($($existingGraph.Account)) — no new sign-in" } elseif ($UseBrowserAuth) { Connect-MgGraph -Scopes $graphScopes -NoWelcome -ErrorAction Stop } else { Write-Host "" Write-Host " You'll see a CODE and URL below — visit the URL in any browser," -ForegroundColor Yellow Write-Host " paste the code, and complete sign-in. The script will continue automatically." -ForegroundColor Yellow Write-Host "" Connect-MgGraph -Scopes $graphScopes -NoWelcome -UseDeviceCode -ErrorAction Stop } $graphConnected = $true } catch { $msg = $_.Exception.Message if ($msg -match 'WithLogging' -or $msg -match 'Method not found' -or $msg -match 'Microsoft\.Identity\.Client') { Write-Host "" Write-Warn2 "Detected an MSAL/Graph SDK version mismatch in this PowerShell session." Write-Warn2 "This happens when older Microsoft.Graph modules are loaded alongside newer dependencies." Write-Host "" Write-Host " Quickest fix (recommended):" -ForegroundColor Cyan Write-Host " 1. Close this PowerShell window completely." Write-Host " 2. Open a FRESH PowerShell 7 session (not 5.1 — Microsoft.Graph dropped 5.1 support)." Write-Host " 3. Run: Update-Module Microsoft.Graph -Force" Write-Host " 4. Re-run this script." Write-Host "" Write-Host " Alternative (refresh the modules in place):" -ForegroundColor Cyan Write-Host " Update-Module Microsoft.Graph.Applications -Force" Write-Host " Get-Module Microsoft.Graph* | Remove-Module -Force" Write-Host " (then re-run)" Write-Host "" Write-Host " Or force the browser flow if device code itself failed:" -ForegroundColor Cyan Write-Host " .\Setup-TATERDepartureWatch.ps1 -AppId $AppId -UseBrowserAuth" Write-Host "" if (-not $ServicePrincipalObjectId) { throw "Aborted due to MSAL/Graph SDK mismatch — see remediation above." } } if (-not $ServicePrincipalObjectId) { Write-Host "" Write-Warn2 "Graph sign-in failed: $msg" Write-Warn2 "The enterprise application object id cannot be resolved without Graph." Write-Warn2 "Either fix sign-in and re-run, or look the id up elsewhere and pass it:" Write-Host " az ad sp show --id $AppId --query id -o tsv" Write-Host " .\Setup-TATERDepartureWatch.ps1 -AppId $AppId -ServicePrincipalObjectId " throw "Aborted — Graph sign-in failed and no -ServicePrincipalObjectId was supplied." } Write-Warn2 "Graph sign-in failed: $msg" Write-Warn2 "Continuing with the supplied -ServicePrincipalObjectId. The Entra grant audit" Write-Warn2 "in Step 9 will report NOT RUN — an unrun audit is not a clean audit." } if ($graphConnected) { $graphContext = Get-MgContext $tenantId = $graphContext.TenantId Write-OK "Connected to tenant $tenantId as $($graphContext.Account)" } # ────────────────────────────────────────────────────────────────────────── # Step 3 — Resolve the ENTERPRISE APPLICATION (service principal) object id # ────────────────────────────────────────────────────────────────────────── # This is the single most commonly botched value in this whole sequence. # New-ServicePrincipal -ObjectId wants the object id of the ENTERPRISE # APPLICATION (the service principal in the tenant), NOT the object id of the # APP REGISTRATION. Passing the app registration's object id is accepted here # and then fails later, at New-ManagementRoleAssignment, with the misleading # error "Couldn't find a service principal" — pointing you at the wrong step. # $appDisplayName stays $null until the DIRECTORY tells us what the app is # called. A hardcoded placeholder would be written into the tenant as the # Exchange service principal's DisplayName and printed in the summary as # "Application name", asserting a directory value this script never read — on # the fallback path the app is in fact the org's existing compliance-scanning # registration, under whatever name it already has. $spObjectId = $null $appDisplayName = $null $appDisplayLabel = "(not resolved — no Graph session)" if ($graphConnected) { Write-Step "Resolving the enterprise application (service principal) for AppId $AppId…" $entraSp = Get-MgServicePrincipal -Filter "appId eq '$AppId'" -ErrorAction SilentlyContinue | Select-Object -First 1 if (-not $entraSp) { Write-Host "" Write-Fail "No enterprise application (service principal) exists in this tenant for AppId $AppId." Write-Host "" # Distinguish the two most likely causes rather than guessing. $maybeAppByObjectId = $null try { $maybeAppByObjectId = Get-MgApplication -ApplicationId $AppId -ErrorAction SilentlyContinue } catch { $maybeAppByObjectId = $null } if ($maybeAppByObjectId) { Write-Warn2 "The value you passed is an APP REGISTRATION OBJECT ID, not an application (client) id." Write-Warn2 "Re-run with the client id instead:" Write-Host " -AppId $($maybeAppByObjectId.AppId)" } else { Write-Warn2 "Either the AppId is wrong, or the application is registered in another tenant" Write-Warn2 "and has never been consented into this one." Write-Host "" Write-Host " This script deliberately does NOT create the enterprise application." -ForegroundColor Cyan Write-Host " Creating one silently would mean this script provisioned the identity it then" Write-Host " grants mailbox read access to. An administrator makes that decision, not a script." Write-Host "" Write-Host " If the application is correct and simply not present in this tenant, create it:" -ForegroundColor Cyan Write-Host " az ad sp create --id $AppId" Write-Host " # or: New-MgServicePrincipal -AppId $AppId" Write-Host " Then re-run this script." } Write-Host "" Add-Check "Enterprise application resolved" "FAIL" "No service principal for AppId $AppId in tenant $tenantId" if ($graphConnected -and -not $graphPreexisting) { Disconnect-MgGraph | Out-Null } throw "Aborted — no enterprise application (service principal) for AppId $AppId." } $spObjectId = $entraSp.Id if ($entraSp.DisplayName) { $appDisplayName = $entraSp.DisplayName $appDisplayLabel = $entraSp.DisplayName } else { $appDisplayLabel = "(service principal has no DisplayName)" } Write-OK "Enterprise application: '$appDisplayLabel'" Write-Host " Service principal (ENTERPRISE APPLICATION) object id : $spObjectId" -ForegroundColor DarkGray Write-Host " This is NOT the app registration object id. New-ServicePrincipal -ObjectId takes THIS value." -ForegroundColor DarkGray if ($ServicePrincipalObjectId -and $ServicePrincipalObjectId -ne $spObjectId) { Write-Warn2 "-ServicePrincipalObjectId was supplied as $ServicePrincipalObjectId but Graph resolves" Write-Warn2 "the enterprise application to $spObjectId. Using the value Graph returned." } Add-Check "Enterprise application resolved" "PASS" "$appDisplayLabel ($spObjectId)" } else { $spObjectId = $ServicePrincipalObjectId Write-Warn2 "Using the supplied -ServicePrincipalObjectId ($spObjectId) without verifying it against Graph." Write-Warn2 "If this is an app REGISTRATION object id rather than an ENTERPRISE APPLICATION object id," Write-Warn2 "New-ManagementRoleAssignment will later fail with `"Couldn't find a service principal`"." Add-Check "Enterprise application resolved" "NOT RUN" "Supplied by hand; not verified against Graph" } # ────────────────────────────────────────────────────────────────────────── # Step 4 — Connect to Exchange Online # ────────────────────────────────────────────────────────────────────────── Write-Step "Connecting to Exchange Online…" try { $exoConn = @() try { $exoConn = @(Get-ConnectionInformation -ErrorAction SilentlyContinue | Where-Object { $_.State -eq 'Connected' }) } catch { $exoConn = @() } if ($exoConn.Count -gt 0) { Write-OK "Reusing the existing Exchange Online session ($($exoConn[0].UserPrincipalName)) — no new sign-in" } else { Connect-ExchangeOnline -ShowBanner:$false -Device | Out-Null Write-OK "Connected to Exchange Online" } } catch { Write-Fail "Exchange Online connection failed: $($_.Exception.Message)" Write-Warn2 "Nothing was provisioned and nothing was verified." Write-Warn2 "Sign in as an Exchange Administrator and re-run — this script is idempotent." if ($graphConnected -and -not $graphPreexisting) { Disconnect-MgGraph | Out-Null } throw "Aborted — could not connect to Exchange Online." } # ────────────────────────────────────────────────────────────────────────── # Step 5 — Organization customization (IsDehydrated) # ────────────────────────────────────────────────────────────────────────── # Custom scopes and role assignments require the Exchange org to be # "customized" — a one-time, harmless provisioning step that fresh tenants # haven't done. Without it, New-ManagementScope fails telling you to run # Enable-OrganizationCustomization first. $orgCfg = Get-OrganizationConfig -ErrorAction SilentlyContinue if ($orgCfg -and $orgCfg.IsDehydrated) { if ($noChanges) { Write-Warn2 "Exchange org is not customized (IsDehydrated). -VerifyOnly: not enabling it." Add-Check "Exchange organization customization" "FAIL" "IsDehydrated = True; run without -VerifyOnly" } elseif ($PSCmdlet.ShouldProcess("Exchange organization", "Enable-OrganizationCustomization")) { Write-Step "Exchange org is not yet customized — running Enable-OrganizationCustomization (one-time)…" try { Enable-OrganizationCustomization -ErrorAction Stop # The confirming read is the ONLY evidence that customization landed. # Track the last SUCCESSFUL read separately from the loop variable: # if every Get-OrganizationConfig in the poll returns nothing, a guard # written as `if ($orgCfg -and $orgCfg.IsDehydrated)` is FALSE for the # null case, so the throw it was meant to protect never fires and an # entirely unverified org is recorded as PASS. $customizationConfirmed = $false $lastGoodOrgCfg = $null $waited = 0 while ($waited -lt 300) { Start-Sleep -Seconds 20; $waited += 20 $orgProbe = Get-OrganizationConfig -ErrorAction SilentlyContinue if ($orgProbe) { $lastGoodOrgCfg = $orgProbe if (-not $orgProbe.IsDehydrated) { $customizationConfirmed = $true; break } } Write-Host " …still provisioning ($waited s)" } if (-not $customizationConfirmed) { if ($lastGoodOrgCfg) { throw "Enable-OrganizationCustomization is still provisioning after 5 minutes (IsDehydrated is still True). Wait 10-15 minutes and re-run this script — it is idempotent." } throw "Enable-OrganizationCustomization could not be confirmed — Get-OrganizationConfig returned nothing during the 5-minute poll, so the org's customization state was never read back. Re-run this script (it is idempotent) once Get-OrganizationConfig responds." } $orgCfg = $lastGoodOrgCfg Write-OK "Organization customization enabled" # Set ONLY on confirmed success. $provisioningChangedNow downgrades a # genuine positive-test FAIL to NOT RUN ("propagation may explain # this"); setting it from an unconfirmed change would excuse a real # failure with a delay that never happened. $provisioningChangedNow = $true Add-Check "Exchange organization customization" "PASS" "Enabled and confirmed IsDehydrated = False during this run" } catch { # Covers both "the cmdlet errored" and "the cmdlet returned but the # confirming read never showed IsDehydrated = False". Neither is a # customized org, and neither may be reported as one. Write-Fail "Enable-OrganizationCustomization did not complete verifiably: $($_.Exception.Message)" Write-Warn2 "The management scope cannot be created until this succeeds. Run it manually:" Write-Host " Connect-ExchangeOnline" Write-Host " Enable-OrganizationCustomization" Add-Check "Exchange organization customization" "FAIL" $_.Exception.Message } } else { Add-Check "Exchange organization customization" "NOT RUN" "Skipped (-WhatIf)" } } elseif ($orgCfg) { Write-OK "Exchange organization is already customized" Add-Check "Exchange organization customization" "PASS" "IsDehydrated = False" } else { Write-Warn2 "Get-OrganizationConfig returned nothing — cannot tell whether the org is customized." Add-Check "Exchange organization customization" "NOT RUN" "Get-OrganizationConfig returned no result" } # ────────────────────────────────────────────────────────────────────────── # Step 6 — Find or create the watch group # ────────────────────────────────────────────────────────────────────────── # The group is the fence. Everything downstream — the scope filter, the role # assignment, the tests — only means anything relative to this group's # membership. Write-Step "Finding or creating the watch group '$GroupName'…" $groupObj = $null $groupIsMailEnabled = $false $groupIdentity = $GroupName $existingDg = Get-DistributionGroup -Identity $GroupName -ErrorAction SilentlyContinue | Select-Object -First 1 if ($existingDg) { $groupObj = $existingDg $groupIsMailEnabled = $true $groupIdentity = $existingDg.Identity Write-OK "Found existing mail-enabled group '$GroupName' ($($existingDg.PrimarySmtpAddress))" if ($existingDg.RecipientTypeDetails -ne 'MailUniversalSecurityGroup') { Write-Warn2 "Group type is $($existingDg.RecipientTypeDetails), not MailUniversalSecurityGroup." Write-Warn2 "Scoping still works, but a non-security distribution group is a weaker container" Write-Warn2 "for a surveillance boundary — anyone who can manage the DL can widen the scope." } Add-Check "Watch group present" "PASS" "$GroupName ($($existingDg.RecipientTypeDetails))" } else { # It may exist as a non-mail-enabled security group; MemberOfGroup scoping # works from the DN either way, so use it rather than creating a duplicate. $existingPlainGroup = Get-Group -Identity $GroupName -ErrorAction SilentlyContinue | Select-Object -First 1 if ($existingPlainGroup) { $groupObj = $existingPlainGroup $groupIdentity = $existingPlainGroup.Identity Write-OK "Found existing group '$GroupName' ($($existingPlainGroup.RecipientTypeDetails)) — not mail-enabled" Write-Warn2 "Membership of a non-mail-enabled group is managed in Entra, not with" Write-Warn2 "Add-DistributionGroupMember. Mailbox membership steps below will report NOT RUN." Add-Check "Watch group present" "PASS" "$GroupName ($($existingPlainGroup.RecipientTypeDetails), not mail-enabled)" } elseif ($noChanges) { Write-Fail "Group '$GroupName' does not exist. -VerifyOnly: not creating it." Add-Check "Watch group present" "FAIL" "Group '$GroupName' not found" } elseif ($PSCmdlet.ShouldProcess($GroupName, "New-DistributionGroup -Type Security")) { # Derive an alias + primary SMTP address. Alias characters are restricted; # underscore is legal, most punctuation is not. $groupAlias = $GroupName -replace '[^A-Za-z0-9._-]', '-' $groupSmtp = $GroupPrimarySmtpAddress if (-not $groupSmtp) { $defaultDomain = Get-AcceptedDomain -ErrorAction SilentlyContinue | Where-Object { $_.Default -eq $true } | Select-Object -First 1 if (-not $defaultDomain) { Write-Fail "Could not determine the tenant's default accepted domain." Write-Warn2 "Re-run with an explicit address: -GroupPrimarySmtpAddress '$groupAlias@yourdomain.com'" Add-Check "Watch group present" "FAIL" "No default accepted domain and no -GroupPrimarySmtpAddress" } else { $groupSmtp = "$groupAlias@$($defaultDomain.DomainName)".ToLower() } } if ($groupSmtp) { try { # Membership of this group IS the surveillance boundary: adding a # mailbox to it grants an application read access to that person's # mail. Closed join/depart restrictions mean nobody adds or removes # themselves — a change of scope goes through an owner. Leaving the # gate at tenant defaults would leave the boundary's own gate # unstated, which is the thing this script exists not to do. $newDgParams = @{ Name = $GroupName DisplayName = $GroupName Alias = $groupAlias Type = 'Security' PrimarySmtpAddress = $groupSmtp MemberJoinRestriction = 'Closed' MemberDepartRestriction = 'Closed' ErrorAction = 'Stop' } if ($GroupManagedBy -and $GroupManagedBy.Count -gt 0) { $newDgParams['ManagedBy'] = $GroupManagedBy } $groupObj = New-DistributionGroup @newDgParams $groupIsMailEnabled = $true $groupIdentity = $groupObj.Identity $provisioningChangedNow = $true Write-OK "Created mail-enabled security group $groupSmtp" # Directory replication before the DN is reliably readable. Start-Sleep -Seconds 5 Add-Check "Watch group present" "PASS" "Created $groupSmtp during this run" } catch { Write-Fail "Group creation failed: $($_.Exception.Message)" Write-Warn2 "Create it manually and re-run — this script is idempotent:" Write-Host " Connect-ExchangeOnline" Write-Host " New-DistributionGroup -Name '$GroupName' -Alias '$groupAlias' -Type Security -PrimarySmtpAddress '$groupSmtp' ``" Write-Host " -MemberJoinRestriction Closed -MemberDepartRestriction Closed -ManagedBy " Add-Check "Watch group present" "FAIL" $_.Exception.Message } } } else { Add-Check "Watch group present" "NOT RUN" "Skipped (-WhatIf)" } } # ────────────────────────────────────────────────────────────────────────── # Step 6a — Who may change the membership of the fence # ────────────────────────────────────────────────────────────────────────── # The scope is only as tight as the group, and the group is only as tight as # the set of people who can add someone to it. That set belongs in the verified # output, not in tenant defaults — the script already makes this argument about # non-security distribution lists, and it applies with more force to the group # it created for this purpose. if (-not $groupObj) { Add-Check $checkNameGroupGate "NOT RUN" "Watch group is missing — nothing to read" } elseif (-not $groupIsMailEnabled) { Write-Warn2 "'$GroupName' is not mail-enabled; join/depart restrictions are an Exchange" Write-Warn2 "distribution-group concept. Verify its owners and membership gate in Entra." Add-Check $checkNameGroupGate "NOT RUN" "Group is not mail-enabled — membership gate is managed in Entra, not readable here" } else { $gateGroup = $null try { $gateGroup = Get-DistributionGroup -Identity $groupIdentity -ErrorAction Stop } catch { $gateGroup = $null } if (-not $gateGroup) { Write-Warn2 "Could not re-read '$GroupName' to check its membership gate." Add-Check $checkNameGroupGate "NOT RUN" "Get-DistributionGroup returned nothing on re-read" } else { $joinR = [string]$gateGroup.MemberJoinRestriction $departR = [string]$gateGroup.MemberDepartRestriction $owners = @($gateGroup.ManagedBy | Where-Object { $_ }) $ownerNames = "(none)" if ($owners.Count -gt 0) { $ownerNames = ($owners | ForEach-Object { $_.ToString() }) -join ', ' } Write-Host " Join restriction : $joinR" -ForegroundColor DarkGray Write-Host " Depart restriction : $departR" -ForegroundColor DarkGray Write-Host " ManagedBy : $ownerNames" -ForegroundColor DarkGray $gateProblems = @() if ($joinR -ne 'Closed') { $gateProblems += "MemberJoinRestriction is '$joinR', not Closed" } if ($departR -ne 'Closed') { $gateProblems += "MemberDepartRestriction is '$departR', not Closed" } if ($owners.Count -eq 0) { $gateProblems += "ManagedBy is empty — no named owner" } if ($gateProblems.Count -eq 0) { Write-OK "Watch group membership is gated (Closed/Closed, owner: $ownerNames)" Add-Check $checkNameGroupGate "PASS" "Join/Depart Closed; ManagedBy: $ownerNames" } else { Write-Fail "Watch group membership gate is weaker than the boundary it protects." foreach ($p in $gateProblems) { Write-Host " - $p" -ForegroundColor Red } Write-Warn2 "Adding a mailbox to this group grants an application read access to that" Write-Warn2 "person's mail. Nothing was changed — set it deliberately:" Write-Host " Set-DistributionGroup -Identity '$GroupName' -MemberJoinRestriction Closed ``" Write-Host " -MemberDepartRestriction Closed -ManagedBy " Add-Check $checkNameGroupGate "FAIL" ($gateProblems -join '; ') } } } # ────────────────────────────────────────────────────────────────────────── # Step 6b — Mailbox membership # ────────────────────────────────────────────────────────────────────────── $membershipChangedNow = $false $memberAddresses = @() $mailboxResolved = $null if ($groupObj -and $groupIsMailEnabled) { # The outer @() is load-bearing: a group with zero members must yield an # empty ARRAY, not $null, or the .Count guards below silently misbehave. $memberAddresses = @( Get-DistributionGroupMember -Identity $groupIdentity -ErrorAction SilentlyContinue | Select-Object -ExpandProperty PrimarySmtpAddress | Where-Object { $_ } | ForEach-Object { $_.ToString().ToLower() } ) } if (-not $Mailbox) { Write-Host " No -Mailbox supplied; group membership left as-is ($($memberAddresses.Count) member(s))." -ForegroundColor DarkGray Add-Check "Mailbox added to watch group" "NOT RUN" "No -Mailbox supplied" } elseif (-not $groupObj) { Write-Warn2 "No watch group to add '$Mailbox' to." Add-Check "Mailbox added to watch group" "NOT RUN" "Watch group is missing" } elseif (-not $groupIsMailEnabled) { Write-Warn2 "'$GroupName' is not mail-enabled — add '$Mailbox' to it in Entra, then re-run." Add-Check "Mailbox added to watch group" "NOT RUN" "Group is not mail-enabled; manage membership in Entra" } else { try { $mailboxResolved = Get-Mailbox -Identity $Mailbox -ErrorAction Stop } catch { $mailboxResolved = $null } if (-not $mailboxResolved) { Write-Fail "Mailbox '$Mailbox' not found in this tenant." Write-Warn2 "Check the address (a shared-mailbox conversion keeps the object but can change the UPN)." Add-Check "Mailbox added to watch group" "FAIL" "Mailbox '$Mailbox' not found" } else { $mbxAddr = $mailboxResolved.PrimarySmtpAddress.ToString() if ($memberAddresses -contains $mbxAddr.ToLower()) { Write-OK "$mbxAddr is already a member of $GroupName" Add-Check "Mailbox added to watch group" "PASS" "$mbxAddr already a member" } elseif ($noChanges) { Write-Fail "$mbxAddr is NOT a member of $GroupName. -VerifyOnly: not adding it." Add-Check "Mailbox added to watch group" "FAIL" "$mbxAddr not a member" } elseif ($PSCmdlet.ShouldProcess($mbxAddr, "Add-DistributionGroupMember -Identity $GroupName")) { try { Add-DistributionGroupMember -Identity $groupIdentity -Member $mbxAddr -ErrorAction Stop Write-OK "Added $mbxAddr to $GroupName" Write-Warn2 "Exchange caches this membership evaluation. Expect 30 minutes to 2 hours" Write-Warn2 "before the authorization tests below can see it." $membershipChangedNow = $true $provisioningChangedNow = $true $memberAddresses += $mbxAddr.ToLower() Add-Check "Mailbox added to watch group" "PASS" "$mbxAddr added during this run" } catch { Write-Fail "Could not add $mbxAddr to $GroupName : $($_.Exception.Message)" Add-Check "Mailbox added to watch group" "FAIL" $_.Exception.Message } } else { Add-Check "Mailbox added to watch group" "NOT RUN" "Skipped (-WhatIf)" } } } # ────────────────────────────────────────────────────────────────────────── # Step 7 — Management scope bound to the group's DISTINGUISHED NAME # ────────────────────────────────────────────────────────────────────────── # MemberOfGroup takes the group's DISTINGUISHED NAME. An SMTP address, alias or # GUID is either rejected or, worse, accepted into a filter that matches # nothing — which produces a scope that fences everything out and looks like a # propagation delay. # # MemberOfGroup matches DIRECT MEMBERS ONLY. Nesting a group inside the watch # group adds nobody to the scope. If you need to watch ten mailboxes, ten # mailbox objects go in this group. $groupDn = $null $expectedFilter = $null $scopeReady = $false $scopeCreatedNow = $false if ($groupObj) { try { $groupDn = (Get-Group -Identity $groupIdentity -ErrorAction Stop).DistinguishedName } catch { $groupDn = $null } } if (-not $groupDn) { Write-Fail "Could not resolve the distinguished name for group '$GroupName'." Write-Warn2 "The management scope filter cannot be built without it." Add-Check "Management scope bound to the group" "FAIL" "Group distinguished name unresolved" } else { Write-OK "Group distinguished name resolved" Write-Host " $groupDn" -ForegroundColor DarkGray # OPATH string values are single-quoted, and an apostrophe inside the value # terminates it. -GroupName is operator-supplied and becomes the CN of this # DN, so a group called "O'Brien Watch" would otherwise emit an unbalanced # filter that New-ManagementScope rejects. Doubling is the OPATH escape. # Commas and equals signs need no escaping — OPATH treats the quoted value # as opaque. $escapedGroupDn = $groupDn -replace "'", "''" $expectedFilter = "MemberOfGroup -eq '$escapedGroupDn'" $existingScope = Get-ManagementScope -Identity $ScopeName -ErrorAction SilentlyContinue | Select-Object -First 1 if ($existingScope) { $actualFilter = "" if ($existingScope.RecipientFilter) { $actualFilter = $existingScope.RecipientFilter.ToString() } # Compare the WHOLE filter, normalised — never a substring, and never # -like (where [, ], * and ? in a group name are wildcard metacharacters # that make a correct filter compare FALSE). # # A substring test accepts a filter that is a strict SUPERSET of the # intended one, e.g. # MemberOfGroup -eq '' -or MemberOfGroup -eq '' # That names the watch group and mentions MemberOfGroup, so it passes — # and the Mail.Read assignment below then binds to the WIDER fence, which # is precisely the outcome this check exists to prevent. Any additional # clause is therefore treated as a mismatch, not as a superset that # "includes" what we wanted. $normActual = ConvertTo-NormalisedFilter $actualFilter $normExpected = ConvertTo-NormalisedFilter $expectedFilter # Exchange may store the DN with the apostrophes un-doubled, so accept # the unescaped rendering of the same single clause as equivalent. $normExpectedRaw = ConvertTo-NormalisedFilter ("MemberOfGroup -eq '$groupDn'") $filterExtras = @() if ($normActual -match '(^|\s)-(or|and|ne|not[a-z]*)(\s|$)') { $filterExtras += "contains an additional operator (-or / -and / -ne / -not*)" } $memberOfGroupTerms = ([regex]::Matches($normActual, 'memberofgroup')).Count if ($memberOfGroupTerms -gt 1) { $filterExtras += "contains $memberOfGroupTerms MemberOfGroup terms; exactly one is expected" } $filterMatches = $false if ($normActual -and $filterExtras.Count -eq 0) { if (($normActual -eq $normExpected) -or ($normActual -eq $normExpectedRaw)) { $filterMatches = $true } } if ($filterMatches) { Write-OK "Management scope '$ScopeName' already exists and is bound to '$GroupName'" $scopeReady = $true Add-Check "Management scope bound to the group" "PASS" "$ScopeName -> $GroupName (whole filter matches exactly)" } else { Write-Host "" Write-Fail "Management scope '$ScopeName' EXISTS BUT DOES NOT MATCH THE EXPECTED FILTER." Write-Host " Expected : $expectedFilter" -ForegroundColor Red Write-Host " Actual : $actualFilter" -ForegroundColor Red foreach ($x in $filterExtras) { Write-Host " Note : the existing filter $x" -ForegroundColor Red } Write-Host "" Write-Warn2 "This is a security problem, not a cosmetic one. If a role assignment is bound to" Write-Warn2 "this scope, the application's mailbox access is fenced by whatever that filter" Write-Warn2 "actually selects — which is not the watch group, and may be far wider." if ($filterExtras.Count -gt 0) { Write-Warn2 "A filter that NAMES the watch group and then widens with another clause is the" Write-Warn2 "dangerous case: it looks right at a glance and fences a larger set of mailboxes." } Write-Warn2 "Nothing was changed. Decide deliberately, then do ONE of:" Write-Host " # (a) inspect it, confirm nothing else relies on it, then repoint it:" Write-Host " Connect-ExchangeOnline" Write-Host " Get-ManagementRoleAssignment | Where-Object { `$_.CustomResourceScope -eq '$ScopeName' }" Write-Host " Set-ManagementScope -Identity '$ScopeName' -RecipientRestrictionFilter `"$expectedFilter`"" Write-Host " # (b) leave it alone and use a different scope name:" Write-Host " .\Setup-TATERDepartureWatch.ps1 -AppId $AppId -ScopeName 'TATER-DepartureWatch-2'" Write-Host "" $scopeFailDetail = "Existing scope '$ScopeName' filter is not exactly the expected single MemberOfGroup clause" if ($filterExtras.Count -gt 0) { $scopeFailDetail = "Existing scope '$ScopeName' filter is WIDER than expected: " + ($filterExtras -join '; ') } Add-Check "Management scope bound to the group" "FAIL" $scopeFailDetail } } elseif ($noChanges) { Write-Fail "Management scope '$ScopeName' does not exist. -VerifyOnly: not creating it." Add-Check "Management scope bound to the group" "FAIL" "Scope '$ScopeName' not found" } elseif ($PSCmdlet.ShouldProcess($ScopeName, "New-ManagementScope -RecipientRestrictionFilter MemberOfGroup")) { Write-Step "Creating management scope '$ScopeName' (MemberOfGroup — DIRECT members only)…" try { New-ManagementScope -Name $ScopeName -RecipientRestrictionFilter $expectedFilter -ErrorAction Stop | Out-Null Write-OK "Created management scope '$ScopeName'" $scopeReady = $true $scopeCreatedNow = $true $provisioningChangedNow = $true Add-Check "Management scope bound to the group" "PASS" "Created $ScopeName during this run" } catch { Write-Fail "Management scope creation failed: $($_.Exception.Message)" Write-Warn2 "Run it manually as an Exchange Administrator, then re-run this script:" Write-Host " Connect-ExchangeOnline" Write-Host " Enable-OrganizationCustomization # one-time; skip if already enabled" Write-Host " New-ManagementScope -Name '$ScopeName' -RecipientRestrictionFilter `"$expectedFilter`"" Add-Check "Management scope bound to the group" "FAIL" $_.Exception.Message } } else { Add-Check "Management scope bound to the group" "NOT RUN" "Skipped (-WhatIf)" } Write-Host " Note: MemberOfGroup matches DIRECT members only — nested groups do NOT expand." -ForegroundColor DarkGray } # ────────────────────────────────────────────────────────────────────────── # Step 8 — Exchange service principal + scoped 'Application Mail.Read' # ────────────────────────────────────────────────────────────────────────── # The Exchange service principal object is distinct from the Entra enterprise # application, but it is created FROM it: -ObjectId takes the Entra ENTERPRISE # APPLICATION object id resolved in Step 3. # # The role is the READ-ONLY "Application Mail.Read". Not Mail.ReadWrite, not # Mail.Send. Departure Watch reads metadata; it has no reason to be able to # modify or send anything from the mailbox, and an assignment that could would # survive long after the watch window closed. # # Idempotent + best-effort in the sense that a failure here leaves the group and # scope in place — but it is NOT advisory: without this assignment the app # cannot read the mailbox at all, so a failure is reported as FAIL, loudly. $assignmentReady = $false $assignmentCreatedNow = $false $roleName = "Application Mail.Read" # Printed by BOTH Step 8 catch blocks. Every command here is runnable in the # still-open Exchange session, and every OPATH filter it prints is the ESCAPED # one actually used above — copy-pasting an unescaped DN would fail for exactly # the operator whose group name contains an apostrophe. function Show-ExchangeRbacRemediation { Write-Warn2 "Until this succeeds, every metadata read will 403 with" Write-Warn2 "`"[RAOP] : Blocked by tenant configured AppOnly AccessPolicy settings.`"" Write-Warn2 "Run these manually as an Exchange Administrator:" Write-Host " Connect-ExchangeOnline" Write-Host " Enable-OrganizationCustomization # one-time; skip if already enabled" $spLine = " New-ServicePrincipal -AppId $AppId -ObjectId $spObjectId" if ($appDisplayName) { $spLine = "$spLine -DisplayName '$appDisplayName'" } Write-Host $spLine if ($expectedFilter) { Write-Host " New-ManagementScope -Name '$ScopeName' -RecipientRestrictionFilter `"$expectedFilter`"" } else { Write-Host " New-ManagementScope -Name '$ScopeName' -RecipientRestrictionFilter `"MemberOfGroup -eq ''`"" } Write-Host " New-ManagementRoleAssignment -App $AppId -Role '$roleName' -CustomResourceScope '$ScopeName'" } if (-not $spObjectId) { Write-Fail "No service principal object id — cannot register the Exchange service principal." Add-Check "Exchange service principal registered" "FAIL" "No enterprise application object id" Add-Check "Role '$roleName' scoped to '$ScopeName'" "NOT RUN" "Service principal unresolved" } elseif (-not $scopeReady) { Write-Warn2 "Management scope is not in place — skipping the role assignment." Write-Warn2 "Assigning the role without a working scope would grant UNSCOPED mailbox read." Add-Check "Exchange service principal registered" "NOT RUN" "Management scope not in place" Add-Check "Role '$roleName' scoped to '$ScopeName'" "NOT RUN" "Management scope not in place — refusing to assign unscoped" } else { # The service-principal registration and the role assignment get SEPARATE # try/catch blocks. Wrapped in one, a throw from New-ServicePrincipal jumps # past the "Exchange service principal registered" Add-Check entirely and # the summary silently renders one row fewer — the counts omit it, and a # vanished row is worse than a NOT RUN row, because NOT RUN is visible and # counted while absence is not. $spStepThrew = $false try { # 1. Register the app's Exchange service principal (distinct object from # the Entra enterprise application, created from its object id). $exoSp = Get-ServicePrincipal -Identity $AppId -ErrorAction SilentlyContinue if ($exoSp) { Write-OK "Exchange service principal already registered" Add-Check "Exchange service principal registered" "PASS" "Already present" } elseif ($noChanges) { Write-Fail "Exchange service principal is not registered. -VerifyOnly: not registering it." Add-Check "Exchange service principal registered" "FAIL" "Not registered" } elseif ($PSCmdlet.ShouldProcess($AppId, "New-ServicePrincipal")) { Write-Step "Registering the Exchange service principal (ObjectId = ENTERPRISE APPLICATION object id)…" # -DisplayName is omitted when Graph never told us the app's name. # Writing a placeholder into the tenant would name the object # something that matches no app registration in the directory. $newSpParams = @{ AppId = $AppId; ObjectId = $spObjectId; ErrorAction = 'Stop' } if ($appDisplayName) { $newSpParams['DisplayName'] = $appDisplayName } New-ServicePrincipal @newSpParams | Out-Null Write-OK "Registered Exchange service principal (ObjectId $spObjectId)" $provisioningChangedNow = $true Add-Check "Exchange service principal registered" "PASS" "Registered during this run" } else { Add-Check "Exchange service principal registered" "NOT RUN" "Skipped (-WhatIf)" } } catch { $spStepThrew = $true Write-Fail "Exchange service principal registration failed: $($_.Exception.Message)" Write-Warn2 "If the error says `"Couldn't find a service principal`", the -ObjectId below is" Write-Warn2 "an app REGISTRATION object id rather than an ENTERPRISE APPLICATION object id." Add-Check "Exchange service principal registered" "FAIL" $_.Exception.Message } if ($spStepThrew) { Write-Warn2 "Not evaluating the role assignment — the service principal step errored, so any" Write-Warn2 "result would describe a half-provisioned state." Add-Check "Role '$roleName' scoped to '$ScopeName'" "NOT RUN" "Exchange service principal step errored — role assignment not evaluated" Show-ExchangeRbacRemediation } else { try { # 2. Assign the read-only role, scoped. The idempotency check filters on # BOTH the assignee and the scope — matching on role alone would # false-positive against a different application's assignment. $existingAssignment = Get-ManagementRoleAssignment -Role $roleName -ErrorAction SilentlyContinue | Where-Object { $_.RoleAssigneeName -eq $spObjectId -and $_.CustomResourceScope -eq $ScopeName } if ($existingAssignment) { Write-OK "Role '$roleName' already assigned (scope $ScopeName)" $assignmentReady = $true Add-Check "Role '$roleName' scoped to '$ScopeName'" "PASS" "Already assigned" } elseif ($noChanges) { Write-Fail "Role '$roleName' is not assigned. -VerifyOnly: not assigning it." Add-Check "Role '$roleName' scoped to '$ScopeName'" "FAIL" "Not assigned" } elseif ($PSCmdlet.ShouldProcess("$AppId -> $ScopeName", "New-ManagementRoleAssignment -Role '$roleName'")) { New-ManagementRoleAssignment -App $AppId -Role $roleName -CustomResourceScope $ScopeName -ErrorAction Stop | Out-Null Write-OK "Assigned '$roleName' (scope $ScopeName)" Write-OK "Allow up to 30 minutes to propagate on default-deny tenants." $assignmentReady = $true $assignmentCreatedNow = $true $provisioningChangedNow = $true Add-Check "Role '$roleName' scoped to '$ScopeName'" "PASS" "Assigned during this run" } else { Add-Check "Role '$roleName' scoped to '$ScopeName'" "NOT RUN" "Skipped (-WhatIf)" } } catch { Write-Fail "Exchange RBAC grant failed: $($_.Exception.Message)" Add-Check "Role '$roleName' scoped to '$ScopeName'" "FAIL" $_.Exception.Message Show-ExchangeRbacRemediation } } } # ────────────────────────────────────────────────────────────────────────── # Step 8b — EVERY Exchange role assignment this application holds # ────────────────────────────────────────────────────────────────────────── # A real union path into wider mailbox access, and it costs one cmdlet in a # session the script already holds. Exchange evaluates the SUM of an app's role # assignments, so a mailbox-data role assigned to this app with NO # CustomResourceScope — or with a scope wider than the one provisioned above — # widens the fence regardless of how tightly that one is drawn. This is a # union BETWEEN TWO EXCHANGE ASSIGNMENTS, which is genuine; it is not the # Entra-grant-overrides-Exchange-scope claim that Step 10 documents as false. # Until this check existed it was caught only by the OPTIONAL negative test. if (-not $spObjectId) { Add-Check $checkNameOtherRoles "NOT RUN" "Service principal unresolved — assignments not enumerated" } else { Write-Step "Enumerating every Exchange management role assignment held by this application…" try { $exoSpForApp = @(Get-ServicePrincipal -ErrorAction Stop | Where-Object { $_.AppId -eq $AppId }) if ($exoSpForApp.Count -eq 0) { throw "No Exchange service principal is registered for AppId $AppId — there are no assignments to enumerate." } $appSpObjectIds = @($exoSpForApp | ForEach-Object { [string]$_.ObjectId }) $appAssignments = @(Get-ManagementRoleAssignment -ErrorAction Stop | Where-Object { $_.RoleAssigneeType -eq 'ServicePrincipal' -and $appSpObjectIds -contains [string]$_.RoleAssignee }) if ($appAssignments.Count -eq 0) { Write-Host " No Exchange management role assignments are held by this application." -ForegroundColor DarkGray } # "Application " is the Exchange app-only MAILBOX DATA role # family (Application Mail.Read / Mail.ReadWrite / Mail.Send / # Calendars.* / Contacts.* / MailboxSettings.* ...). Those are the ones # whose scoping determines which mailboxes this app can touch. $unscopedMailboxRoles = @() $otherUnscopedRoles = @() foreach ($ra in $appAssignments) { $raRole = [string]$ra.Role $raScope = [string]$ra.CustomResourceScope $raScopeLabel = $raScope if (-not $raScope) { $raScopeLabel = "(none — UNSCOPED)" } Write-Host " Role: $raRole" -ForegroundColor DarkGray Write-Host " CustomResourceScope : $raScopeLabel" -ForegroundColor DarkGray Write-Host " RecipientReadScope : $($ra.RecipientReadScope)" -ForegroundColor DarkGray if ($raScope -eq $ScopeName) { continue } $isMailboxDataRole = ($raRole -like 'Application *') if ($isMailboxDataRole) { $unscopedMailboxRoles += "$raRole -> $raScopeLabel" } else { $otherUnscopedRoles += "$raRole -> $raScopeLabel" } } if ($unscopedMailboxRoles.Count -gt 0) { Write-Host "" Write-Fail "MAILBOX-DATA ROLE ASSIGNMENT OUTSIDE '$ScopeName' — the scope is not effective." foreach ($u in $unscopedMailboxRoles) { Write-Host " $u" -ForegroundColor Red } Write-Host "" Write-Warn2 "A second mailbox-data assignment UNIONS with the scoped one. The application's" Write-Warn2 "reach is the sum of its assignments, not the narrowest of them." Write-Warn2 "Inspect each one and decide with a human in the loop — this script removes nothing:" Write-Host " Get-ManagementRoleAssignment | ? RoleAssigneeType -eq 'ServicePrincipal' | Format-List Name,Role,RoleAssignee,CustomResourceScope,RecipientReadScope,RecipientWriteScope" Write-Host "" Add-Check $checkNameOtherRoles "FAIL" ("Mailbox-data role(s) outside '$ScopeName': " + ($unscopedMailboxRoles -join '; ')) } elseif ($appAssignments.Count -eq 0 -and $assignmentReady) { # An empty list is not evidence of safety. We KNOW an assignment for # this app exists — Step 8 just confirmed or created it — so an # enumeration returning nothing means the enumeration did not see # what is there, and a clean-looking result would be a lie. Write-Fail "ENUMERATION RETURNED NOTHING, yet Step 8 confirmed an assignment for this app." Write-Warn2 "The empty result is inconsistent with known state, so it clears nothing. Check:" Write-Host " Get-ManagementRoleAssignment | ? RoleAssigneeType -eq 'ServicePrincipal' | Format-List Name,Role,RoleAssignee,CustomResourceScope,RecipientReadScope" Add-Check $checkNameOtherRoles "NOT RUN" "Enumeration returned 0 assignments while a known assignment exists — result not trustworthy" } else { $otherDetail = "no mailbox-data role assignment outside '$ScopeName'" if ($appAssignments.Count -eq 0) { $otherDetail = "0 assignments returned; this app holds no Exchange management role assignment at all" } if ($otherUnscopedRoles.Count -gt 0) { Write-Warn2 "This app also holds non-mailbox-data Exchange role assignment(s). They do not" Write-Warn2 "grant mailbox content access, but review them — an app that can run Exchange" Write-Warn2 "management cmdlets can create its own scopes and assignments:" foreach ($o in $otherUnscopedRoles) { Write-Host " $o" -ForegroundColor Yellow } $otherDetail += "; also holds (review): " + ($otherUnscopedRoles -join '; ') } else { Write-OK "No Exchange role assignment on this app grants mailbox access outside '$ScopeName'." } Add-Check $checkNameOtherRoles "PASS" ("$($appAssignments.Count) assignment(s) examined; " + $otherDetail) } } catch { Write-Fail "ENUMERATION NOT PERFORMED — $($_.Exception.Message)" Write-Warn2 "A second, wider mailbox-data assignment on this app would be undetected. Run:" Write-Host " Get-ManagementRoleAssignment | ? RoleAssigneeType -eq 'ServicePrincipal' | Format-List Name,Role,RoleAssignee,CustomResourceScope,RecipientReadScope" Add-Check $checkNameOtherRoles "NOT RUN" "Role-assignment enumeration errored: $($_.Exception.Message)" } } # ────────────────────────────────────────────────────────────────────────── # Step 9 — VERIFICATION: positive and negative authorization tests # ────────────────────────────────────────────────────────────────────────── # This section is the point of the script. Provisioning proves nothing on its # own: a scope that permits everything and a scope that permits exactly one # mailbox both make the positive test pass. Only the negative test distinguishes # them. Write-Host "" Write-Host "────────────────────────────────────────────────────────────────────────────" -ForegroundColor Cyan Write-Host " Verification" -ForegroundColor Cyan Write-Host "────────────────────────────────────────────────────────────────────────────" -ForegroundColor Cyan $testCmdAvailable = $null -ne (Get-Command Test-ServicePrincipalAuthorization -ErrorAction SilentlyContinue) $positiveInScope = $false $positiveRan = $false $positiveSubject = $null if (-not $testCmdAvailable) { Write-Fail "Test-ServicePrincipalAuthorization is not available in this session." Write-Warn2 "Update the Exchange module and re-run — neither test can be performed without it:" Write-Host " Update-Module ExchangeOnlineManagement -Force" Add-Check $checkNamePositive "NOT RUN" "Test-ServicePrincipalAuthorization not available" Add-Check $checkNameNegative "NOT RUN" "Test-ServicePrincipalAuthorization not available" } else { # ---- 9a. POSITIVE test ------------------------------------------------ if ($mailboxResolved) { $positiveSubject = $mailboxResolved.PrimarySmtpAddress.ToString() } elseif ($memberAddresses.Count -gt 0) { $positiveSubject = $memberAddresses[0] } if (-not $positiveSubject) { Write-Warn2 "No in-scope mailbox to test with — the watch group has no members." Write-Warn2 "This is a legitimate state (the fence exists before anyone is inside it)," Write-Warn2 "but it means access has NOT been shown to work. Re-run with -Mailbox ." Add-Check $checkNamePositive "NOT RUN" "Watch group has no member mailbox to test" } else { Write-Step "POSITIVE test — $positiveSubject should be IN scope…" try { $posResult = Test-ServicePrincipalAuthorization -Identity $AppId -Resource $positiveSubject -ErrorAction Stop $positiveRan = $true $posInScopeRows = @($posResult | Where-Object { $_.InScope -eq $true }) if ($posInScopeRows.Count -gt 0) { $positiveInScope = $true $grantedRoles = ($posInScopeRows | ForEach-Object { $_.RoleName }) -join ', ' Write-OK "POSITIVE test: $positiveSubject → InScope True (expected). Roles: $grantedRoles" Add-Check $checkNamePositive "PASS" "$positiveSubject in scope via: $grantedRoles" } elseif ($provisioningChangedNow) { # This run changed something that Exchange caches. A negative # result now is the EXPECTED reading, not evidence of a fault. Write-Warn2 "POSITIVE test: $positiveSubject → InScope False." Write-Warn2 "This run created or changed the group membership / scope / role assignment," Write-Warn2 "and Exchange caches those evaluations for 30 minutes to 2 hours." Write-Warn2 "A False result right now is EXPECTED and does NOT mean the scope is broken." Write-Warn2 "Re-check in 30-120 minutes:" Write-Host " Connect-ExchangeOnline" Write-Host " Test-ServicePrincipalAuthorization -Identity $AppId -Resource $positiveSubject # expect InScope True" Add-Check $checkNamePositive "NOT RUN" "Inconclusive — provisioning changed this run; membership cache is 30 min - 2 h" } else { Write-Fail "POSITIVE test: $positiveSubject → InScope False." Write-Warn2 "Nothing changed during this run, so propagation does not explain this." Write-Warn2 "The application cannot read the mailbox it is supposed to watch." Write-Warn2 "Check that $positiveSubject is a DIRECT member of '$GroupName' (nested groups do not count)," Write-Warn2 "and that the role assignment's CustomResourceScope is '$ScopeName'." Add-Check $checkNamePositive "FAIL" "$positiveSubject not in scope and nothing changed this run" } } catch { Write-Fail "POSITIVE test errored: $($_.Exception.Message)" Add-Check $checkNamePositive "NOT RUN" "Cmdlet error: $($_.Exception.Message)" } } # ---- 9b. NEGATIVE test ------------------------------------------------ # A positive result alone is compatible with the app being able to read # every mailbox in the tenant. The negative test is the Exchange-side # evidence that the scope BOUNDS anything. Note what it is: the cmdlet # reports the STORED RBAC CONFIGURATION, not enforced state, and it says # nothing about whether this tenant enforces RBAC for Applications in the # first place. Only the empirical app-only read in Step 10c settles that. if (-not $NegativeTestMailbox) { Write-Host "" Write-Fail "NEGATIVE TEST NOT RUN — boundedness has NOT been demonstrated." Write-Host "" Write-Warn2 "The checks above can show that the application CAN read the watched mailbox." Write-Warn2 "They cannot show that it CANNOT read anything else. An app with tenant-wide" Write-Warn2 "mailbox read passes every positive test in this script, identically." Write-Warn2 "Run this now against any licensed mailbox that is NOT in '$GroupName':" Write-Host " Connect-ExchangeOnline" Write-Host " Test-ServicePrincipalAuthorization -Identity $AppId -Resource someone@yourdomain.com # expect InScope False" Write-Warn2 "Or re-run this script with -NegativeTestMailbox ." Write-Host "" Add-Check $checkNameNegative "NOT RUN" "-NegativeTestMailbox not supplied; boundedness unproven" } else { # RESOLVE FIRST, THEN COMPARE. $memberAddresses holds resolved PRIMARY # SMTP addresses, and -NegativeTestMailbox is any accepted identity: an # alias, a UPN, a display name, a GUID or a secondary SMTP address. A # raw-string comparison therefore misses a genuine group member supplied # by any of those forms, runs the "negative" test against a mailbox that # is in scope BY DESIGN, and prints the full active-exposure alarm on a # tenant that is correctly fenced. This script already knows identities # are not primary SMTP addresses — it warns about exactly that when a # shared-mailbox conversion changes a UPN. $negMailbox = $null try { $negMailbox = Get-Mailbox -Identity $NegativeTestMailbox -ErrorAction Stop } catch { $negMailbox = $null } $negAddr = $null $negIsMember = $false if ($negMailbox) { $negAddr = $negMailbox.PrimarySmtpAddress.ToString() $negIsMember = $memberAddresses -contains $negAddr.ToLower() } if (-not $negMailbox) { Write-Warn2 "NEGATIVE test subject '$NegativeTestMailbox' has no mailbox in this tenant." Write-Warn2 "That is INCONCLUSIVE, not a pass — a non-existent mailbox is out of scope for" Write-Warn2 "every application, so the result would prove nothing. Pick a real, licensed" Write-Warn2 "mailbox that is not in '$GroupName' and re-run." Add-Check $checkNameNegative "NOT RUN" "'$NegativeTestMailbox' is not a mailbox — inconclusive" } elseif ($negIsMember) { Write-Warn2 "NEGATIVE test subject '$NegativeTestMailbox' resolves to $negAddr, which IS a" Write-Warn2 "member of '$GroupName'. It is in scope by design, so testing it proves nothing" Write-Warn2 "about boundedness. Pick a mailbox outside the group and re-run." Add-Check $checkNameNegative "NOT RUN" "Subject resolves to $negAddr, a group member — invalid negative test" } else { Write-Step "NEGATIVE test — $negAddr should be OUT of scope…" try { $negResult = Test-ServicePrincipalAuthorization -Identity $AppId -Resource $negAddr -ErrorAction Stop $negInScopeRows = @($negResult | Where-Object { $_.InScope -eq $true }) if ($negInScopeRows.Count -gt 0) { $negRoles = ($negInScopeRows | ForEach-Object { $_.RoleName }) -join ', ' Write-Host "" Write-Fail "NEGATIVE TEST FAILED — $negAddr IS IN SCOPE." Write-Fail "The application can read a mailbox that is NOT in the watch group." Write-Fail "There is no effective scoping. Treat this as an active exposure." Write-Host " Roles reporting in scope: $negRoles" -ForegroundColor Red Write-Host "" Write-Warn2 "Do not proceed on the assumption that Departure Watch is bounded. Find the" Write-Warn2 "wider grant before anything reads this mailbox. Most likely causes:" Write-Warn2 " 1. A second management role assignment on the same app with no" Write-Warn2 " CustomResourceScope, or one scoped to something far wider — this is" Write-Warn2 " what the 'other Exchange role assignments' check above enumerates." Write-Warn2 " 2. A mailbox-reaching grant on the Office 365 Exchange Online resource," Write-Warn2 " e.g. full_access_as_app — see the least-privilege check below." Write-Warn2 " 3. The tenant does not enforce RBAC for Applications at all, in which" Write-Warn2 " case no Exchange management scope bounds anything and the Graph" Write-Warn2 " grant is effectively tenant-wide. The empirical read settles this." Write-Warn2 "NOTE (#2199): if this app holds a plain Graph Mail.Read grant, that IS now a" Write-Warn2 "problem in itself — the collector refuses tokens carrying it (TOKEN_ROLES_TOO_BROAD)." Write-Warn2 "The dedicated Departure Watch app runs on Mail.ReadBasic.All only." Write-Host " Get-ManagementRoleAssignment | ? RoleAssigneeType -eq 'ServicePrincipal' | Format-List Name,Role,RoleAssignee,CustomResourceScope,RecipientWriteScope" Write-Host "" Add-Check $checkNameNegative "FAIL" "$negAddr is IN SCOPE — the app is not bounded by '$ScopeName'" } elseif ($positiveInScope) { Write-OK "NEGATIVE test: $negAddr → InScope False (expected — the scope is bounding access)" Add-Check $checkNameNegative "PASS" "$negAddr out of scope while $positiveSubject is in scope" } else { # Both tests returned not-in-scope. That is exactly what a role # assignment which has not propagated at all looks like, so the # negative result carries no information about boundedness. Write-Warn2 "NEGATIVE test: $negAddr → InScope False, but the POSITIVE test did not return" Write-Warn2 "InScope True either. Everything reporting out-of-scope is indistinguishable" Write-Warn2 "from an assignment that has not propagated yet, so this does NOT demonstrate" Write-Warn2 "boundedness — it only shows nothing is readable right now." Write-Warn2 "Re-run the verification once the positive test passes:" Write-Host " .\Setup-TATERDepartureWatch.ps1 -AppId $AppId -NegativeTestMailbox $negAddr -VerifyOnly" Add-Check $checkNameNegative "NOT RUN" "Inconclusive — positive test did not pass, so a False here proves nothing" } } catch { Write-Warn2 "NEGATIVE test errored: $($_.Exception.Message)" Add-Check $checkNameNegative "NOT RUN" "Cmdlet error: $($_.Exception.Message)" } } } } Write-Host "" Write-Host " Membership cache: Exchange evaluates MemberOfGroup against a cached view." -ForegroundColor DarkGray Write-Host " A mailbox added minutes ago will legitimately test out-of-scope for 30 minutes" -ForegroundColor DarkGray Write-Host " to 2 hours. This script reports that state as NOT RUN, never as PASS or FAIL." -ForegroundColor DarkGray # ────────────────────────────────────────────────────────────────────────── # Step 10 — Entra grant audit # ────────────────────────────────────────────────────────────────────────── # THIS SECTION WAS WRONG AND WAS REWRITTEN (ADO #146). It used to FAIL the run # whenever the app held a Microsoft Graph mail application permission, on the # theory that an unscoped Entra grant UNIONS with, and therefore defeats, the # Exchange management scope. That theory is disproven by this repository's own # working code: # # Setup-TATEREmailIntake.ps1 grants its intake app TENANT-WIDE Entra Graph # application permissions (Mail.ReadWrite, Mail.Send, MailboxSettings.ReadWrite) # AND a management-scope role assignment bound to a single mailbox alias. That # configuration has run at a RAOP-enforcing tenant since 2026-06-05, correctly # scoped to one mailbox. If the Graph grant defeated the Exchange scope, that # app would have held tenant-wide mail read/write/send for months — and the # [RAOP] 403 that the whole Exchange remediation exists to fix could never have # happened, because the tenant-wide Graph grant was consented the entire time # and every read still failed until the SCOPED assignment landed. # # The corrected model, and what each part is worth: # # Graph mail-read grant REQUIRED. Without it the client-credentials token # carries no mail scope and Graph rejects the call # before Exchange RBAC is consulted — nothing is # readable. Its presence is a PREREQUISITE, not a # finding; its ABSENCE is the finding. # Exchange management scope Narrows WHICH mailboxes — where the tenant # enforces RBAC for Applications (RAOP). # Empirical negative read The ONLY thing that separates "scoped by RAOP" # from "genuinely tenant-wide": # 403 with [RAOP] -> bounded # 200 -> UNBOUNDED, the real danger # no Graph grant -> nothing readable at all # # So this audit no longer renders a mail grant's existence as an exposure. It # answers three separate questions and records three separate checks: # # 10a Is the REQUIRED Graph mail-read grant present? PASS / FAIL / NOT RUN # 10b Is anything granted beyond what a read-only PASS / FAIL / NOT RUN # feature needs? (write, send, full_access_as_app, # EWS/IMAP/POP, anything on the Exchange resource) # 10c Is access empirically bounded? always NOT RUN here — # this script holds an admin's delegated sign-in, not the application's # own credentials, so it cannot perform the read. It prints the commands. # # THIS AUDIT MUST PROVE IT RAN. An empty result set is not a clean result: if a # resource service principal fails to resolve, every assignment on it is # filtered out and the flagged list is empty for the worst possible reason. So # PASS requires positive evidence — both resource SPs resolved, a non-empty role # map, and every enumerated assignment on those resources mapped to a KNOWN role # name. Anything short of that is NOT RUN with the specific reason. Write-Host "" Write-Host "────────────────────────────────────────────────────────────────────────────" -ForegroundColor Cyan Write-Host " Entra application-permission audit (Graph + Exchange Online)" -ForegroundColor Cyan Write-Host "────────────────────────────────────────────────────────────────────────────" -ForegroundColor Cyan $readGrants = @() # Graph roles that satisfy the REQUIREMENT $excessGrants = @() # roles wider than a metadata-only reader can justify $script:collectorForbiddenGrants = @() # roles the collector itself refuses (#2199) — fatal, not merely wide # The ONE Graph application role that satisfies the requirement — and it is # deliberately no longer a family. ADO #1859/#2199: the collector demands a # DEDICATED app whose token carries envelope-only Mail.ReadBasic.All, and # assertTokenRoles (departureWatchCollectTimer.ts DW_REQUIRED_ROLES / # DW_FORBIDDEN_ROLES) REFUSES a token carrying Mail.Read, Mail.ReadWrite or # Mail.ReadWrite.All with TOKEN_ROLES_TOO_BROAD. So the roles this list used to # accept as "satisfying" (Mail.Read, Mail.ReadWrite) now make collection # guaranteed to fail — a broader grant is not a superset here, it is a # disqualifier. Matched EXACTLY, never by wildcard. $graphReadRoles = @( 'Mail.ReadBasic.All' ) # Roles the COLLECTOR ITSELF refuses (TOKEN_ROLES_TOO_BROAD). Presence of any # of these is a hard FAIL with its own message — distinct from generic # least-privilege excess, because the impact is not "wider than needed", it is # "the feature will not run at all". Mirror of DW_FORBIDDEN_ROLES in # api/src/functions/departureWatchCollectTimer.ts — keep in sync. $graphCollectorForbiddenRoles = @( 'Mail.Read', 'Mail.ReadWrite', 'Mail.ReadWrite.All' ) # Graph application roles beyond read. Least privilege still applies: a WRITE or # SEND grant on a read-only feature is a real finding however well it is scoped, # and MailboxSettings.ReadWrite is worse than it looks — it can set a forwarding # rule on the mailbox being watched. $graphExcessRoles = @( 'Mail.ReadWrite', 'Mail.ReadWrite.All', 'Mail.ReadWrite.Shared', 'Mail.Send', 'Mail.Send.Shared', 'MailboxSettings.ReadWrite' ) # Office 365 Exchange Online roles that reach mailbox data. NONE of these is # needed: the reader speaks Graph only. Anything matching here is surplus, and # full_access_as_app is the widest mailbox permission Microsoft publishes. # Wildcards are deliberate on this resource — erring toward flagging. $exchangeMailPatterns = @( 'full_access_as_app', 'EWS.*', 'IMAP.*', 'POP.*', 'Mail.*', 'MailboxItem.*', 'MailboxSettings.*' ) # Known Graph roles that are neither required by this feature nor beyond read. # Listed so an ordinary tenant does not trip the unclassified guard below — but # printed, and named in the check detail, because they are still grants nobody # asked this feature to need. MailboxSettings.Read reads mailbox configuration # (including forwarding rules); it is a read, so it is advisory, not a failure. $graphAdvisoryRoles = @( 'MailboxSettings.Read', 'Mail.ReadBasic' ) # A Graph role that looks mail-shaped but matches NONE of the lists above must # block PASS rather than fall through as harmless. Same discipline as an # unresolvable app role id: a permission we cannot classify is a permission we # cannot clear, and silence here is how an empty finding list becomes a lie. $graphMailShapedPatterns = @('Mail.*', 'MailboxItem.*', 'MailboxSettings.*') if (-not $graphConnected) { Write-Fail "AUDIT NOT PERFORMED — no Microsoft Graph session." Write-Warn2 "An unrun audit is not a clean audit. Until this runs, neither the presence of the" Write-Warn2 "REQUIRED Graph mail-read grant nor the absence of surplus grants is established." Write-Warn2 "Run it yourself:" Write-Host " az ad sp show --id $AppId --query id -o tsv" Write-Host " az rest --method GET --uri `"https://graph.microsoft.com/v1.0/servicePrincipals//appRoleAssignments`"" Write-Warn2 "Inspect grants on BOTH $graphResourceAppId (Microsoft Graph)" Write-Warn2 "and $exchangeResourceAppId (Office 365 Exchange Online)." Add-Check $checkNameGraphGrant "NOT RUN" "No Graph session — grants not enumerated" Add-Check $checkNameLeastPriv "NOT RUN" "No Graph session — grants not enumerated" } elseif (-not $spObjectId) { Write-Fail "AUDIT NOT PERFORMED — no enterprise application object id." Add-Check $checkNameGraphGrant "NOT RUN" "Service principal unresolved — grants not enumerated" Add-Check $checkNameLeastPriv "NOT RUN" "Service principal unresolved — grants not enumerated" } else { Write-Step "Auditing application-permission grants on '$appDisplayLabel'…" try { $auditResources = @( [pscustomobject]@{ Name = 'Microsoft Graph'; ResourceAppId = $graphResourceAppId } [pscustomobject]@{ Name = 'Office 365 Exchange Online'; ResourceAppId = $exchangeResourceAppId } ) $roleMap = @{} # "|" -> role value $auditedResourceNames = @{} # resourceSpId -> friendly resource name $unresolvedResources = @() foreach ($res in $auditResources) { $resSp = Get-MgServicePrincipal -Filter "appId eq '$($res.ResourceAppId)'" -ErrorAction Stop | Select-Object -First 1 if (-not $resSp) { # A filter that matches nothing does NOT throw under # -ErrorAction Stop. Record it explicitly — silence here is what # turns "examined nothing" into a clean-looking PASS. $unresolvedResources += "$($res.Name) ($($res.ResourceAppId))" continue } $auditedResourceNames[[string]$resSp.Id] = $res.Name foreach ($r in $resSp.AppRoles) { $roleMap["$($resSp.Id)|$($r.Id)"] = $r.Value } } $assignments = @(Get-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $spObjectId -All -ErrorAction Stop) $auditedAssignments = @($assignments | Where-Object { $auditedResourceNames.ContainsKey([string]$_.ResourceId) }) $otherAssignments = @($assignments | Where-Object { -not $auditedResourceNames.ContainsKey([string]$_.ResourceId) }) Write-Host " $($assignments.Count) application-permission grant(s) total; $($auditedAssignments.Count) on an audited mailbox resource." -ForegroundColor DarkGray $unmappedRoleIds = @() $unclassifiedRoles = @() $advisoryGrants = @() $manageAsAppGrants = @() Write-Host "" Write-Host " Grants on the audited resources:" -ForegroundColor DarkGray if ($auditedAssignments.Count -eq 0) { Write-Host " (none)" -ForegroundColor DarkGray } foreach ($a in $auditedAssignments) { $resourceName = $auditedResourceNames[[string]$a.ResourceId] $isGraphResource = ($resourceName -eq 'Microsoft Graph') $roleKey = "$($a.ResourceId)|$($a.AppRoleId)" $roleValue = $null if ($roleMap.ContainsKey($roleKey)) { $roleValue = $roleMap[$roleKey] } if (-not $roleValue) { # Do NOT relabel this into a string that can never match the # patterns and then move on. An app role id we cannot resolve is # an app role we cannot clear, and it must block PASS. $unmappedRoleIds += "$resourceName / $($a.AppRoleId)" Write-Host " $resourceName : (UNRESOLVED app role $($a.AppRoleId))" -ForegroundColor Yellow continue } $grantRecord = [pscustomobject]@{ Resource = $resourceName Role = $roleValue AssignmentId = $a.Id CreatedOn = $a.CreatedDateTime Why = "" } # Classify. ADO #2199: collector-forbidden roles are checked FIRST # and win — Mail.ReadWrite used to be recorded as "satisfies the # read requirement AND is excess", which let a broad grant half-pass. # Under the dedicated-app contract it is neither: assertTokenRoles # refuses the token outright, so the feature cannot run. $isRead = $false $isExcess = $false $isAdvisory = $false $isCollectorForbidden = $false if ($isGraphResource) { if ($graphCollectorForbiddenRoles -contains $roleValue) { $isCollectorForbidden = $true $script:collectorForbiddenGrants += $grantRecord $grantRecord.Why = "the Departure Watch collector REFUSES tokens carrying this role (TOKEN_ROLES_TOO_BROAD) — collection cannot run while this grant exists on the dedicated app" } if ($graphReadRoles -contains $roleValue) { $isRead = $true } if ($graphExcessRoles -contains $roleValue) { $isExcess = $true if (-not $grantRecord.Why) { $grantRecord.Why = "grants more than read on a read-only feature" } } if ($graphAdvisoryRoles -contains $roleValue) { $isAdvisory = $true } if (-not $isRead -and -not $isExcess -and -not $isAdvisory) { # Mail-shaped but unrecognised. Must block PASS, not pass by # falling off the end of the list. foreach ($pat in $graphMailShapedPatterns) { if ($roleValue -like $pat) { $unclassifiedRoles += "$resourceName / $roleValue" Write-Host " $resourceName : $roleValue (UNCLASSIFIED mail-shaped role)" -ForegroundColor Yellow break } } } } else { # Office 365 Exchange Online. The reader speaks Graph only, so # every mailbox-reaching role here is surplus by construction. foreach ($pat in $exchangeMailPatterns) { if ($roleValue -like $pat) { $isExcess = $true $grantRecord.Why = "mailbox access via the Exchange resource; this feature reads through Graph and needs nothing here" if ($roleValue -eq 'full_access_as_app') { $grantRecord.Why = "full_access_as_app — full access to EVERY mailbox in the tenant" } break } } } if ($isRead) { $readGrants += $grantRecord } if ($isExcess) { $excessGrants += $grantRecord } if ($isAdvisory) { $advisoryGrants += "$resourceName / $roleValue" } $label = " $resourceName : $roleValue" if ($isCollectorForbidden) { Write-Host "$label [FATAL — the collector refuses tokens carrying this role]" -ForegroundColor Red } elseif ($isRead -and -not $isExcess) { Write-Host "$label [satisfies the REQUIRED read grant]" -ForegroundColor Green } elseif ($isExcess) { Write-Host "$label [BEYOND least privilege]" -ForegroundColor Red } elseif ($isAdvisory) { Write-Host "$label [read-only, but not needed by this feature — review]" -ForegroundColor Yellow } elseif ($roleValue -like 'Exchange.ManageAsApp*') { $manageAsAppGrants += "$resourceName / $roleValue" Write-Host $label -ForegroundColor DarkGray } else { Write-Host $label -ForegroundColor DarkGray } } if ($otherAssignments.Count -gt 0) { Write-Host "" Write-Host " Grants on other resources (not a mailbox-access route; listed for completeness):" -ForegroundColor DarkGray foreach ($a in $otherAssignments) { $otherResName = [string]$a.ResourceDisplayName if (-not $otherResName) { $otherResName = "(resource $($a.ResourceId))" } Write-Host " $otherResName : app role $($a.AppRoleId)" -ForegroundColor DarkGray } } Write-Host "" if ($manageAsAppGrants.Count -gt 0) { Write-Warn2 "This app also holds Exchange.ManageAsApp: $($manageAsAppGrants -join ', ')" Write-Warn2 "That is a SELF-ESCALATION path, not a mailbox grant — an app that can run" Write-Warn2 "Exchange management cmdlets can create its own management scopes and role" Write-Warn2 "assignments, and therefore widen the fence this script just built. It is" Write-Warn2 "expected on a shared compliance-scanning registration; know that it is there." Write-Warn2 "" Write-Warn2 "This BLOCKS the 'configured as bounded' verdict below. A fence the" Write-Warn2 "application can move is not a fence, however correct it looks right now." # ADO #1857 review. Previously advisory only: an app holding this scored # "least privilege — PASS" and, with the other checks green, printed # "CONFIGURED AS BOUNDED" — for an application the script itself says can # rewrite that boundary. Deliberately NOT folded into the least-privilege # check: this is not a mailbox grant and calling it one would misname the # risk. It is a separate fact that invalidates the bounded CLAIM. $script:appCanWidenFence = $true } # Reasons the audit cannot claim to have run. Collected, not inferred. $auditBlockers = @() if ($unresolvedResources.Count -gt 0) { $auditBlockers += "resource service principal(s) not resolved in this tenant: " + ($unresolvedResources -join ', ') } if ($roleMap.Count -eq 0) { $auditBlockers += "the app-role map is EMPTY — no role name could be resolved for any grant" } if ($unmappedRoleIds.Count -gt 0) { $auditBlockers += "app role id(s) not present in the resource's AppRoles: " + ($unmappedRoleIds -join ', ') } if ($unclassifiedRoles.Count -gt 0) { $auditBlockers += "mail-shaped Graph role(s) this script cannot classify: " + ($unclassifiedRoles -join ', ') } # ---- 10a. Is the REQUIRED Graph mail-read grant present? ----------- # This is the check whose verdict used to be inverted. A Graph mail-read # grant is what makes the token carry mail scope; without it Graph 403s # before Exchange RBAC is consulted and NO mailbox is readable. Write-Host "" if ($auditBlockers.Count -gt 0) { Write-Fail "REQUIRED-GRANT CHECK INCOMPLETE — the enumeration cannot be trusted." foreach ($b in $auditBlockers) { Write-Warn2 " - $b" } Add-Check $checkNameGraphGrant "NOT RUN" ("Enumeration incomplete: " + ($auditBlockers -join '; ')) } elseif ($script:collectorForbiddenGrants.Count -gt 0) { # ADO #2199 — this branch outranks "required grant present". A token # carrying Mail.Read alongside Mail.ReadBasic.All is refused just as # hard as one without the required role: assertTokenRoles fails it # with TOKEN_ROLES_TOO_BROAD, so collection cannot run either way. $forbNames = ($script:collectorForbiddenGrants | ForEach-Object { $_.Role }) -join ', ' Write-Host "" Write-Fail "FORBIDDEN GRANT PRESENT — the collector will refuse this application's token." Write-Host "" Write-Warn2 "This application holds: $forbNames" Write-Warn2 "Departure Watch runs on a DEDICATED app whose token must carry envelope-only" Write-Warn2 "Mail.ReadBasic.All and NOTHING broader. The collector verifies the token's own" Write-Warn2 "roles claim and refuses TOKEN_ROLES_TOO_BROAD — so with this grant in place," Write-Warn2 "every collection tick fails no matter what Exchange scoping says. This is the" Write-Warn2 "opposite of the guidance an older version of this script printed; that guidance" Write-Warn2 "predates the dedicated-app contract (ADO #1859) and was wrong under it." Write-Host "" Write-Host " Remove the broad grant and keep (or add) Mail.ReadBasic.All only:" -ForegroundColor Cyan foreach ($fg in $script:collectorForbiddenGrants) { Write-Host " Remove-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $spObjectId -AppRoleAssignmentId $($fg.AssignmentId) # $($fg.Role)" } Write-Host "" Add-Check $checkNameGraphGrant "FAIL" "Collector-forbidden grant(s) present: $forbNames — token will be refused (TOKEN_ROLES_TOO_BROAD)" } elseif ($readGrants.Count -gt 0) { $readNames = ($readGrants | ForEach-Object { $_.Role }) -join ', ' Write-OK "Graph mail-read grant present: $readNames" Write-Host " Mail.ReadBasic.All is the ONE grant this feature runs on — envelope fields" -ForegroundColor DarkGray Write-Host " only, verified from the token's own roles claim at every collection tick." -ForegroundColor DarkGray Write-Host " Which mailboxes it can reach is bounded by the ApplicationAccessPolicy" -ForegroundColor DarkGray Write-Host " fencing this dedicated app — MEASURED to bind (2026-08-17), where an" -ForegroundColor DarkGray Write-Host " Exchange management scope was MEASURED not to (2026-08-08)." -ForegroundColor DarkGray Add-Check $checkNameGraphGrant "PASS" "Present on Microsoft Graph: $readNames" } else { Write-Host "" Write-Fail "NO Mail.ReadBasic.All GRANT — this application cannot read ANY mailbox." Write-Host "" Write-Warn2 "This is not a scoping problem and waiting will not fix it. Without the Graph" Write-Warn2 "application permission the client-credentials token carries no mail scope, so" Write-Warn2 "Graph rejects every read with 403 — and the collector additionally verifies the" Write-Warn2 "token's roles claim, requiring envelope-only Mail.ReadBasic.All specifically." Write-Warn2 "Departure Watch will never collect anything in this state." Write-Host "" Write-Host " Grant Mail.ReadBasic.All (APPLICATION, not delegated) on Microsoft Graph and admin-consent it:" -ForegroundColor Cyan Write-Host " az ad app permission add --id $AppId ``" Write-Host " --api $graphResourceAppId ``" Write-Host " --api-permissions 693c5e45-0940-467d-9b8a-1022fb9d42ef=Role # Mail.ReadBasic.All" Write-Host " az ad app permission admin-consent --id $AppId" Write-Host "" Write-Warn2 "Grant Mail.ReadBasic.All ONLY. Do NOT grant Mail.Read, Mail.ReadWrite or" Write-Warn2 "Mail.Send — the collector refuses a token carrying any of those (ADO #1859)," Write-Warn2 "and the Entra portal's picker offers a confusable bare 'Mail.ReadBasic' next" Write-Warn2 "to 'Mail.ReadBasic.All'; the .All application role is the one this needs." Write-Host "" Add-Check $checkNameGraphGrant "FAIL" "No Mail.ReadBasic.All grant — nothing can be read; feature is inert" } # ---- 10b. Least privilege ------------------------------------------ # Scoping does not excuse over-granting. A WRITE grant on a read-only # feature is a real finding however well the mailbox set is fenced. Write-Host "" if ($auditBlockers.Count -gt 0) { Write-Fail "LEAST-PRIVILEGE CHECK INCOMPLETE — its empty result is NOT a clean result." Write-Warn2 "Re-run as Global Reader or higher, then check by hand:" Write-Host " az rest --method GET --uri `"https://graph.microsoft.com/v1.0/servicePrincipals/$spObjectId/appRoleAssignments`"" Add-Check $checkNameLeastPriv "NOT RUN" ("Enumeration incomplete: " + ($auditBlockers -join '; ')) } elseif ($excessGrants.Count -gt 0) { Write-Fail "GRANTS BEYOND WHAT THIS READ-ONLY FEATURE NEEDS." Write-Host "" foreach ($g in $excessGrants) { Write-Host " Resource : $($g.Resource)" -ForegroundColor Red Write-Host " Role : $($g.Role)" -ForegroundColor Red Write-Host " Why it matters: $($g.Why)" -ForegroundColor Red Write-Host " Assignment id : $($g.AssignmentId)" -ForegroundColor Red Write-Host " Granted : $($g.CreatedOn)" -ForegroundColor Red Write-Host "" } Write-Warn2 "Departure Watch reads message METADATA. It never writes, sends, or opens a" Write-Warn2 "mailbox over EWS/IMAP/POP. Each role above therefore exceeds what the feature" Write-Warn2 "needs, and the Exchange management scope does not make an over-broad permission" Write-Warn2 "appropriate — it only limits which mailboxes it applies to." Write-Warn2 "full_access_as_app in particular is full access to every mailbox in the tenant;" Write-Warn2 "whether RAOP bounds it here is exactly what the empirical read below settles." Write-Host "" Write-Host " This script does NOT remove anything, deliberately." -ForegroundColor Cyan Write-Host " This is a SHARED application registration — the same app used for Graph" Write-Host " compliance scanning and, on many tenants, email intake. Removing a grant breaks" Write-Host " those silently. VERIFY nothing else relies on it before removing:" Write-Host "" foreach ($g in $excessGrants) { Write-Host " Remove-MgServicePrincipalAppRoleAssignment -ServicePrincipalId $spObjectId -AppRoleAssignmentId $($g.AssignmentId)" } Write-Host "" Write-Host " (Requires a Graph session with AppRoleAssignment.ReadWrite.All — this script" Write-Host " holds read-only scopes and cannot perform the removal even if asked.)" Write-Host "" Write-Warn2 "(#2199) The one grant the feature requires is Mail.ReadBasic.All. A plain" Write-Warn2 "Mail.Read grant on the DEDICATED Departure Watch app is itself fatal — the" Write-Warn2 "collector refuses tokens carrying it. On a SHARED registration, verify what" Write-Warn2 "else uses a grant before removing it; on the dedicated app, remove Mail.Read." Write-Host "" $excessNames = ($excessGrants | ForEach-Object { "$($_.Resource)/$($_.Role)" }) -join ', ' Add-Check $checkNameLeastPriv "FAIL" "Grant(s) beyond a read-only feature's needs: $excessNames" } else { $auditedNamesList = ($auditResources | ForEach-Object { $_.Name }) -join ' + ' Write-OK "No grant beyond read on $auditedNamesList for this application." Write-Host " $($auditedAssignments.Count) grant(s) on those resources examined; all resolved to known app roles." -ForegroundColor DarkGray $leastPrivDetail = "$($auditedAssignments.Count) grant(s) on $auditedNamesList examined and all mapped; no write, send, or Exchange-resource mailbox grant" if ($advisoryGrants.Count -gt 0) { Write-Warn2 "Read-only grant(s) this feature does not need (review, not a failure):" foreach ($ad in $advisoryGrants) { Write-Host " $ad" -ForegroundColor Yellow } $leastPrivDetail += "; not needed but read-only (review): " + ($advisoryGrants -join ', ') } Add-Check $checkNameLeastPriv "PASS" $leastPrivDetail } } catch { Write-Fail "AUDIT NOT PERFORMED — $($_.Exception.Message)" Write-Warn2 "An unrun audit is not a clean audit. Most likely the signed-in account lacks" Write-Warn2 "Application.Read.All / Directory.Read.All. Re-run as Global Reader or higher." Add-Check $checkNameGraphGrant "NOT RUN" "Audit errored: $($_.Exception.Message)" Add-Check $checkNameLeastPriv "NOT RUN" "Audit errored: $($_.Exception.Message)" } } # ────────────────────────────────────────────────────────────────────────── # Step 10c — Boundedness: the empirical negative read # ────────────────────────────────────────────────────────────────────────── # This is the check that actually distinguishes "scoped by RAOP" from "genuinely # tenant-wide", and it is the one this script structurally cannot perform: it # holds an administrator's DELEGATED sign-in, and the question is what the # APPLICATION's own app-only token can reach. Answering it needs that app's # client credentials, which this script does not have and will not handle. # # So this records NOT RUN, always, with the commands to run. It never records # PASS. A script that could clear its own hardest check by assertion would be # certifying the one thing it has no evidence for — and that is the exact shape # of the bug this rewrite exists to remove. Write-Host "" Write-Host "────────────────────────────────────────────────────────────────────────────" -ForegroundColor Cyan Write-Host " Boundedness — the empirical negative read (this script cannot perform it)" -ForegroundColor Cyan Write-Host "────────────────────────────────────────────────────────────────────────────" -ForegroundColor Cyan Write-Host "" Write-Warn2 "Everything above describes CONFIGURATION. Test-ServicePrincipalAuthorization reports" Write-Warn2 "the stored RBAC configuration, not enforced state. Only an app-only read against a" Write-Warn2 "mailbox OUTSIDE the watch group demonstrates what this application can actually reach:" Write-Host "" Write-Host " 403 whose body contains [RAOP] -> BOUNDED. This is the result you want." -ForegroundColor DarkGray Write-Host " 200 -> UNBOUNDED. Active exposure — stop and fix." -ForegroundColor DarkGray Write-Host " 403 without [RAOP] -> no Graph mail grant, or consent missing." -ForegroundColor DarkGray Write-Host "" Write-Host " Run this where the application's client secret is available — NOT here:" -ForegroundColor Cyan Write-Host "" # Single-quoted throughout: these lines contain $body/$tok/$_/$select, and a # double-quoted string would interpolate them into the printed command. Write-Host ' $body = @{ client_id = ""; client_secret = ""' Write-Host ' scope = "https://graph.microsoft.com/.default"; grant_type = "client_credentials" }' Write-Host ' $tok = (Invoke-RestMethod -Method POST -Body $body `' Write-Host ' -Uri "https://login.microsoftonline.com//oauth2/v2.0/token").access_token' Write-Host '' Write-Host ' # NEGATIVE READ — a mailbox that is NOT in the watch group.' Write-Host ' try {' Write-Host ' $r = Invoke-WebRequest -Method GET -Headers @{ Authorization = "Bearer $tok" } `' Write-Host ' -Uri "https://graph.microsoft.com/v1.0/users//mailFolders/inbox"' Write-Host ' "UNBOUNDED — HTTP $($r.StatusCode). This app can read outside the watch group."' Write-Host ' } catch {' Write-Host ' $code = $_.Exception.Response.StatusCode.value__' Write-Host ' "HTTP $code : $($_.ErrorDetails.Message)" # expect 403 containing [RAOP]' Write-Host ' }' Write-Host "" Write-Host " Substitute for this run:" -ForegroundColor DarkGray Write-Host " = $AppId" -ForegroundColor DarkGray Write-Host " = $tenantId" -ForegroundColor DarkGray if ($NegativeTestMailbox) { Write-Host " = $NegativeTestMailbox" -ForegroundColor DarkGray } else { Write-Host " = any licensed mailbox that is not in '$GroupName'" -ForegroundColor DarkGray } Write-Host "" Write-Warn2 "Repeat the same read against a mailbox that IS in the group and expect 200. A 403 on" Write-Warn2 "both proves nothing — that is what a missing grant looks like, not what a fence does." Write-Host "" Add-Check $checkNameBounded "NOT RUN" "Requires an app-only token this script does not hold; run the printed negative read (403 [RAOP] = bounded, 200 = unbounded)" # ────────────────────────────────────────────────────────────────────────── # Step 11 — Summary # ────────────────────────────────────────────────────────────────────────── # Completeness guard. Every check below MUST appear in the results table, and a # check that is absent is worse than one that failed: NOT RUN is visible and # counted, absence is neither — the table just renders one row fewer and the # counts silently omit it. Anything that never got recorded is materialised here # as NOT RUN so it cannot vanish, whatever path the run took. $expectedChecks = @( "Enterprise application resolved", "Exchange organization customization", "Watch group present", $checkNameGroupGate, "Mailbox added to watch group", "Management scope bound to the group", "Exchange service principal registered", "Role '$roleName' scoped to '$ScopeName'", $checkNameOtherRoles, $checkNamePositive, $checkNameNegative, $checkNameGraphGrant, $checkNameLeastPriv, $checkNameBounded ) foreach ($expected in $expectedChecks) { if (@($script:checks | Where-Object { $_.Name -eq $expected }).Count -eq 0) { Add-Check $expected "NOT RUN" "Never recorded during this run — treat the check as unperformed and report this as a script defect" } } $passCount = @($script:checks | Where-Object { $_.Status -eq 'PASS' }).Count $failCount = @($script:checks | Where-Object { $_.Status -eq 'FAIL' }).Count $skipCount = @($script:checks | Where-Object { $_.Status -eq 'NOT RUN' }).Count Write-Host "" Write-Host "════════════════════════════════════════════════════════════════════════════" -ForegroundColor Magenta Write-Host " TATER Departure Watch — results" -ForegroundColor Magenta Write-Host "════════════════════════════════════════════════════════════════════════════" -ForegroundColor Magenta Write-Host "" foreach ($c in $script:checks) { $label = $c.Name if ($label.Length -lt 46) { $label = $label.PadRight(46) } $colour = "Yellow" if ($c.Status -eq 'PASS') { $colour = "Green" } if ($c.Status -eq 'FAIL') { $colour = "Red" } Write-Host (" {0} {1}" -f $label, $c.Status) -ForegroundColor $colour if ($c.Detail) { Write-Host " $($c.Detail)" -ForegroundColor DarkGray } } Write-Host "" Write-Host " PASS: $passCount FAIL: $failCount NOT RUN: $skipCount" Write-Host "" Write-Host " Tenant : $tenantId" Write-Host " Application (client) : $AppId" Write-Host " Application name : $appDisplayLabel" Write-Host " Enterprise app object : $spObjectId" -ForegroundColor DarkGray Write-Host " Watch group : $GroupName" Write-Host " Management scope : $ScopeName" Write-Host " Role assigned : $roleName (read-only)" Write-Host "" if ($failCount -gt 0) { Write-Fail "$failCount check(s) FAILED. Do not treat the Exchange side as provisioned." } if ($skipCount -gt 0) { Write-Warn2 "$skipCount check(s) did NOT RUN. A check that did not run is not a check that passed —" Write-Warn2 "each one above says what is still unproven and how to prove it." } # ────────────────────────────────────────────────────────────────────────── # Step 12 — What this does and does not mean # ────────────────────────────────────────────────────────────────────────── Write-Host "" Write-Host "════════════════════════════════════════════════════════════════════════════" -ForegroundColor Magenta Write-Host " EXCHANGE SIDE ONLY — nothing is being monitored" -ForegroundColor Magenta Write-Host "════════════════════════════════════════════════════════════════════════════" -ForegroundColor Magenta Write-Host "" Write-Warn2 "This script provisioned and verified the EXCHANGE half of Departure Watch." Write-Warn2 "It did NOT start monitoring anything. A provisioned mailbox is not a monitored one." Write-Host "" Write-Host " Before a single message is read, ALL of the following must also happen:" -ForegroundColor Yellow Write-Host " 1. TATER Security sets DEPARTURE_WATCH_ENABLED=1 on the API. This is a" Write-Host " platform kill switch, not an org setting - nothing you do in the" Write-Host " product clears it, and today it is OFF." Write-Host " 2. Your organization records a policy attestation: the policy document" Write-Host " carrying the monitoring notice, how staff were notified, the" Write-Host " acknowledgement campaign, the jurisdictions, and a named counsel" Write-Host " reviewer. It expires annually, and expiry STOPS collection." Write-Host " 3. A named human authorizes the specific case, with a written business" Write-Host " justification. An API key or agent cannot do this." Write-Host " 4. The case is activated, which re-verifies this Exchange boundary and" Write-Host " refuses if it is absent." Write-Host "" Write-Host " What DOES exist in the application today: the REST API, collection every" -ForegroundColor Gray Write-Host " 15 minutes, a daily briefing, retention enforcement, and delivery to the" Write-Host " case's recorded recipients. There is no user interface and there are no" Write-Host " MCP tools - the authorization paths are deliberately excluded from MCP." Write-Host "" Write-Host " Running this script alone reads no mailbox, produces no briefing, and" -ForegroundColor Yellow Write-Host " raises no security signal." -ForegroundColor Yellow Write-Host "" # Boundedness is a LADDER, not a single verdict, and the top rung is one this # script cannot climb. Configuration evidence (the Exchange negative test plus a # least-privilege grant set) is what a run here can produce; the empirical # app-only read is what actually settles it. Saying "bounded" on configuration # alone would overstate exactly as far as the old audit understated. $negStatus = Get-CheckStatus $checkNameNegative $grantStatus = Get-CheckStatus $checkNameGraphGrant $leastPrivStatus = Get-CheckStatus $checkNameLeastPriv $boundedStatus = Get-CheckStatus $checkNameBounded $otherRolesStatus = Get-CheckStatus $checkNameOtherRoles $configuredBounded = ($negStatus -eq 'PASS') -and ($leastPrivStatus -eq 'PASS') -and ($otherRolesStatus -eq 'PASS') ` -and (-not $script:appCanWidenFence) Write-Host " What this run is good for:" -ForegroundColor Cyan if ($configuredBounded) { Write-Host " 1. The Exchange permission boundary exists and is CONFIGURED AS BOUNDED:" Write-Host " the NEGATIVE test puts a mailbox outside the group out of scope, no Exchange" Write-Host " role assignment reaches past the scope, and no Entra grant exceeds read." Write-Host " That is configuration evidence. It is not yet an observation." -ForegroundColor Yellow } else { Write-Host " 1. The Exchange permission boundary exists. IT IS NOT SHOWN BOUNDED." -ForegroundColor Yellow Write-Host " NEGATIVE test (Exchange scope) : $negStatus" -ForegroundColor Yellow Write-Host " Other Exchange role assignments : $otherRolesStatus" -ForegroundColor Yellow Write-Host " Entra grants least privilege : $leastPrivStatus" -ForegroundColor Yellow if ($script:appCanWidenFence) { Write-Host " App can widen its own fence : YES (Exchange.ManageAsApp)" -ForegroundColor Yellow Write-Host " This one is not fixable by re-running. Either the feature moves to a" Write-Host " dedicated registration without Exchange.ManageAsApp, or the boundary is" Write-Host " accepted as one this application could change." } Write-Host " These are configuration evidence and all must be satisfied before the" Write-Host " empirical read below is even worth running." } Write-Host " Required Graph mail-read grant : $grantStatus" Write-Host " Empirical negative read : $boundedStatus (this script cannot perform it)" Write-Host " 2. When the TATER side ships, it VERIFIES this provisioning and deliberately" Write-Host " refuses to create it. TATER cannot add a mailbox to its own scoping group:" Write-Host " an application that can widen its own scope is not fenced." Write-Host "" Write-Host "Next steps:" -ForegroundColor Cyan Write-Host " 1. Resolve any FAIL above before anything else. Note the two kinds: a missing" Write-Host " Graph mail-read grant means the feature CANNOT READ AT ALL, while an excess" Write-Host " grant or a failed negative test means it may read TOO MUCH." Write-Host " 2. If the NEGATIVE test did not PASS, run it — it is the Exchange-side evidence:" Write-Host " Test-ServicePrincipalAuthorization -Identity $AppId -Resource " Write-Host " 3. Perform the EMPIRICAL NEGATIVE READ printed above. It is the only step that" Write-Host " distinguishes an application bounded by RAOP from one that is tenant-wide, and" Write-Host " no amount of re-running this script can substitute for it." Write-Host " 4. Re-run this script with -VerifyOnly in ~2 hours to confirm the cached membership" Write-Host " evaluation has caught up:" Write-Host " .\Setup-TATERDepartureWatch.ps1 -AppId $AppId -NegativeTestMailbox -VerifyOnly" Write-Host " 5. Do not tell anyone monitoring is active. It is not." Write-Host "" Write-Host " Help doc: $helpDocUrl" Write-Host "" # Sessions are deliberately LEFT OPEN. Setup-TATEREmailIntake.ps1 disconnects # Exchange Online before printing its manual remediation commands, so anyone who # copy-pastes them gets "The term 'New-ServicePrincipal' is not recognized." # Every command printed above is runnable in this session, right now. Write-Host " The Exchange Online and Microsoft Graph sessions are still open so the commands" -ForegroundColor DarkGray Write-Host " above can be pasted directly. Run these when you are done:" -ForegroundColor DarkGray Write-Host " Disconnect-ExchangeOnline -Confirm:`$false" -ForegroundColor DarkGray if ($graphConnected) { Write-Host " Disconnect-MgGraph" -ForegroundColor DarkGray } Write-Host "" # ────────────────────────────────────────────────────────────────────────── # Terminating status — see .NOTES for the contract # ────────────────────────────────────────────────────────────────────────── # Without this the script exits 0 past the Exchange connect step no matter what # it found, so a wrapper, CI step, MSP runner or && chain reads "NEGATIVE TEST # FAILED" and "TENANT-WIDE ENTRA MAILBOX GRANT FOUND" as success. A verification # script that cannot signal a failed verification is not a verification. # 2 = something FAILED (an active exposure, or a missing required grant) # 1 = nothing failed but something is UNPROVEN (NOT RUN) # 0 = every check PASSED — NOT reachable from a script-only run, because the # boundedness check needs an app-only read this script cannot perform. # See .NOTES. Automate on `-le 1`, not on `-eq 0`. if ($failCount -gt 0) { Write-Host " Exit code 2 — at least one check FAILED." -ForegroundColor Red Write-Host "" exit 2 } if ($skipCount -gt 0) { if ($skipCount -eq 1 -and $boundedStatus -eq 'NOT RUN') { Write-Host " Exit code 1 — every check this script CAN perform passed. The one NOT RUN is the" -ForegroundColor Yellow Write-Host " empirical negative read, which requires the application's own credentials." -ForegroundColor Yellow Write-Host " This is the best result a script-only run can produce; 0 is not reachable here." -ForegroundColor Yellow } else { Write-Host " Exit code 1 — every check that ran passed, but $skipCount did NOT RUN and are unproven." -ForegroundColor Yellow } Write-Host "" exit 1 } Write-Host " Exit code 0 — all $passCount check(s) PASSED." -ForegroundColor Green Write-Host "" exit 0