Evidence Agent

Automatically collect compliance evidence by navigating Microsoft admin portals with an AI-assisted browser and by running PowerShell control scripts — all from a job queue managed inside TATER.

What the Evidence Agent Does

Gathering compliance evidence is one of the most time-consuming parts of a security audit. The TATER Evidence Agent automates that work. It runs inside the TATER Agent (the Windows Go binary, v2.1+) on any machine that can reach Microsoft admin portals. When a job is created, the agent:

• Opens Microsoft Edge in a focused app window and navigates to the relevant portal (Entra ID, Exchange Admin Center, Microsoft 365 Defender)
• Waits for you to sign in (up to five minutes), then takes over navigation autonomously
• Uses Claude AI to interpret each screenshot and decide what to click, read, or scroll next
• Records each finding as a comment on the control's evidence tab, tagged with Source: Evidence Agent

For endpoint and PowerShell-based controls, the agent skips the browser entirely and runs the corresponding Controls/*.ps1 script directly, then formats the result as a structured finding.

Prerequisites

Requirement	Details
TATER Agent v2.1+	The Go-based Windows agent must be installed and running (either as a Windows service or in interactive/tray mode). Download the latest MSI from Settings → Endpoint Agent.
Anthropic API Key	Required for the AI vision step. Configure in Settings → Features → AI Compliance Analyst. The org key takes precedence over the environment variable fallback.
Microsoft Edge	Required for browser-based collection jobs. Edge must be installed on the agent machine. The agent launches it in app mode — no manual browser setup is needed.
TATER API Key on agent machine	The agent authenticates to the TATER API using its API key (set during MSI installation or in %ProgramData%\TATER\config.json). It polls for pending evidence jobs using this key.
Auditor role or higher	Required to create evidence jobs from the TATER web application.
Network access to Microsoft portals	The agent machine must be able to reach entra.microsoft.com, admin.exchange.microsoft.com, and security.microsoft.com.

How the Collection Loop Works

The Evidence Agent runs as a background goroutine inside the TATER Agent process. Here is the complete flow from job creation to recorded evidence:

1. Job created: An Auditor creates a job from the TATER web app, specifying one or more control IDs and a job type (Browser or PowerShell).
2. Agent polls: The agent checks GET /api/evidence/jobs?pending=1 every 15 seconds using its API key.
3. Job claimed: When a pending job is found, the agent sends a PATCH request to claim it, setting its status to running and recording the agent's hostname.
4. Dispatch: Based on job type, the agent takes one of two paths:
   • Browser path: Edge opens to the portal, the agent waits for sign-in, then enters the control loop
   • PowerShell path: The agent locates the matching Controls/*.ps1 script and runs it through the embedded PowerShell runner
5. Control loop (browser): For each control in the job, the agent takes a screenshot and calls POST /api/evidence/step. The server sends the screenshot and control context to Claude, which returns an action to take (click, scroll, navigate, or read). The agent executes the action and repeats — up to 15 steps per control.
6. Evidence recorded: When Claude determines the screen contains relevant evidence, or when it issues a done action, the server posts a comment to the control via the Comments API. The comment body contains Claude's structured finding, including what was observed and how it maps to the compliance requirement.
7. Job complete: After all controls are processed, the agent sends a final PATCH to mark the job completed, including total AI token usage.

Creating an Evidence Job

From the Evidence Agent Page

From a Control Detail Panel

You can also create an evidence job for a single control directly from that control's detail panel:

1. Open any control on the Controls page by clicking its row
2. In the detail panel, click the Collect Evidence button (shown when the control is in Manual Review status or has no recent evidence)
3. Confirm the job type in the prompt — TATER pre-selects the appropriate type based on the control's product code (ENT/EXO/DEF controls default to Browser; endpoint controls default to PowerShell)
4. Click Create Job. You are redirected to the Evidence Agent page to monitor progress

The Browser Session

Signing In

When the agent starts a browser job, it opens Microsoft Edge in app mode (a focused, chromeless window) and navigates to https://entra.microsoft.com. At this point, the agent waits for you to complete sign-in.

You have up to five minutes to authenticate. The agent detects a completed sign-in by polling the browser's URL every two seconds — authentication is considered complete when the URL is no longer on login.microsoftonline.com and is on a *.microsoft.com or *.azure.com domain.

Portal Navigation

Once signed in, the agent takes over. It navigates to the relevant section of the admin portal for each control. The portals the agent can work with include:

Product Code	Portal	URL
ENT	Microsoft Entra admin center	entra.microsoft.com
EXO	Exchange admin center	admin.exchange.microsoft.com
DEF	Microsoft 365 Defender portal	security.microsoft.com
SPO	SharePoint admin center	admin.microsoft.com/sharepoint
PUR	Microsoft Purview compliance portal	purview.microsoft.com

The AI Step Loop

For each control, the agent executes up to 15 steps. Each step follows this cycle:

1. Screenshot: The agent captures a full-page screenshot of the current browser state
2. API call: The screenshot, the control's title, description, and audit procedure are sent to POST /api/evidence/step
3. Claude analysis: Claude examines the screenshot and determines what to do next, returning a structured action
4. Action execution: The agent executes one of the following actions:
   • navigate — Load a specific URL in the browser
   • click — Click an element identified by its visible text label
   • scroll — Scroll the page to reveal more content
   • read — Record what is visible on screen as a finding (continues the loop)
   • done — Mark the control as complete and move to the next one

When Claude issues a read or done action, the server immediately posts the finding as a comment on the control. The comment includes a description of what was observed, whether it appears to satisfy the control requirement, and any notable configuration values that were visible on screen.

PowerShell Evidence Jobs

For controls that correspond to endpoint or local machine checks, the Evidence Agent skips the browser and runs the PowerShell control script directly.

Script Discovery

The agent resolves the control script by matching the control ID prefix against files in the Controls/ directory on the agent machine. For example, a job for control ENT_024 resolves to Controls/ENT_024_MFACapable.ps1. The agent uses its embedded PowerShell runner to execute the script and captures the Meta and Result JSON objects returned by the script.

Result Format

The result is formatted as a structured evidence comment with the following information:

• Status: Pass, Fail, Manual Review, or Skipped
• Finding Note: Human-readable description from the control script's FindingNote field
• Method: How the check was performed (e.g., "Graph API", "Registry check")
• Collected at: Timestamp of when the script ran

The comment is posted with targetType: 'control' and source: 'evidence-agent' so it appears in the control's evidence tab and is searchable in the audit log.

Reviewing Evidence Results

Evidence Agent Dashboard

Navigate to Security → Evidence Agent to see all jobs for your organization:

• Status badge: Pending (yellow), Running (blue), Completed (green), Failed (red)
• Agent hostname: Which machine picked up the job
• Job type: Browser or PowerShell
• Controls: Number of controls in the job and how many have been processed
• Token usage: Total AI input and output tokens consumed by the job (reflects cost)
• Created / Completed timestamps

Click any job row to open the job detail view, which shows a step-by-step log of every action the agent took, including the screenshot thumbnail and Claude's reasoning at each step.

Evidence in Control Panels

Evidence collected by the agent appears alongside manually entered comments in each control's detail panel. To find agent-collected evidence:

1. Open the control from the Controls page
2. Click the Evidence or Comments tab
3. Comments tagged Source: Evidence Agent were collected automatically
4. Click any comment to see the full finding text, including portal screenshots embedded in the step log

Audit Trail

Every evidence comment is recorded in TATER's activity audit log. The entry shows:

• Action: create
• Entity type: comment
• Source: evidence-agent
• Control ID and job ID
• Agent hostname and timestamp

This makes it straightforward to show auditors exactly when, how, and from which machine each piece of evidence was collected.

Troubleshooting

Agent Not Picking Up Jobs

If a job stays in Pending status for more than 30 seconds:

• Check agent is running: Open the system tray on the agent machine. The TATER shield icon should be present. If not, start the agent service (sc start TATERAgent) or launch it from the Start menu shortcut.
• Check API connectivity: The agent must be able to reach https://api.tatersecurity.com. Open the local dashboard (tray icon → Open Dashboard) and check the configuration status panel for API connection errors.
• Verify API key: The agent's API key must be valid and belong to the same organization as the job. Check %ProgramData%\TATER\config.json for the apiBase and organizationId values. Generate a new key in Settings → API Keys and re-run the installer's repair mode if needed.
• Check agent logs: View logs from the tray menu (View Logs) or look in %ProgramData%\TATER\logs\. Look for lines containing [evidence] or [poll].

Browser Not Opening

If the job transitions to Running but no browser window appears:

• Verify Microsoft Edge is installed: The agent uses the system-installed Edge executable. If Edge is not at its default location (C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe), it will fall back to the default browser. Check agent logs for [evidence] launching edge.
• Session must be interactive: The agent must be running in a session where it can open a visible window. If the agent is running as a Windows service under SYSTEM, the browser window will not appear on your desktop. Run the agent in interactive mode (double-click tater-agent.exe) when you need browser evidence collection, or configure the service to run under your user account.
• Check for crashes: Look in agent logs for [evidence] browser error messages. Chromedp errors often indicate a missing or incompatible Edge version.

Sign-In Timeout

If the job fails with a sign-in timeout error (after the five-minute window expires):

• Watch for the Edge window opening on the agent machine immediately after the job transitions to Running. You have a five-minute window from that moment.
• If you are using Conditional Access policies that add extra verification steps (hardware token, number matching), allow extra time. The five-minute window begins when the browser opens, not when you start signing in.
• If MFA prompts time out before you can respond (e.g., push notification expiry), try switching your sign-in method to TOTP code entry, which gives you more time.
• Re-run the job — a new five-minute window starts fresh for each job attempt.

Anthropic API Key Not Configured

If a browser job fails immediately with an error like "AI Analyst is not configured":

• Navigate to Settings → Features in the TATER web app.
• Expand the AI Compliance Analyst card.
• Enter your Anthropic API key and click Save. The key is encrypted at rest using AES-256-GCM.
• The key is resolved in priority order: org-level key (Settings → Features) → tenant-level key → ANTHROPIC_API_KEY environment variable on the Function App. If none are set, browser evidence jobs cannot proceed.
• PowerShell evidence jobs do not require an API key — they run the control script directly without calling Claude.

Incomplete or Partial Evidence

If evidence is collected for some controls in a job but not others:

• 15-step limit reached: If a control required more than 15 browser interactions, the agent stops and moves on. The last recorded step's finding is posted. Check the step log to see how far the agent got.
• Navigation failure: Some portal pages require specific permissions. If the signed-in account does not have access to a blade, the agent will see a permission error on screen and Claude will log a finding indicating the page was inaccessible.
• Portal layout changes: Microsoft occasionally updates admin portal layouts. If Claude cannot identify relevant UI elements, it will record what it observed and stop. Re-running the job after a portal update may yield different results.