AI Compliance Analyst

A conversational AI co-worker built into TATER that reads your live scan data and can autonomously create risk acceptances, assign controls, document evidence, and trigger remediations — all from a chat window in the sidebar.

What Is the AI Compliance Analyst?

The AI Compliance Analyst is a full agentic AI assistant embedded directly in the TATER application, accessible from AI Assistant in the sidebar navigation. Unlike a general-purpose chatbot, it has direct access to your organization's compliance data through a set of purpose-built tools it can call autonomously.

You can ask it to do real compliance work, not just answer questions:

• "What's our overall compliance score right now, and which framework is performing worst?"
• "Find all Critical and High failing controls in Entra ID and assign them to John with a due date of next Friday."
• "We're failing the legacy authentication control but need it on for one service account. Create a risk acceptance."
• "Document that we verified DMARC enforcement manually in Exchange Admin Center yesterday."
• "What does the DMARC control require and can you trigger the automated fix?"

The analyst understands TATER's frameworks (CIS Microsoft 365 Foundations Benchmark, CISA SCuBA, DISA STIGs), product codes (ENT, EXO, DEF, SPO, SPT, PUR, PBI, PPL), and control status values (Pass, Fail, Manual Review, Skip, Override). It knows that Teams SPT_* controls often return Skip because those checks require delegated authentication unavailable in automated contexts, so it will not flag those as problems unless you ask.

Setting Up the API Key

The AI Analyst requires an Anthropic API key. The key is stored encrypted at rest (AES-256-GCM) in your organization's settings.

The 9 Compliance Tools

The AI Analyst can call nine tools autonomously during a conversation. You never need to invoke these manually — the AI decides when and how to use them based on your request. Understanding what each tool does helps you frame requests effectively.

Tool	What It Does	When the AI Uses It
get_recent_scans	Retrieves recent compliance scans (up to 10), including scan date, type, pass/fail counts, and compliance percentage.	Whenever you ask about current status, scores, or want context before taking action.
get_scan_results	Gets control-by-control results from a specific scan. Supports filtering by status (Fail, Manual, Pass) and severity (Critical, High, Medium, Low). Returns up to 80 controls per call.	When you ask which controls are failing, want to review a specific area, or before creating assignments or risk acceptances.
get_overrides	Retrieves existing risk acceptances (overrides) for the organization. Used to check whether a risk acceptance already exists before creating a duplicate.	Before creating any risk acceptance, to avoid duplicates and show you existing justifications.
create_risk_acceptance	Creates a formal risk acceptance (override) with a professional 50–150 word justification, expiry date, and optional compensating controls. Records in the Overrides container and the audit log.	When you explicitly request a risk acceptance, or when the AI has confirmed the action via ask_user.
add_evidence_comment	Adds an auditable evidence comment to a specific control. The comment is tagged with source: 'ai-analyst' in the audit trail. Appears in the control's Evidence tab.	When you describe manual verification steps you have completed, or ask the AI to document something you tell it.
get_people	Lists team members in the People directory for the organization, including name, title, and email. Used to match names to People records for assignment.	Before creating assignments, to look up valid People records.
assign_control	Assigns a control to a person with a due date and optional notes. Creates an Assignment record visible on the Assignments page and in the assignee's task list.	When you ask the AI to delegate control remediation to a team member.
ask_user	Pauses the agentic loop and presents a question to you in the chat interface. The AI waits for your typed response before continuing. Used to confirm high-impact actions, gather context the AI does not have, or verify intent before bulk operations.	Before creating more than five risk acceptances at once, before remediating a Critical or High severity finding, or anytime the AI needs clarification.
trigger_remediation	Triggers an automated remediation script for a control that has a remediation available in the catalog. Requires the organization to have remediation enabled and a webhook URL configured. The AI always uses ask_user to confirm before triggering.	When you explicitly ask the AI to fix a control, and only after confirmation.

The Agentic Loop

The AI Analyst does not respond in a single shot. It runs an agentic loop that can make up to 12 tool calls before returning a final answer. This allows it to chain multiple operations together seamlessly.

For example, when you ask it to assign all failing Entra ID controls to a team member, it will:

1. Call get_recent_scans to find the latest scan
2. Call get_scan_results filtered to status=Fail and product=ENT
3. Call get_people to find the team member's People record
4. Call ask_user to confirm: "I found 7 failing Entra ID controls. I'll assign them all to John Smith with a due date of next Friday. Shall I proceed?"
5. After you confirm: call assign_control seven times, once per control
6. Summarize what was done

While the loop runs, you see each tool call appear in the chat as a wrench icon followed by the tool name. Tool calls update to a checkmark when they complete successfully, or an X if they fail. The loop pauses visibly whenever ask_user is called — a question box appears in the chat and the AI waits for your response before continuing.

The ask_user Pause Mechanism

The ask_user tool is the primary mechanism for keeping you in control. When the AI needs input or wants to confirm an action, it injects a question box directly into the chat thread. The agentic loop suspends completely until you respond.

Common situations that trigger ask_user:

• The AI is about to create more than five risk acceptances at once
• The AI has identified a Critical or High severity finding to remediate or override
• The AI needs context you have not provided (e.g., which team member to assign to, or the business reason for an exception)
• The AI is unsure whether a control's current state is intentional or a misconfiguration
• Bulk operations that affect many records at once

Type your response into the question box and press Enter (or click Submit). The AI incorporates your answer and continues the loop from where it paused.

Example Conversations

Checking Compliance Status

Assigning Failing Controls to a Team Member

Creating a Risk Acceptance

Documenting Manual Evidence

Triggering an Automated Remediation

Built-In Safety Rules

The AI Analyst operates under a set of safety rules baked into its system prompt. These rules protect your environment from unintended bulk changes and ensure high-risk actions always have a human checkpoint:

Rule	Behavior
Bulk risk acceptances	The AI will not create more than 5 risk acceptances without first using ask_user to confirm intent and show you a summary of what it is about to create.
Critical / High severity overrides	Any risk acceptance or remediation for a Critical or High severity finding always triggers ask_user first, regardless of how explicitly you phrased the request.
Teams SPT_* controls	The AI knows Teams controls frequently return Skip status because Cs* cmdlets require delegated authentication. It will not flag these as problems or recommend remediations for them unless you specifically ask.
Audit trail tagging	Every record created by the AI Analyst (overrides, assignments, comments) is tagged with source: 'ai-analyst' in the audit log so you can distinguish AI-created records from human-created ones.
Remediation confirmation	The AI always uses ask_user before calling trigger_remediation, regardless of severity. Automated remediations make changes to your tenant and cannot be rolled back automatically.

Audit Trail

All actions taken by the AI Analyst are recorded in TATER's activity audit log. Navigate to Administration → Activity Log to review them. You can filter by:

• Entity type: Select override, assignment, or comment to see only AI-created records of that type
• Date range: Narrow to the period when a conversation occurred
• Source: The source field on AI-created records is set to ai-analyst, distinguishing them from records created manually or by runbooks

Each audit entry includes the specific values written (e.g., the justification text, the assigned person, the expiry date), the timestamp, and the user account whose session was active when the AI ran the action.

Tips and Limitations

Tips for Effective Use

• Be specific about scope. Saying "fix our Exchange issues" is broad — the AI will ask clarifying questions. "Create risk acceptances for all failing EXO controls with severity Medium or lower" gives it enough to work with directly.
• Specify timeframes for assignments. The AI can accept relative dates like "next Friday," "end of month," or "in two weeks" and converts them to absolute dates.
• Chain related requests. You can make complex compound requests in one message: "Find all Critical failing controls, assign them to the security team with a two-week deadline, and ask me before creating any risk acceptances." The AI handles the sequencing.
• Use it after each scan. The most productive time to run the AI Analyst is immediately after a new scan completes. Ask for a summary, then ask it to triage the new failures against existing overrides and assignments.
• The conversation is session-scoped. Conversation history is stored in the browser tab. Closing the tab or opening a new session starts a fresh conversation. The AI does not have memory of previous sessions.

Limitations

• 80-control result cap per call. When retrieving scan results, the AI fetches up to 80 controls per tool call. For very large scans with hundreds of failures, it may need to make multiple calls or ask you to narrow the scope by product code or severity.
• 12-iteration loop limit. The agentic loop runs at most 12 tool-call iterations per message. Very large bulk operations (e.g., assigning 50 controls individually) may hit this limit. Break large tasks into smaller segments if needed.
• 30-message conversation window. The API sends the last 30 messages as conversation context. Very long conversations may lose early context. Start a fresh session for a new topic.
• No read access to Settings or credentials. The AI cannot read your Anthropic API key, SIEM config, or webhook URLs. It also cannot modify organization settings.
• Remediation requires prior configuration. The trigger_remediation tool only works if your organization has remediation enabled and a webhook URL configured in Settings → Remediation. The AI will tell you if this is not set up.
• The AI does not run scans. It can read existing scan results but cannot initiate a new scan. Use the Scans page or the Azure Automation runbook schedule for that.

Troubleshooting

AI Analyst Not Responding / "Not Configured" Error

If the chat shows a message like "AI Analyst is not configured for this organization":

• Navigate to Settings → Features and check the AI Compliance Analyst card. Enter your Anthropic API key and save.
• If your organization shares a key configured at the tenant level, ask your SuperAdmin to verify the key is saved correctly.
• The key must be a valid Anthropic key starting with sk-ant-. Keys generated from the Anthropic API console are the only supported format.

Tool Calls Failing

If a tool call shows an X (failure) in the chat:

• get_recent_scans / get_scan_results: Ensure you have at least one scan imported. Navigate to the Scans page to verify.
• get_people: Your organization must have People records. Add team members on the People page before asking the AI to make assignments.
• create_risk_acceptance: You must have OrgAdmin or higher role. Auditors and Viewers cannot create overrides even through the AI.
• trigger_remediation: Remediation must be enabled for your organization (Settings → Remediation) and the control must have a matching script in the remediation catalog.
• add_evidence_comment: You need Auditor role or higher to add comments.

Slow or Incomplete Responses

The AI Analyst runs against the Anthropic API in real time. Response time depends on Anthropic API latency and the complexity of the tool chain being executed. If a response appears stalled:

• Wait up to 60 seconds for complex multi-step operations before assuming a failure. The tool call indicators in the chat update as each step completes.
• If the chat shows tool call indicators that never resolve, check your browser's network tab for a stalled request to /api/ai/chat. A timeout or network disruption may have interrupted the call. Refresh the page and retry.
• Large scan result sets slow down initial tool calls. Try narrowing the request: "only show me Critical severity EXO failures" rather than "all failures."

AI Took an Unexpected Action

If the AI created a record you did not intend:

• Navigate to the relevant page (Overrides, Assignments, or Comments) and delete the record manually.
• Check the audit log (Administration → Activity Log) to confirm exactly what was created and when. Filter by source: ai-analyst.
• In future conversations, be more specific about what you want confirmed before the AI acts. Phrases like "check with me before creating anything" or "don't take action yet, just tell me what you would do" put the AI in a planning-only mode.