AI Security Scorecard for MSPs v0.3

Lead capture

Business details

NameBusiness emailCompanyWebsiteMSP revenue bandPrimary interest

Biggest AI concern

🔒 Your details are used only to send your scorecard result and related follow-up from Myles Agnew, and are never sold or shared with third parties. See the privacy policy.

Risk domain

Governance and ownership

0/9

Do you have a named owner accountable for AI governance, approval decisions, and control maintenance?

Names an accountable owner for AI risk and delivery. Without ownership, AI adoption becomes informal and hard to govern.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Secure AI Toolkit: use the AI owner role brief and weekly review rhythm.

Do you maintain an approved / restricted / prohibited AI tool register?

Controls tool sprawl and prevents staff using random unmanaged AI services with client or internal data.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Secure AI Toolkit: create the AI Tool Approval Register and review it monthly.

Do staff have a written AI acceptable-use policy that is easy to find and enforce?

Turns vague “be careful with AI” advice into practical rules staff can follow.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Secure AI Toolkit: publish the AI acceptable-use policy template.

Risk domain

Data classification and handling

0/9

Have you defined which data classes may and may not be entered into AI tools?

Prevents sensitive client data, credentials, security findings, and regulated data from being mishandled.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Secure AI Toolkit: adopt the data handling matrix.

Are staff trained not to paste credentials, secrets, private keys, tokens, or sensitive client data into AI tools?

Targets the highest-impact misuse pattern for MSPs and security teams.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Run a staff micro-brief and add the rule to onboarding.

Do you minimise, redact, or synthetic-test data before using AI for internal or client scenarios?

Reduces data exposure while still allowing practical AI workflow testing.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add a redaction/minimisation step to every AI workflow template.

Risk domain

Tool approval and vendor control

0/9

Do you review vendor retention, model-training, privacy, and admin-control settings before approving AI tools?

Ensures AI vendor choices are based on security posture, not convenience or hype.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Use the vendor review checklist before approving any new AI service.

Do approved AI tools use MFA, SSO, role-based access, or central identity controls where available?

Prevents unmanaged accounts becoming a weak point in your AI operating layer.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Move approved tools onto managed identity controls where possible.

Are browser extensions, plugins, copilots, and AI integrations reviewed before staff use them?

Controls hidden data exfiltration paths through extensions and integrations.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add extensions/integrations to the approval register, not just web apps.

Risk domain

Prompt and input safety

0/9

Do staff have prompt safety rules for client data, security issues, confidential information, and regulated content?

Gives staff practical boundaries at the point where mistakes happen: the prompt box.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Use the prompt safety rules and prompt vault starter templates.

Are reusable prompts/templates reviewed before client-facing or security-sensitive use?

Avoids scaling bad advice, hidden data leakage, or unsafe instructions across the team.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Create a reviewed prompt library for approved business workflows.

Do you treat system prompts as guidance, not as secret or sufficient security controls?

Prevents false confidence in prompt-only protection for sensitive workflows.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Use permission boundaries, data controls, and human approval—not prompt secrecy.

Risk domain

Output validation and human review

0/9

Are AI-generated client communications reviewed by a human before sending?

Protects trust, accuracy, tone, and liability in client-facing work.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add a human-review rule for all outbound client content.

Are AI-generated scripts, commands, technical recommendations, and security advice tested or reviewed before use?

Prevents AI mistakes from becoming outages, misconfigurations, or bad security advice.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Require lab/test review before technical output is used in production.

Do staff verify factual claims, sources, assumptions, and limitations in AI-generated output?

Reduces hallucination risk and improves professional-grade delivery quality.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add source/assumption verification to your AI output checklist.

Risk domain

Access control and connected systems

0/9

Do AI tools connected to email, files, ticketing, CRM, billing, identity, or client systems require separate approval?

Connected AI has higher blast radius than chat-only AI and needs stronger controls.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Create a connected-system approval gate.

Do AI automations and assistants use least-privilege access?

Limits damage if an automation behaves incorrectly or is abused.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Review OAuth scopes, API keys, service accounts, and mailbox/file permissions.

Are high-impact AI actions gated by human approval?

Prevents autonomous actions from touching billing, credentials, production systems, or client communications without oversight.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Require human approval for destructive, financial, identity, or client-visible actions.

Risk domain

Knowledge base / RAG data security

0/9

Are documents reviewed and classified before upload to AI knowledge bases?

Avoids turning the knowledge base into an unmanaged dump of sensitive data.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add a document intake checklist for AI knowledge sources.

Is knowledge-base access limited by role, client, team, or need-to-know?

Prevents cross-client or cross-role information leakage through AI retrieval.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Segment knowledge bases by client and permission boundary.

Do you have a process to remove outdated, incorrect, or sensitive documents from AI knowledge bases?

Keeps AI answers accurate and reduces persistence of sensitive data.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Schedule KB review and removal as part of service operations.

Risk domain

Automation and agent permissions

0/9

Do you maintain an inventory of AI automations/agents and what actions they can take?

Creates visibility over the operational AI layer before it becomes unmanageable.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Create an automation register with owner, trigger, systems touched, and rollback.

Can AI automations be disabled, paused, or rolled back quickly?

Gives you an emergency brake when an automation misfires.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Document disable/rollback steps for each automation before production use.

Are AI automations tested in a low-risk environment before production use?

Catches bad prompts, bad triggers, permissions mistakes, and workflow edge cases early.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Use a staging/sandbox runbook for every new automation.

Risk domain

Monitoring, logging, and incident response

0/9

Do you log AI tool usage or AI automation actions where possible?

Provides evidence for troubleshooting, governance, incident response, and client assurance.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Capture activity logs, endpoint payloads, and automation runs where practical.

Do staff know how to report accidental sensitive-data disclosure to an AI tool?

Ensures mistakes are contained quickly instead of hidden.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add AI disclosure handling to the incident response playbook.

Do you review AI incidents, near misses, risky use cases, and control exceptions?

Converts incidents into stronger operating controls.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Add AI risk review to your monthly security operations rhythm.

Risk domain

MSP reseller readiness

0/9

Do you have a packaged AI readiness or AI governance service you can sell to clients?

Turns your internal controls into a productized client-facing offer.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Use the MSP service packaging kit to define scope, pricing, and deliverables.

Do you have client-facing templates: proposal, discovery questions, policy template, and delivery checklist?

Reduces custom work and helps MSPs sell/deliver consistently.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Build a standard client pack before selling bespoke AI projects.

Do you have a repeatable delivery process that avoids custom AI project sprawl?

Protects margins and keeps the AI service line operationally sane.

0 Not in place / unknown 1 Informal or inconsistent 2 Partly documented / partly enforced 3 Owned, enforced, reviewed

Control action: Use fixed workflow menus and route custom work outside the self-serve system.