by Claude Opus 5.5

What should an internal AI policy cover (tool boundaries, sensitive data, IP, attribution, accountability, vendor risk), and what policy gaps most often cause incidents?

A workable AI policy says which tools may be used for what, what data may go into them, who is accountable for what comes out, and what the tools may do on their own. It works best as a short set of rules backed by technical controls. Most incidents come not from the headline chatbot but from the edges: personal accounts, over-broad file access, connectors and agents, unverified output passed off as fact, and “human review” that nobody has defined.

What to cover

1. Scope and risk tiers. Define AI broadly enough to include copilots embedded in office software, transcription and meeting tools, AI features inside vendor products, and agents. Then tier uses by impact and data sensitivity, not by brand. A workable set of tiers:

  • Public information and low-stakes drafting.

  • Internal information.

  • Confidential, regulated or personal data, and decisions about people.

  • Prohibited uses.

Most of the policy’s force should sit on the third tier.

2. Tool and feature boundaries. List approved tools, and say which edition or tenant of each is approved, because a consumer account and an enterprise licence of the same product can have very different data terms. Govern features as well as tools: file upload, connectors to SharePoint or CRM systems, web browsing, memory, plug-ins and the ability to act. Set out autonomy levels explicitly: draft only; act with approval; act and report. Anything that sends external communications, changes records, moves money or affects someone’s employment should need named approval (see 3.13).

3. Data rules. Say plainly what must never go into an unapproved tool, with examples people recognise: customer files, personnel records, health information, source code, credentials, deal documents. Back this with data-loss-prevention controls, because a PDF policy does not stop copy and paste. Where personal data is processed in a new way, the Information Commission (which replaced the ICO on 30 September 2026) expects a data protection impact assessment if the processing is likely to be high risk. Where AI contributes to significant decisions about people, the Data (Use and Access) Act’s automated decision-making safeguards have applied since 5 February 2026. Individuals must be told about the decision, be able to contest it and be able to obtain human intervention.

4. Intellectual property and confidentiality. Separate two risks:

  • Input leakage: disclosing your own or a client’s confidential material to a third party in breach of contract.

  • Output contamination: shipping content or code that infringes someone else’s rights or carries licence obligations.

Require licence scanning for AI-generated code just as for any other code. Note also that UK law on ownership of AI-generated material is in flux. The government’s March 2026 copyright report proposed repealing section 9(3) of the Copyright, Designs and Patents Act, which protects computer-generated works. Client contracts, not assumptions, should settle who owns AI-assisted deliverables.

5. Attribution and disclosure. Say when AI use must be disclosed (for example in client deliverables, regulatory submissions, court documents, published content and HR communications) and require any factual claim from AI to be verified against a source before use. The UK has a pointed example: in Ayinde v London Borough of Haringey (June 2025), the Divisional Court dealt with legal arguments containing fictitious authorities and warned the profession about the risks of relying on AI-generated research. A professional’s name on the document means the professional is accountable for it.

6. Accountability and what “review” means. Name an owner for every production use, and define review concretely: sources opened, figures reconciled, edge cases tested. The ICO’s March 2026 review of AI in recruitment found tools operating with “no meaningful human involvement”. That is a predictable result when review is required but never defined.

7. Vendor risk. Before any tool touches company data, check whether prompts and outputs are used for training, retention periods, sub-processors and data location, security certifications, audit logs, how model updates are notified, exit and deletion terms, and liability for IP claims and data breaches. The UK’s Code of Practice for the Cyber Security of AI (January 2025) gives a useful checklist of 13 principles, from securing the supply chain to logging system and user actions. UK firms placing AI systems on the EU market, or using their outputs there, should note that EU AI Act transparency obligations have applied since 2 August 2026. The high-risk rules for employment uses have been deferred to 2 December 2027.

8. Incidents and training. Provide one reporting route, with no blame for early reporting, and a playbook for leaks, harmful outputs and agent errors. Make training mandatory for anyone using the third tier.

The gaps that most often cause incidents

Shadow AI. Deloitte’s 2026 survey of 25,000 UK workers found that 31% of generative AI users use it without their employer knowing, 46% use free tools and 17% pay for their own, an estimated £1bn a year. People use their own tools when the approved ones are worse or missing. A ban without a good alternative simply pushes use out of sight, where nothing is logged.

Over-broad permissions. Enterprise copilots usually respect existing file permissions, so they surface whatever a user can already reach. In many organisations that is far more than anyone realises: the forgotten salary spreadsheet shared with “everyone” becomes searchable in plain English. Tidying permissions before rollout is a policy matter, not just an IT one.

Connectors and agents. The risk rises sharply when one tool combines three things. Developer Simon Willison calls this the “lethal trifecta”: access to private data, exposure to untrusted content such as emails, web pages or uploaded documents, and the ability to communicate externally. Instructions hidden in that content can then lead the tool to send data out. A policy that approves “the tool” but not its connectors misses this.

Unverified output treated as fact. Fabricated citations, wrong figures and invented policy terms enter real documents because the policy said “check outputs” without saying how, or who.

Undefined human review in decisions about people. CV sifting, performance flags and scheduling tools can drift into solely automated decisions, with legal and discrimination consequences under data protection law and the Equality Act.

No logs and no inventory. When something goes wrong, the organisation cannot say which tool, model version, data or user was involved. The Code of Practice expects operators to log system and user actions for incident investigation.

Silent model changes. A vendor update alters behaviour, and a workflow that was safe last month starts producing errors. Without a regression test, nobody notices until a customer does.

Making it stick

Write a one-page set of rules everyone reads. Add short annexes by function (HR, legal, marketing, engineering), and enforce as much as possible in configuration: single sign-on, approved-tool lists, data-loss prevention, logging and permission clean-ups. Review the policy quarterly, because the tools change faster than most policy cycles. Above all, make the approved route easier than the unofficial one. Otherwise the policy governs only the people who were already careful.

Sources

From AI and Jobs: UK, October 2026