Copilot on Autopilot: How Quiet Convenience Incubates AI Governance Breaches

Key points

• Microsoft Copilot is easy and convenient, so many companies have activated it, without supervision.

• Although familiar, it is just another hairy generative AI system, that needs governance especially audits, monitoring and explainability

• At your company, an urgent retro-fit of backend controls may be needed to minimise Copilot's AI governance risks

It's easy they said, just a few clicks they said, it saves you tons of time they said.  Like Clippy from the 90s, Microsoft Copilot sits there eager and ready to serve you at the stroke of button. Microsoft products like Word and Excel seem to work pretty well, giving us a (false) sense of security, so Copilot feels familiar, like a part of the family. What could possibly go wrong?

Lots. How many of those hundreds of thousands of companies using Copilot are aware of the gradual, unnoticed, and dangerous progression of data, privacy and policy violations of an unsupervised Copilot implementation?

Launched in 2023, Microsoft Copilot had 218 million active users across Windows, app and website in 2025.

It all started because Microsoft made enabling Copilot pretty darn easy. Next, you just tell it to get to work, doing your work. It's a ridiculously convenient extra pair of hands in Word, Excel and Powerpoint for everyday tasks.

We all know every generative AI tool comes with a disclaimer, the outputs are never guaranteed to be correct. But who cares, if all I'm doing is creating a nice looking powerpoint slide deck or my excel formula involves complex calculations that's beyond my comprehension.  In low risk contexts, you can knock yourself out. But every now then, you'll be pressed for time, and, accidently cross a governance line releasing sensitive information or bypassing an internal policy.  A line, if crossed, that somebody in your company should care about.

This is a salient reminder that enabling Copilot hands over delegating decision support—and sometimes decision-making—to a machine that is probabilistic, opaque, and difficult to constrain if not setup correctly.

As with most things in technology, it’s not just about the technology, this time it’s the absence of key administrative and management controls of unsupervised AI:

• Human-in-the-loop review or approval

• Approved use cases

• Monitoring and logging of activity for traceability and auditing

• Policy enforcement and breach notifications

How do the lack of these controls show up in the real world?  For starters, it can result in misuse, errors or compliance failures. Humans sometimes make mistakes such as not adding up a column of numbers correctly in an excel spreadsheet. You find out about it because some detailed-oriented manager reviews your report and spots the error. With AI, the trouble is people don't tend to double-check as frequently after they get comfortable with AI helping them (the garbage in, gospel out fallacy).

How humans and machines interact is crucial in safely using these tools. Think of a robotic floor vacuum, if you accidentally leave a door open it will still go into a room that you hadn't intended it to vacuum.  The consequences of a formula error or access to a room are all unintended. After all, we are just trying to get our stuff done, more quickly. So it is no surprise that occasionally we'll leave a door open?

Here's a list some of the issues that you might cause using unsupervised CoPilot:

• You inadvertently enter sensitive data into a prompt

• You ignore internal policies or bypass official systems and controls

• Your data ends up being processed or stored outside approved environments

• You inadvertently disclose private information via generated outputs

• You violate industry regulations (eg provide misleading advice)

• You apply outdated or incorrect legal interpretations

• You provide your boss with made up data (eg hallucinate)

• You progressively produce inconsistent outputs for similar inputs

• You miss cultural expectations and offend someone

• You miss the fairness checks

• You discriminate, in recruitment or service delivery processes

• You use Copilot in unapproved ways

• You integrate Copilot into workflows informally

• You accelerate work without checkpoints

• You make decisions faster than your internal governance processes can keep up

None of these are intended. But using an unsupervised Copilot system creates a systemic governance gap. Speed and efficiency wins at the expense of control, and potentially serious privacy or data breaches.

How to mitigate risk of unintended Copilot consequences

Firstly, if you are using Copilot regularly, raise awareness of the potential issues with your IT department or service provider. You need to know if what you are doing is appropriate or not. Hopefully, your company has already solved this problem but it's always good to check.

Secondly, if the appropriate administration controls in Copilot haven't been set (oops!), you will be amongst many friends in cyber-space. So someone in your organisation needs to ask Microsoft for guidance on what controls are currently available at the Enterprise level or user level. Here's a quick (non-exhaustive) summary of actions according to Microsoft:

1. Set access permissions for connectors to ensure sensitive content is not overshared.

2. Prevent Copilot from accessing or surfacing sensitive data in Teams, SharePoint, or email.

3. Configure privacy settings to ensure user conversations are not used to train the underlying model.

4. Turn on Microsoft 365 Unified Audit Logs to track user interactions and detect potential misuse, such as prompt injection attacks.

5. In Copilot Studio, enable human supervision for "computer use" tasks. When the model triggers certain actions (e.g., sending emails, browsing), it can be set to require a human reviewer to approve the action.

6. Use the "Send Feedback" option within the Copilot interface to report incorrect or inappropriate responses, which helps train and monitor the system.

7. Regularly check settings to ensure Copilot is not accessing unnecessary data, such as saved passwords or auto-fill data.

Although, using unsupervised copilot seems mostly harmless, there is an insidious side, with a gradual, unnoticed, and dangerous progression of data, privacy and policy violations if it persists.  In most use cases, you won't have much to worry about with Copilot and it will work just fine. But in high risks contexts, you need to be vigilant.

Relevant AI Governance Statements for Copilot implementation

• Technical Statement 1 - Define who owns the Copilot system in your company (by default they own its governance)

• Technical Statement 4 - Enable auditing and audit on a recurring cycle

• Technical Statement 5 - Ensure you can provide explainability to any decisions it makes

• Technical Statement 10 - Define where human oversight is required in the outputs or actions

• Technical Statement 38 - Undertake ongoing monitoring for unintended consequences, safety, security, compliance and so on

Stay safe, Bruce

ABOUT ME

I write all my own content, you can tell by the odd typo and occasional missing word. I use AI for research. I also teach organisations how to implement the Australian AI Governance Standard and confidently transition to AI systems. To learn about my upcoming public AI Governance workshops visit: Public workshops

To learn more about AI Governance, check out my Hitchhikers Guide to AI Governance Podcast visit:  Hitchhikers Guide to AI Governance Podcast