Generative AI Data Leaks: A Business Guide to ChatGPT and Copilot

An account coordinator at a Montréal marketing agency needs to segment a list of 300 clients before a campaign launch. Short on time, she pastes the entire file, names, emails and purchase history included, into the free version of a chatbot on her personal account. The output is great. Nobody will ever know where that data went.

In Gatineau, an accounting firm switches on Copilot for the whole team. The next day, a summer intern asks, "What does everyone here earn?" Copilot answers: the payroll spreadsheet had been shared with "Everyone" in SharePoint for three years. Nobody hacked anything.

That's what most generative AI data leaks look like: not a dramatic breach, but ordinary use on top of neglected settings. The good news is that both can be fixed.

How leaks actually happen

The Canadian Centre for Cyber Security sums it up plainly: staff may enter sensitive corporate data or personal information into AI prompts, where it can end up exposed. In practice, five scenarios keep coming up:

1. Copy-paste into a personal account. A client list, a patient email, a contract to summarize.
2. Document uploads. Financial statements, bids, HR files sent "to save time."
3. Browser extensions and meeting bots. A plugin that reads every page you visit, a transcription assistant that joins Teams calls, sometimes without participants realizing it.
4. Internal oversharing. An assistant connected to your files finds everything the user can technically open, including things they'd never have stumbled on alone.
5. Shadow AI. Accounts created with a work email address, without leadership or the privacy officer knowing.

Personal account vs business account: what changes

The gap between consumer and business versions is real, but often misunderstood.

Note that Microsoft recently renamed Microsoft 365 Copilot to "Microsoft Copilot"; according to Microsoft, its security and privacy commitments haven't changed. Microsoft also states that customers outside the European Union may have their queries processed in the U.S., the EU or other regions. That detail matters under Law 25.

Copilot: the real risk is oversharing

According to Microsoft, Copilot only surfaces data the user has at least view permission for. That's reassuring, provided your permissions are clean. In many small businesses, years of "Everyone" links and inherited folders say otherwise.

Before turning Copilot on for the whole team:

• find SharePoint sites, Teams and OneDrive folders shared organization-wide or through anonymous links;
• apply sensitivity labels (Microsoft Purview) to payroll, HR and client folders: Copilot honours the usage rights on encrypted content;
• start with a pilot group and test what Copilot returns to sensitive questions;
• retain and review interaction history with your compliance tools.

Our Microsoft 365 security audit article covers the sharing settings to check first.

The Law 25 angle: what AI changes for Quebec businesses

Generative AI doesn't come with a new Quebec law, but it triggers several existing sections of the Act respecting the protection of personal information in the private sector. (Outside Quebec, federal and provincial privacy laws set their own, broadly similar expectations.)

• Section 17: communication outside Quebec. Before communicating personal information outside Quebec, a business must conduct a privacy impact assessment (PIA). It weighs the sensitivity of the information, the purpose, the protection measures, including contractual ones, and the legal framework of the destination. The communication must be covered by a written agreement. The same applies when you entrust a provider outside Quebec with processing or storing that information on your behalf. "Outside Quebec" includes Ontario.
• Section 3.3: new systems. Any project to acquire or develop an information system involving personal information requires a PIA. Connecting an AI assistant to your CRM counts. Our article on privacy impact assessments and security explains the process.
• Section 18.3: the provider as mandatary. The contract must be in writing and spell out protection measures, use limited to the mandate and destruction at the end.
• Section 12.1: automated decisions. If a tool screens résumés or credit applications on its own, the person must be informed.

And if an employee has already pasted personal information into an unapproved tool? That can amount to a confidentiality incident, to be logged in your incident register and assessed for risk of serious injury. These are legal calls: confirm them with your legal counsel.

Usage policy template: the checklist

A generative AI policy fits on one page. At a minimum, it should cover:

• The list of approved tools and the requirement to use the company account.
• A simple data classification: public, internal, confidential, personal information, and what's allowed in each tool.
• Explicit don'ts: personal information in an unapproved tool, passwords, documents under a non-disclosure agreement.
• The rule to anonymize or redact before submitting a document.
• Mandatory human review: AI can be wrong, and the author stays accountable for the result.
• Prior approval for browser extensions and meeting bots.
• What to do if sensitive data was submitted by mistake.
• Training at onboarding, then an annual refresher.
• A named policy owner and a review at least twice a year.

Rolling it out in four weeks

1. Week 1: inventory. An anonymous survey on actual use, plus a look at the apps connected to your Microsoft 365 or Google Workspace accounts.
2. Week 2: choice and PIA. One business tool, a documented assessment, a reviewed contract.
3. Week 3: clean-up. Permissions, sharing links, sensitivity labels.
4. Week 4: launch. Signed policy, 30-minute training, unapproved tools blocked through web filtering.

Our team can help with the PIA, the policy and the configuration through our Law 25 compliance service. The goal isn't to slow AI down. It's to know exactly where your data goes.