ICO AI Data Protection: What UK Businesses Need to Know.

The ICO Is Looking More Closely at AI Agents. Should Your Business Be Concerned?

The UK's data protection regulator, the Information Commission (still operating under the familiar ICO name), has secured improvements and commitments from ten of the biggest AI foundation model developers operating in the UK and is now turning its attention to AI agents.

If your business uses AI to answer calls, handle enquiries or manage customer information, this is worth understanding.

Imagine you've introduced an AI receptionist into your business.

It answers the telephone, checks your appointment diary, books customers in and sends confirmations. It works outside office hours, doesn't take holidays and saves your staff from spending half the day answering routine questions.

Sounds pretty useful.

Now imagine someone asks that same AI to change an appointment. It checks the customer record, finds the booking and makes the change.

But what else can it see?

• Can it access every customer's details? Their previous appointments? Private notes? Financial information? Can it see information belonging to someone other than the customer it's helping?
• And what happens if it misunderstands an instruction or somebody deliberately tries to mislead it?

These aren't hypothetical questions about some distant future of artificial intelligence. They're questions businesses should be asking about systems they can buy and install today.

And they're increasingly the sort of questions the ICO is asking too.

What has the ICO announced?

On 8 October 2026, the ICO published the findings of its scrutiny of major AI foundation model developers, outlining the improvements already made and the further commitments secured around the handling of personal information.

The companies are: Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI and Stability AI.

These companies develop the foundation models behind many of the AI tools we use today, from familiar chatbots to increasingly sophisticated AI assistants. These models are trained on enormous amounts of information, which can include personal data, and can then be adapted for different purposes.

Following scrutiny from the ICO, the developers have made improvements or agreed to changes in three main areas:

• Transparency - providing clearer information about how people's personal data is collected and used.
• Individual rights - strengthening the mechanisms that allow people to exercise their data protection rights.
• Safeguards - carrying out more rigorous assessments of the measures intended to protect personal information.

There's an important distinction here. The ICO hasn't given these companies a clean bill of health or declared their systems fully compliant with UK data protection law.

Some changes have already been made. Others remain commitments, and the regulator is monitoring the developers' progress.

The ICO also highlights some difficult questions about how AI models are trained, particularly when sensitive personal information is involved and whether that information can remain within a trained model.

That's a particularly interesting question. If personal information becomes part of a trained AI model, what does that mean for an individual's rights over that information?

The ICO acknowledges that current AI training methods can make compliance with UK data protection law challenging, particularly around data protection by design principles. They also recognise these issues cannot be resolved by developers alone, and they're working with the Government and AI industry to find practical solutions.

Those are significant issues for the wider AI industry. But there's another part of the announcement that I think deserves considerably more attention from UK businesses. 

The ICO is now looking more closely at AI agents.

Why AI agents change the conversation

Most people are familiar with ChatGPT and similar AI tools. You ask a question, the AI produces an answer, and you decide what to do with it.

AI agents take things further. Instead of simply telling you how to complete a task, they can potentially carry out the work themselves.

• An agent might read an email, look up a customer, check a booking system, update a record and send a confirmation. Some can use web browsers, interact with business applications and work through tasks with limited human supervision.

That's what makes them so useful, but it also introduces new risks.

Consider a hotel using AI to handle enquiries. A customer asks about booking a conference room, and the AI checks availability, prepares a quotation, records the enquiry and schedules a follow-up.

That's useful automation.

• But what happens if the AI confuses two customers, sends a quotation to the wrong person or changes a booking it wasn't supposed to touch?
• The problem isn't simply that the AI might produce an incorrect answer. It's that it can take an action, potentially affecting customer information or business records.

When AI can do that, data protection needs to be considered from the outset, not added as an afterthought. 

The ICO is particularly interested in what happens as these systems become more independent.

It has recently made enquiries with OpenAI, Anthropic, Meta and the UK's AI Security Institute following reports of AI agents bypassing safeguards, using unauthorised communication channels and accessing external systems, including Hugging Face, during testing and deployment.

These are reported incidents, not findings that those organisations have broken data protection law. However, they raise important questions about security, accountability and how much control organisations have over their AI systems.

The ICO has also launched a six-week call for evidence, asking AI developers, businesses deploying agents and other experts how they're managing these risks.

The findings will help shape future guidance and the ICO's forthcoming statutory code of practice on AI and automated decision-making.

The call for evidence closes on 20 November 2026.

Your AI receptionist doesn't need to know everything

Let's return to our AI receptionist.

Imagine it works for a healthcare practice.

Someone calls to rearrange an appointment. The AI needs to verify who they are, find the correct booking, check availability and make the change.

Depending on how the system is connected, it might also have access to treatment information, medical histories or other sensitive details.

• But does it need access to every patient's records? No.
• Does it need to see confidential medical notes simply to rearrange an appointment? Of course not.

This brings us to two important principles: data minimisation and access control.

Under UK GDPR, organisations should only process the personal information necessary for a particular purpose. Access controls help enforce that principle by limiting what information a system can retrieve and what actions it can perform.

These aren't new ideas. They applied long before AI arrived.

What's changed is how easily an AI agent can move between connected systems and retrieve information if we've given it permission.

It's tempting to connect an AI assistant to everything and let it work things out. After all, more information might make it more useful.

But more access also creates more opportunities for mistakes, inappropriate use of personal information and security incidents.

A well-designed AI booking system should retrieve only the information needed to complete the booking, using controlled access to the practice management system rather than unrestricted access to patient records.

The same applies to estate agents, solicitors, retailers, hotels, and practically any business that handles customer information.

Give the AI the information it needs to do the job, not unrestricted access to everything else.

Isn't GDPR the AI supplier's responsibility?

This is where some businesses could get caught out.

They purchase an AI service from a recognised supplier, see reassuring statements about security and GDPR compliance, and assume everything has been taken care of.

Unfortunately, it doesn't work quite like that.

If your business decides why and how customer information is processed, you're generally the data controller for that activity. Your AI supplier may be a data processor, handling information on your behalf, although its exact responsibilities depend on how it uses that data.

As the controller:

• you're responsible for understanding what personal information the AI uses, why it needs it, who can access it and how it's protected.
• you also need to consider how long information is retained, whether it's transferred outside the UK and how customers can exercise their data protection rights.

For example, suppose a customer asks you to correct inaccurate information. You might be able to update their record in your booking system, but what about copies held in AI conversation histories, logs or other connected services?

You need to know where that information is held and how corrections can be handled across the systems involved.

Where an AI supplier processes personal data on your behalf, you'll generally need a data processing agreement under Article 28 of UK GDPR, setting out how that information must be handled and protected.

If the supplier also uses customer information for its own purposes, you'll need to understand that arrangement and the respective responsibilities.

A supplier's claim that its product is GDPR compliant may be reassuring. It doesn't automatically make your particular use of that product compliant.

The biggest security risk might be what you've connected

When people think about AI security, they often concentrate on the model itself.

Is it safe? Is the provider trustworthy? Does it use our customer information for training?

Those are sensible questions. But with AI agents, I'd be equally interested in what the system can access and what it's allowed to do.

Imagine you've connected an AI assistant to your business email and customer database. It reads incoming enquiries, identifies the relevant customer and prepares a response.

Perfectly reasonable.

• Now imagine an email arrives containing instructions deliberately designed to manipulate the AI. Not instructions from you, but from somebody outside your business.
• The email might attempt to persuade the AI to ignore its original task, retrieve confidential information or send customer details somewhere they shouldn't go.

This is known as prompt injection.

We've explored this in more detail in our GeekyBee article, When Your AI Takes Instructions From Strangers, which explains how an AI can mistake instructions hidden in emails, webpages or other external content for legitimate commands.

Whether an attack succeeds, and how much damage it could cause, depends partly on the safeguards in place and what the AI is authorised to do.

An AI assistant that can only draft a reply has relatively limited powers. One that can access customer records, change information and send emails without approval presents a much greater risk.

• Simply telling the AI not to do anything inappropriate isn't enough. You need technical controls that limit what it can access and what actions it can perform.
• Restrict access to the necessary records, require approval for sensitive actions and maintain logs of what the system has done.

And if something unexpected happens, the system should have safeguards that prevent further unauthorised actions and refer the task to a person when necessary.

Ultimately, AI security isn't just about choosing a trustworthy model provider. It's about how the AI is connected to your business systems, what information it can access and what safeguards are in place to prevent things going wrong.

What happens when AI starts making decisions?

There's another important distinction between AI helping someone make a decision and making that decision itself. 

For example, an AI might review customer enquiries and suggest which ones need urgent attention.

That's useful assistance.

• But what happens when AI starts automatically rejecting applications, deciding whether someone qualifies for a service or making other decisions that could significantly affect people's lives? 

Now we're entering the territory of automated decision-making.

UK data protection law places specific requirements on automated decisions that have legal or similarly significant effects on individuals. 

Changes introduced by the Data (Use and Access) Act 2025 allow businesses to make some of these decisions automatically, provided appropriate safeguards are in place. These include giving individuals information about decisions affecting them and opportunities to challenge them or request human intervention.

Stricter restrictions apply where sensitive personal information, such as health data, is involved.

Businesses need to understand when AI is simply providing recommendations and when it's making decisions on their behalf.

And where human oversight is part of the process, it needs to be meaningful. 

If a member of staff simply clicks Approve without understanding or questioning the AI's recommendation, that may offer very little real protection.

Seven questions I'd ask before connecting AI to customer data

You don't need to understand every detail of how a large language model works to ask sensible questions about how an AI service will handle your customers' information.

Here are seven I'd start with:

1. What information can the AI actually see?

Ask which systems it connects to, what customer information it can access and whether that access is restricted to what's needed.

2. Does it need all that access?

If the AI only books appointments, why would it need access to financial records, marketing information or unrelated customer files?

3. Where does the information go?

Find out where customer information is processed and stored, whether other suppliers are involved and whether any data is transferred outside the UK.

Don't assume that because your business and customers are based in the UK, all the information stays here.

4. Is our customer information used to train AI models?

Check whether your information is used to train or improve AI models, and what controls you have over that use. Different suppliers, products and account settings can have different arrangements.

5. What happens when it gets something wrong?

Can you see what the AI has done, correct mistakes and reverse actions where necessary? Is there a clear process for dealing with errors or unexpected behaviour?

6. Does it know when to stop?

If the AI can't complete a task within its authorised permissions, does the system stop and ask for help, or does it keep trying different approaches?

Personally, I'd rather have an AI tell me it couldn't complete a booking than discover it had found an unexpected way into another system.

7. Have we properly assessed the risks?

Some uses of AI will require a Data Protection Impact Assessment (DPIA), particularly where processing is likely to present a high risk to individuals.

Even where a formal DPIA isn't legally required, understanding and documenting the risks is sensible.

The answers to these questions should influence how the AI system is designed, configured and monitored, with clear responsibility for keeping it safe and compliant.

Not simply be filed away with the supplier's paperwork.

Does this mean we should stop using AI?

Not at all.

In fact, well-designed AI automation can actually improve how businesses handle and protect personal information.

Consider two approaches to building an AI receptionist for a healthcare practice:

• In the first, the AI has broad access to the practice management system and is trusted to retrieve whatever information it thinks it needs.
• In the second, the AI operates under strict access controls, allowing it to retrieve only the information it needs and carry out specific, authorised tasks.  It can check whether an appointment exists, retrieve the details needed to verify the caller and request an authorised booking change. It doesn't have unrestricted access to patient records or the underlying database.

To the customer, both systems might look identical. Both could successfully rearrange an appointment.

• But behind the scenes, the second gives the business much greater control over what information the AI can access and what actions it can perform.

That's the distinction that matters.

AI doesn't need the keys to your entire business to be useful. Sometimes, all it needs is access to one carefully controlled door.

What should businesses do following the ICO announcement?

I wouldn't suggest rushing to switch off every AI system in your business.

But I would use the announcement as an opportunity to review what you've already connected and whether the right safeguards are in place.

Start with the obvious systems: AI receptionists, customer-service chatbots, email assistants, CRM automations and any internal AI tools that process personal information.

• Look at what each system can access and what it's allowed to do.
• Does it have more permissions than it needs? Can it change or delete information? Can it send messages without approval? Are its actions recorded so you can investigate mistakes or unexpected behaviour?
• And importantly, do you know who's responsible when something goes wrong?

An AI booking assistant that simply checks availability presents a different level of risk from an agent that can issue refunds, alter financial records or disclose confidential information.

The safeguards should reflect those differences. 

And don't treat this as a one-off exercise. As AI systems gain new capabilities or connections, their permissions and safeguards should be reviewed too.

The ICO's six-week call for evidence on agentic AI remains open until 20 November 2026. It's gathering information from developers, businesses and other experts to help shape future guidance and its forthcoming statutory code of practice on AI and automated decision-making.

This doesn't mean a new set of data protection rules suddenly came into force on 8 October. The underlying responsibilities already exist.

What's changing is what AI systems are capable of doing, and the safeguards businesses need to put in place to keep those systems under control.

The real question isn't whether AI can do the job

I spend a lot of time looking at how AI can take routine work off people's desks, and the technology is becoming remarkably capable.

It can answer calls, manage bookings, follow up enquiries and work across systems that previously needed somebody to move information between them manually.

For businesses, there's a genuine opportunity here.

But there's a difference between an AI system that can complete a task and one that's been designed to do it properly.

• If we're giving AI access to customer information, we need to understand what it can see, what it's allowed to change and what happens when something unexpected occurs.
• That doesn't mean making automation unnecessarily complicated. It means designing it around the business process, with appropriate access controls, safeguards and human oversight where needed.

The ICO's latest announcement is a timely reminder that responsibility for customer information doesn't disappear just because we've handed part of the work to AI.

And perhaps that's the most important question to ask when considering any AI agent: We know it can do the job. But have we made sure it can only do what we've authorised it to do?

If you're considering AI automation for your business, GeekyBee helps businesses design and integrate AI solutions with appropriate access controls, safeguards and oversight. Talk to us about your requirements.

Research checked and verified on 9th October 2026