What should you ask an agency about how it uses AI?

6 October, 2026

By Jonny Ross, founder of Fleek Marketing

Ask your agency four questions about AI, and listen for whether the answers describe a system or a person. Where does what you know about our business live? Who checks the work, and what do they check it against? What happens if the tool you use changes or disappears? Will the work be the same standard on a Friday afternoon as on a Tuesday morning? An agency that has built properly will answer each one plainly, with a specific example. One that hasn’t will talk about which tools it uses and how good they are now.

That distinction matters more than it did a year ago, because using AI is no longer what separates one supplier from another.

Wide adoption with shallow depth is exactly the situation where the question “do you use AI?” stops being useful. The same ONS research found average use rose only from around 1.4 to 1.6 AI technologies per business over the period. Plenty of teams have picked it up. Far fewer have built anything around it.

Why does it matter how your agency uses AI?

Because what you’re buying from an agency is a consistent standard of work, and AI can either protect that or quietly undermine it.

The most common pattern is AI that lives inside one person’s chat history. A good account manager builds up a long-running conversation that knows your brand, your customers and how you like things written, and the output is decent. Then they go on holiday, change role or leave, and the next person starts from nothing. I spoke about this at Leeds Digital Festival this month. When I asked a packed room who had an AI that knows their business, most hands went up. When I asked who could hand it to a colleague and get the same answer, almost none stayed up. Agencies are no different, and you’re the one who feels it when the quality dips.

The four questions, and what good sounds like

On a phone, scroll the table sideways.

Where does what they know about your business live?

In a good agency, it lives in writing, somewhere every piece of work reads before it starts. That applies to people and AI equally. A new team member and an AI tool should both begin from the same record of who you are, who you sell to, what you’ve tried and what you’ve decided.

This is how we run our own delivery. Each of the roughly 70 websites we host has its own written operating record, which any job on that site reads first, whoever or whatever is doing it. It sounds administrative. It’s the reason the person who picks up an urgent fix at short notice doesn’t have to rediscover how that site was set up.

If the answer you get is about a person, however good they are, the knowledge belongs to them rather than to the relationship.

Who checks the work, and what do they check it against?

Somebody should be checking AI-assisted work, and they should be checking it against a written standard, not a feeling. AI output is fluent, which makes weak work look finished. It also tends to agree with whoever is asking, so “does this look right?” is a poor test.

A good answer names a human reviewer and describes what they check: the brief, the brand rules, the facts, the claims. A better answer adds that the standard is written down, so two reviewers on different days would catch the same things.

What happens if the AI tool they use changes?

It shouldn’t matter much. AI products change their pricing, features and terms regularly, and some will disappear. An agency whose know-how sits in plain documents can move to a different tool in an afternoon. One whose know-how sits inside a single product’s chat history or custom settings is exposed, and so is your work.

This is also a fair test of how seriously they’ve thought about it. “We use the best tool” is a preference. “We could switch tomorrow” is a design decision.

Will the work be the same standard on a bad week?

This is the question most buyers never ask, and the one that matters most over a twelve-month contract. What AI done properly gives an agency is consistency: the same checks, applied the same way, whether the team is fresh on a Tuesday morning or stretched on a Friday afternoon.

Listen for routines rather than effort. “We work really hard to keep standards high” depends on people remembering. “Every page goes through the same checks before it goes live” doesn’t. The first is a promise, the second is a process.

If an agency leads on speed, ask what they do with the time saved. Speed on its own is only useful if it’s pointed at the right work, which is the argument in how we use AI and no-code tools in our own delivery.

Frequently asked questions

Should an agency tell me when it uses AI on my work? It’s reasonable to ask how they use it and to expect a straight answer. What matters more than a label on each piece is whether there’s a clear standard behind the work and a person accountable for it.

Is AI-assisted work cheaper? Sometimes, but price is the wrong thing to lead with. The value to you is consistency and fewer errors. An agency that passes on every saving by cutting the checking has made the work cheaper and worse.

What if my agency is small? Size isn’t the issue. A two-person agency with its knowledge written down will often beat a large one where everything lives in individual accounts. Ask the same four questions either way.

How do I check an agency’s claims if I can’t judge the work myself? Ask them to explain what they’re doing in words you could repeat to someone else. Jonny has written more on how to evaluate a supplier when you can’t judge the work yourself.

Last reviewed September 2026