The Fluency Test Most Firms Are Failing
Last week, I wrote about the management infrastructure that sits beneath AI transformation: trust, guardrails, clear ownership, and an operating model that does not leave people guessing. This week, the gap has become more specific.
Most organisations have plenty of people who can use AI. Far fewer have people who know when to use it, how to challenge it, and where it genuinely adds value.
There is a difference. Literacy is knowing what a model can do. Fluency is being able to decide where it belongs in your team’s work, spot where it is weak, redesign a workflow around it, and explain the change to people whose jobs are moving under their feet.
Boston Consulting Group has put a name to the ambition. Alicia Pittman, BCG’s Global People Chair, says the firm is working towards “the most AI-fluent workforce in the world”, supported by an AI Fluency Ladder that BCG is now making available to clients. It is a serious ambition. It is also where many organisations will make their first mistake: treating fluency as a learning programme rather than a management capability.
Gallup’s research this week shows why. In a survey of 102 CHROs from global Fortune 500 companies, 99% said AI was important to their organisation’s strategy. Yet half said they were not very confident, or not confident at all, in their managers’ ability to guide employees on AI use.
That is the figure I would pay attention to. The board may understand the urgency. The CEO may have a compelling narrative. The technology team may have signed the contracts. But the layer expected to translate all of that into real work is telling HR that it may not be ready.
Gallup’s employee data makes the consequence visible. Among employees at organisations that have implemented AI, 25% say workplace culture has worsened over the past year and 24% say it has improved. The technology itself is not producing a consistent cultural outcome. The manager changes the experience. Employees who strongly agree that their manager champions AI are far more likely to say AI has transformed how work gets done, 33% versus 4%.
This is why I am not convinced by the current rush to give everybody a licence and a two-hour course.
A generic AI course may improve awareness. It does not tell a marketing manager how to rebuild campaign planning, or help an insights director decide what client data an agent can see. And it certainly does not prepare a finance leader to explain why one piece of work has moved to an AI-supported workflow while another still needs a human reviewer.
The most revealing detail in Gallup’s research is that 57% of CHROs are now providing AI training for people managers and 62% are creating centres of excellence or internal AI champions. Those are sensible moves. They will only work if the organisation gives managers real capacity to lead the change. A manager already carrying a broad span of control cannot become an AI translator, coach, quality reviewer, change lead, and business-as-usual operator by simply being sent a new deck.
McKinsey’s research into agentic product development adds the commercial case. It surveyed 334 product and engineering leaders and found that only 25% of director-and-above respondents reported meaningful or top AI acceleration. 30% said team productivity had fallen. The highest-performing organisations were not simply using better tools. They had redesigned workflows, changed roles and decision rights, built verification into the process, and used hands-on coaching to help people work differently.
That is the fluency test. Can your people use AI inside a new system of work, with clear standards and enough confidence to challenge the output? Or have they simply been taught where the chatbox is?
For C-suite leaders, that means three things.
1) Stop reporting AI fluency as a training-completion figure.
Track whether managers can explain where AI belongs in their function, what decisions require human review, and what has changed in their team’s workflow.
2) Build fluency around roles and moments of work.
A research lead, account director, HR business partner, analyst, and finance manager do not need the same programme. They need practical support inside the decisions they make every week.
3) Fund the manager layer properly.
Give managers time to practise, peer forums to compare what is working, access to specialist support, and permission to surface failed experiments early. If change management is treated as an extra task, the organisation will get shallow adoption and private workarounds.
The market will soon divide between companies that can say their people have been trained and companies whose people can actually work differently. The second group will be much smaller, and much further ahead.
A Watermark Is Not a Warranty
Anthropic has outlined how it plans to watermark Claude-generated text in future. The move is part of the company’s response to the European Union’s AI transparency requirements, which came into force for providers on 2 August.
The mechanism is clever. Claude will use a version of Google DeepMind’s SynthID-Text approach, subtly favouring one plausible word choice over another in places where meaning is unchanged. Across a sufficiently long passage, those choices create a machine-detectable probability pattern. A reader sees no visible marker. Someone with the right detection key can assess the likelihood that Claude was involved.
This is a meaningful development for anyone producing content, analysis, research, or executive papers with AI. It begins to answer a question that has become increasingly awkward in client work: where did this come from?
But a watermark is not a warranty.
Anthropic is also clear about the limits. The watermark cannot prove that a passage was written by a human, determine whether another AI system produced it, or distinguish between Claude writing something from scratch and heavily editing a human draft. It is also less effective on short passages, factual text, proofreading and code, where there are fewer stylistic choices available to encode.
Above all, a watermark cannot tell you whether the output is right.
That distinction matters more than it may first appear. MIT Sloan describes the “jagged AI frontier” as the nonintuitive pattern of AI strengths and weaknesses relative to human performance. Research cited by MIT found that generative AI can lift highly skilled workers’ performance by as much as 40% when used within its capability boundary. Used outside that boundary, performance falls by an average of 19 percentage points.
The danger is that AI can produce work that looks polished enough to bypass the normal scepticism. A beautifully written summary with a provenance mark may still omit the crucial interview, apply the wrong market assumption, smooth over contradictory evidence, or turn a hypothesis into a conclusion. The evidence has been made faster. It has not necessarily been made stronger.
For research and insights leaders, this is the beginning of an evidence-chain discipline. Every AI-supported output should be able to answer five plain questions:
MIT Sloan’s governance work this week puts the broader point well. Ethics and governance typically lag AI adoption by at least a couple of years, and leadership teams need to define the boundaries they are prepared to tolerate before a difficult incident forces the decision.
A strong evidence chain gives organisations more confidence to move quickly. They know where AI has added value, where human review remains essential, and where a polished answer should never have made it into a board paper or client recommendation.
Transparency is arriving. The harder work is deciding what your organisation counts as proof.
This piece first appeared in Talent Pools Nexus, our weekly read for leaders in research, insights and consulting. Subscribe on LinkedIn.
