Most enterprises can run an AI pilot. Far fewer can scale one.
McKinsey reports that while 88% of companies use advanced technologies in at least one area, only about one in three has managed to scale those efforts beyond pilots. That gap is rarely about the technology. It is about how the organisation is set up to run AI.
The structure most scaling enterprises share has a name: the AI Center of Excellence.
This guide explains what an AI CoE is, which operating model fits your organisation, and how to stand one up without creating another layer of bureaucracy.
What is an AI Center of Excellence?
An AI Center of Excellence (CoE) is a dedicated team that sets the standards, governance, tools, and talent strategy for AI across an organisation, so that individual business units can adopt AI consistently and safely rather than each reinventing the wheel.
It is not a research lab and it is not a single project team. It is the operating core that turns scattered experiments into a repeatable capability.
Think of it as the difference between ten teams each buying their own tools and writing their own rules, and one shared function that lets all ten move faster on a common foundation.
Why do most AI programmes stall after the pilot?
Most AI programmes stall because a successful pilot proves feasibility, not scalability. Scaling requires shared data standards, governance, reusable infrastructure, and change management, none of which a single pilot team is resourced to build.
McKinsey's finding that only about one in three companies scale beyond pilots reflects this structural gap. The pilot works, then the organisation has no mechanism to extend it.
Without a central function, each new use case restarts from zero on procurement, security review, and vendor selection. The organisation pays the setup cost again and again, and momentum quietly dies.
What does an AI Center of Excellence actually do?
An AI CoE owns the shared capabilities that every AI project needs, so that business units can focus on outcomes rather than plumbing. Its remit is broad but concrete.
A typical CoE is responsible for:
--- Governance: setting policy for data use, risk, and compliance, increasingly aligned to the ISO/IEC 42001 AI management standard.
--- Standards: defining how models are evaluated, monitored for drift, logged, and retired.
--- Talent: concentrating scarce AI skills and lending them to business units.
--- Tooling: providing a vetted, reusable platform so teams do not each buy their own.
--- Prioritisation: choosing which use cases get resourced, based on value and feasibility.
McKinsey notes that AI governance is often centralised precisely within these Centers of Excellence, to manage risk and data oversight as adoption spreads.
Which operating model fits your organisation?
There are three common CoE operating models: centralised, hub-and-spoke, and federated. The right choice depends on how your organisation already makes decisions and where your AI talent sits.
The three models compared:
--- Centralised: one team builds and runs all AI. Fastest to set standards, but can become a bottleneck. McKinsey reports more than half of businesses have adopted a centrally-led setup for generative AI.
--- Hub-and-spoke: a central hub sets standards and shared tools, while embedded specialists ("spokes") deliver inside each business unit. This balances consistency with local speed.
--- Federated: business units run their own AI within light central guardrails. Fast and autonomous, but risks fragmentation without strong governance.
Deloitte advises that a CoE functions best when aligned to the organisational matrix, striking the right balance between centralisation and flexibility. For most Hong Kong mid-market enterprises, a hub-and-spoke model is the pragmatic middle ground.
How do you set up an AI Center of Excellence?
You set up a CoE by starting small with a clear mandate, not by hiring a large team upfront. The goal in the first quarter is one governed, scaled use case that proves the model, not an org chart.
A practical sequence looks like this. First, secure an executive sponsor and a written mandate covering scope and decision rights. Second, pick two or three high-value use cases and set shared standards around them.
Third, stand up the minimum shared tooling and governance, including model evaluation and monitoring. Fourth, deliver one use case end to end, document it, and use it as the template for the next.
The value compounds because the second use case reuses the first one's foundation. To see how model choice fits into this, our guide to small language models is a useful companion, alongside the wider UD Insight hub.
What goes wrong when building a CoE?
The most common failure is building a CoE that governs but does not deliver. When the central team only writes policy and never ships a working use case, business units see it as a checkpoint to route around rather than a partner.
A second pitfall is over-centralising. If every request must pass through one team, the CoE becomes the bottleneck it was meant to remove, and shadow projects appear.
A third is neglecting change management. Tools and standards are the easy part. Getting people to adopt new ways of working, without damaging morale, is the harder and more decisive task.
Key facts to remember
AI Center of Excellence at a glance:
--- Definition: a dedicated team owning AI standards, governance, tooling, and talent across the organisation.
--- Why it matters: McKinsey finds only about one in three companies scale AI beyond pilots.
--- Models: centralised, hub-and-spoke, or federated; McKinsey reports over half of businesses run gen AI centrally.
--- Governance standard: increasingly aligned to ISO/IEC 42001 for AI management.
--- Start small: one executive sponsor, two or three use cases, one governed win in the first quarter.
The strategic takeaway
An AI Center of Excellence is not a bureaucracy. Done well, it is the mechanism that lets an enterprise move from one clever pilot to dependable, organisation-wide capability, without repeating the setup cost every time.
The hard part is not the diagram. It is choosing the operating model that fits how your organisation actually works, and delivering an early win that earns trust. That is where a partner who has helped organisations through several technology cycles earns their place. UD has walked alongside Hong Kong enterprises for 28 years, so that when AI feels cold and complex, you are not facing it alone.
Build an AI Capability That Actually Scales
Standing up an AI Center of Excellence is a strategic decision that shapes every AI project that follows. We'll walk you through every step, from an AI readiness assessment to operating-model design, governance, and your first scaled use case, backed by 28 years of Hong Kong enterprise experience.
Reviewed by the UD AI team.
