Organizing the AI Centre of Excellence: Combining CoE Capability with Delivery Expertise

Strengthen the CoE you need

Before creating a new centre, find the missing owner or unfunded service in the structures you already have.

An existing CoE, architecture practice or platform team may already perform much of this work. Begin by testing the services it actually provides. A new organization chart will not resolve a missing owner, an unfunded service or a queue nobody has capacity to clear.

Retain responsibilities that work. Strengthen a service when demand exceeds its capacity; add one only where the capability is missing. Compare adoption, waiting time, rework and full cost before changing the structure.

Start in one domain that clearly needs change and has a sponsor willing to own it. Place scarce AI expertise there first. Other domains can use its services without building a centre of their own, and take over delivery only once they have the capacity to run it and can meet the agreed conditions. Microsoft similarly relates centralized, hybrid and federated structures to maturity and risk.

For example, a bank’s fraud team may already test and validate AI models well. Rather than building a new centre to do the same work, fund that team to offer validation as a common service to other domains.

Agree on the service owner, access conditions, response commitment, full cost and transfer terms. Executive investment and risk authorities resolve cross-domain priorities. A hosting domain cannot change enterprise policy on its own.

Four services fulfil the CoE mandate

AI Strategy and Portfolio

Decides which AI ideas get funded, and makes sure each has an owner and a business case.

This function converts business priorities into the CoE service and adoption roadmap. It receives and evaluates new AI proposals (intake), identifies dependencies and prepares investment recommendations. The first roadmap covers the new bounded portfolio and its reusable foundations; enterprise coordination grows with demand. Each proposal needs a business sponsor, a measurable problem, an initial view of data readiness and an accountable owner for the expected outcome. Portfolio management should distinguish an experiment from a production commitment. An experiment receives bounded funding to resolve a question. A production initiative requires a credible operating model, delivery capacity and a business case that includes ongoing costs. Finance should validate financial assumptions and confirm how benefits will be recognized.

AI Architecture and Engineering Enablement

Gives teams AI specialists, reusable engineering assets and a marketplace of proven solutions, so no one starts from scratch.

This function provides AI specialists, reusable engineering assets, model-selection guidance and evaluation methods. The function works with Enterprise Architecture on approved patterns and asks platform and engineering teams to build agreed common capabilities (commissions). Each commission needs an owner, funding, consumers and acceptance conditions. The CoE may build early or complex solutions where specialist expertise is concentrated. Business teams continue managing the solutions they own, with named delivery and operating support. Business makers need protected environments, approved building blocks and a supported route from prototype to service.

The marketplace of AI accelerators and agents. This function maintains the marketplace: qualifying entries, keeping ownership and supported versions current, and coordinating changes, support and retirement with their owners. Each entry needs a purpose, permitted data and actions, acceptance evidence, usage and cost information, and a support route.

AI Governance Operations

Keeps track of all AI in use, and makes approvals, exceptions and evidence easy to find.

This function runs the operational processes that make AI oversight usable. It maintains the AI inventory, coordinates assessments, tracks approvals and exceptions, and assembles the evidence required for decisions. The inventory should cover internally developed AI, acquired services and AI features enabled within existing applications. Policy authorities retain their requirements and decisions. Independent validation challenges the evidence and reports its findings directly, not through the people who built the AI it assesses. Building, sharing and publishing require distinct permissions, including publication to the marketplace.

AI Adoption and Capability Development

Builds the skills people need to use, deliver and oversee AI, matched to the decisions they make.

This function works with Human Resources (HR), Learning and business change leaders to build the skills needed to use, deliver and oversee AI. Training should reflect the decisions people make. A frontline user needs to recognize unreliable output and know when to escalate; an engineer needs evaluation and control expertise; an executive needs to understand risk acceptance and the economics of adoption. Business leaders own changed workflows and local adoption. HR owns enterprise learning arrangements and workforce implications; the CoE supplies AI expertise, learning content and communities of practice. Add practical maker coaching and qualification for permitted activities; assess useful adoption rather than course attendance.

These services fulfil the three mandates in different combinations:

• Drive Business Value: AI Strategy and Portfolio; AI Adoption and Capability Development
• Build IT for Agentic Speed and Control: AI Architecture and Engineering Enablement
• Govern AI at Enterprise Scale: AI Governance Operations, connecting both to enterprise authorities, with Finance and business owners validating the economics

Drive Business Value

Consider an illustrative year-end review service. An employee’s review should reflect the work they did. When every manager starts with a blank page or invents a different prompt, the enterprise gets uneven coverage, inconsistent language and repeated effort.

HR sponsors the review-writing initiative and defines the review structure, performance criteria, evidence requirements and measures of success. The CoE helps HR develop the solution using supported model access, reusable prompt patterns, evaluation methods and specialist expertise. Depending on the skills available, an HR consultant, a delivery team or the CoE may build the initial tool.

Consider two managers reviewing similar roles. One has detailed project notes; the other has only brief updates. A useful tool asks both to connect their judgments to the agreed objectives and identify supporting examples. It flags unsupported statements and missing evidence. It should improve the basis for a review rather than turn thin notes into a convincing but unsubstantiated assessment.

Once HR accepts the solution, it can be included in the AI solutions marketplace with HR identified as its business owner. HR continues to manage the review criteria, prompt content, enhancement priorities and adoption through agreed delivery and support arrangements. Other business units can use the service under the established HR standards and access conditions.

Managers remain accountable for the evidence, judgments and final assessments. The tool does not assign ratings or make employment decisions. Access remains limited to the records each person is authorized to use.

HR measures review quality, preparation time, rework and missing evidence against its baseline. The CoE measures the usefulness and reuse of the common capabilities it provides. Consistent criteria and a better-supported narrative are the aim; different performance should still produce different assessments.

Enable Business Users and Citizen Developers

Buying AI licences gives people access to a tool. It does not tell them what they can build, which information they can use or when a useful experiment is ready for colleagues. The CoE should make those decisions easy to understand and provide a practical route for business ideas to become supported services. Citizen developers are business specialists who build or configure solutions using approved tools, with support appropriate to the work.

We recommend three activities with clear boundaries. Use approved AI for permitted drafting, analysis and other tasks, with training in checking the output. Let trained business makers configure prompts, assistants and small workflows within an approved development scope. When a solution is to be shared, establish its business owner, eligible users, independent tests, release conditions and support. Sensitive information, complex integration or consequential actions can require professional IT involvement from the outset. A private prototype can still carry material risk.

Keep experiments in protected environments with permitted or synthetic data, approved connectors, bounded spending and coaching. Separate development, testing and production as the service warrants. Platform controls must enforce the agreed boundaries. An instruction in a prompt cannot substitute for access control.

An HR consultant could configure the review prompts and workflow using CoE-provided patterns, approved tools and coaching. Where additional expertise is needed, the CoE can help build and test the initial solution. HR defines the requirements and accepts the result against its review standards.

The solution can then be listed in the marketplace with its purpose, business owner, supported version, eligible users, data boundaries and support arrangements. HR continues to manage the solution, using the agreed process for changes and releases. The CoE provides continuing access to shared expertise and reusable capabilities.

Share the Service and Protect Each User

Many people using the same AI tool creates an operating responsibility. The enterprise needs to know which version they are using, who can see their information and who will respond when it fails. A common service should preserve each user’s access boundaries, conversation history and permitted context.

Make reuse easy to find and safe to use

The CoE maintains the marketplace catalogue, with each entry’s purpose, approved configuration, eligible users and data, accountable owner, supported version, limitations, change process and support route. Named owners keep their accelerators and agents current. The CoE tracks qualification, uptake, usage and cost, and coordinates correction or retirement of unsupported entries. Building and publication require distinct permissions.

Enforce access through the connected systems

For interactive use, verify each person’s identity and enforce their permitted data and actions through the service and its connectors. Sharing an assistant must not silently lend the maker’s wider permissions to everyone else. Protect user history, retrieved records and persistent memory from unintended cross-user access. An unattended workflow needs a separately approved workload identity, defined scope and operating owner; requiring a person to sign in at every step is not a workable substitute.

Plan for demand and maintain the service

For the HR tool, HR manages its business purpose, review criteria and enhancement priorities. Named delivery and operating owners provide funded technical support. The CoE owns the common AI capabilities assigned to it. Reuse follows the marketplace entry’s access, support and change conditions.

Keep dependency and version records so the service survives its maker changing roles. Approved routine changes can follow automated release paths. Changes to purpose, data sources, permissions or common criteria reopen the affected assessment, with independent tests proportionate to consequence.

For the HR service, plan for the year-end peak: concurrent managers, model capacity, licence limits and response times. Attribute calls and retries to the originating workflow, set budgets and alerts, and measure cost per accepted review alongside quality and rework. Sharing a tool does not automatically reduce its cost. A manager editing an individual draft is ordinary work; changing the shared assessment criteria is a governed service change.

Build IT for Agentic Speed and Control

Development owns the engineering transformation from the Software Development Lifecycle (SDLC) to an Agentic Development Lifecycle (ADLC). The CoE leads AI capability enablement and coordinates dependencies. Enterprise Architecture, platform, Quality Engineering (QE) and service owners retain their decisions and delivery responsibilities.

ML arteka brings CoE enablement and delivery expertise into one agreed plan. We work with the client to define the business problem, available data, scope and acceptance conditions, then organize the required architecture, engineering and assurance work. Each initiative needs named delivery and receiving service owners. Architecture establishes interfaces, governing context, permissions and operating conditions. The client retains investment, risk and release decisions.

Engineering produces versioned specifications, contracts and a bounded implementation. Changes to requirements, models, prompts or material context reopen the affected baseline and checks. QE develops independent test criteria from business rules, risk scenarios and expected outcomes, then reconciles coverage to the requirements. It must be able to challenge a shared mistake in the specification and code.

Before release, verify applicable evidence against the exact candidate and environment. Prove that the receiving resource enforces identity, action scope and limits, and exercise failure and recovery. The service owner accepts monitoring, support, recovery and maintenance obligations. Operating results feed the backlog and prompt reassessment when assumptions change. Existing Agile, DevSecOps and quality practices provide the foundation.

Make the evidence usable for client decisions

A CoE should make it easier to decide whether an AI service can be used, what needs correction and what must stop. Establish evidence requirements when the work is scoped. Keep the intended outcome, applicable rules, exact candidate, independent challenges and operating conditions traceable through delivery.

Business and control owners approve the applicable rules. QE protects the criteria and expected results from unilateral change by the maker. Verifiers check actual effects, including incomplete transactions, duplicate actions and recovery. A successful model response or a passing summary score cannot establish that the business state is correct.

Use applicable evidence for approved routine release paths and route material exceptions to the authorized owner, preserving the decision, conditions, evidence versions and expiry or reassessment trigger. The CoE coordinates that record; the delivery, control and service owners retain their decisions. Appendix A maps supporting assets to these capabilities.

Govern AI at Enterprise Scale

An initiative needs separate decisions on investment, permitted use, technical fitness and readiness to operate. Executive leadership should delegate each decision to a named role and specify what requires escalation. Risk acceptance must remain within the authority and limits of the accepting role; mandatory policy or legal obligations cannot be waived through a project decision. The charter names the release, incident and restoration authorities and their alternates. An operator needs a specific role to contact when a service fails.

The designated AI policy authority owns classification rules; the authorized risk role confirms the applicable tier and assurance scope. Data owners authorize purpose and access. Security, Privacy and other control authorities retain their clearances. The CoE maintains the inventory, coordinates assessments and tracks exceptions, including AI acquired inside existing applications.

Named accountability does not require a manual committee review of every routine change. Authorized owners can approve policy-bound automated paths for defined change classes. A changed intended use, model, governing context, tool permission or risk basis reopens the affected decision. Material exceptions reach a person with authority and response capacity; unrelated approved work should continue.

Keep delivery testing, independent validation and Internal Audit distinct. Evaluation tests the complete workflow, exceptions and recovery. Independent validation challenges fitness evidence in proportion to consequence; Audit assesses the governance and control system. The National Institute of Standards and Technology (NIST) describes governance as continuous and cross-cutting across the lifecycle.

Delegate immediate containment within defined limits. Restoration requires evidence that the service can operate within its approved conditions and that affected business transactions have been reconciled or remain in an explicitly owned compensation process. A healthy service does not establish that every customer outcome is complete.

Stop paying to rediscover the same prompt

A manager can spend an afternoon asking AI to rewrite the same review without improving the evidence behind it. Each attempt consumes tokens and review time. Tokens are the units a model processes: input includes instructions, documents and conversation history; output is what it generates. FinOps and Tokenomics connect that consumption to the cost and value of a review accepted through HR’s process.

Consider 1,000 managers each preparing 20 year-end reviews in an annual cycle. In the less efficient approach, each manager invents a prompt, pastes a large pack of notes and asks for repeated revisions. A shared tool applies HR’s tested structure and criteria, requests the relevant evidence and identifies gaps. Managers use only permitted employee information and check the draft before approval. Both routes must meet the same review and evidence requirements.

Assume six calls per accepted review, averaging 6,000 input and 1,000 output tokens per call, versus two calls averaging 4,000 input and 1,000 output tokens. At illustrative rates of $2 per million input tokens and $10 per million output tokens, model spend for the cycle falls from $2,640 to $720. The potential $1,920 difference is model spend only, before building and maintaining the library. Both examples assume uncached input and no separately billed charges. These are hypothetical averages, not measured savings or vendor prices.

If the improved method also removes six minutes of review and rework per review, it releases 2,000 staff hours over the review cycle. Measure that separately from the token bill and identify how the capacity will be used. Finance should assess the effect on cash costs separately.

The CoE helps HR create the initial review-writing solution and provides the common prompt-library capability, supported model-access patterns and methods for measuring consumption. HR owns the review template, evidence rules and approval process. Once the solution is available through the marketplace, HR continues to manage its content, priorities and adoption, supported by the agreed technical services.

Managers supply relevant evidence and review the resulting drafts. HR and Finance assess whether the changed process improves review quality, releases useful capacity or reduces expenditure. The CoE uses the findings to improve its common capabilities and help other teams adopt supported solutions. Measure all attempts and the complete service cost before claiming a benefit.

Make development efficient without weakening assurance

A developer asks an agent to change the account-servicing application programming interface (API) for an authorized contact-detail update but supplies a broad objective and repeatedly resends large parts of the repository. The agent searches, rewrites code and generates lengthy explanations; failed builds restart the cycle. The developer spends more time directing the agent and correcting its work, while token consumption grows without an accepted change.

The CoE provides maintained, versioned accelerators that Engineering can adopt: specification templates, reusable API patterns, tools for retrieving relevant repository context and agents configured to produce bounded changes. It may help Engineering build an initial agent or accelerator and include it in the marketplace. Engineering can continue managing that solution, with a named owner, supported versions and agreed maintenance arrangements.

In this example, the developer uses the supported template to define the API change, interface contract and acceptance conditions. The accelerators retrieve relevant context and help produce a focused patch. The developer reviews the change and uses the established build and test process to resolve failures. Engineering remains accountable for the accepted change and contributes improvements to the reusable solution.[7]

Using the same illustrative rates, 15 calls per accepted change at 50,000 input and 4,000 output tokens per call cost $2,100 across the cohort. Five calls at 12,000 input and 2,000 output tokens cost $220. The potential $1,880 model-spend difference excludes continuous integration (CI) compute, licences, integration and the maintenance of reusable assets. It is a worked example, not a productivity claim.

The CoE maintains marketplace entry requirements and coordinates reviews and retirement. Engineering maintains its accelerators and agents, including supported versions, evidence and support. Assess entries using cost per accepted change, defects, lead time and complete operating cost. Reuse can reduce discovery; excessive handoffs can increase token consumption.

The FinOps lead reconciles input, cached-input, output and any separately billed usage to actual contract rates and invoices. Attribute delegated calls and retries to the originating workflow. Include review, rework, support and allocated library costs once. Business and Engineering owners confirm accepted outcomes; Finance distinguishes capacity released from cash savings. Protect mandatory checks when setting budgets.[3,6]

Establish the mandate and prove the handoffs

Confirm authority and commitments

Select a domain where transformation need is high and sponsorship, ownership and a workable service mandate can be established. Compare reuse potential, data readiness, consequence and receiving-team capacity. Extend an existing centre where it can carry the mandate. Need sets the priority; readiness determines the scope that can responsibly start.

Prove the handoffs in delivery

ML arteka can help the client implement common services and put the CoE operating model to work through a bounded first portfolio. Agree the business problem, sponsor, delivery team, available data and receiving service owner. Use the work to prove intake, specification, implementation, independent assurance, operational acceptance and benefits reporting. Demonstrate reuse with an actual consumer and agreed acceptance evidence. Include an exception and a recovery exercise: a data-authority gap must reach the correct owner, and a service outside its operating limits must be contained and restored through the named authorities. These exercises test both the implementation and the CoE’s ability to support the decisions it assigns.

Expand through demonstrated need

Invite another domain to adopt common services when it has a demonstrated need, a sponsor and the readiness to use and support them. A domain can consume shared services before establishing its own centre. Create another centre when sustained local demand and delivery responsibilities justify it, with a clear boundary between local services and those provided to the federation. Review capacity, funding, access conditions and control requirements before expanding a service. Where ownership, support or controls remain weak, address the gap before widening authority. The expansion decision should record what will be reused, what genuinely needs to be local and who funds and operates each part.

An initial assessment should leave the executive sponsor able to approve the charter, service ownership, funding and evidence required for the next increment. A calendar date or a count of completed pilots cannot replace that decision.