You'll Get the AI Org Design Wrong Twice

TL;DR

• The two natural responses to scaling internal AI capability both fail: centralise and one team can't keep up with demand; decentralise and every team relearns the same lessons in isolation.
• Hub-and-spoke is the answer: a small central team owns the platform, connectors, and plumbing; functional teams build on top and feed the roadmap back.
• The feedback loop runs both ways. Functional builders aren't just consuming the platform. They're the primary signal source for what the central team builds next.

---

The first instinct is sensible. Form a dedicated AI team. Give them the mandate. Let them build tools for everyone else.

This works until it doesn't. Usually within the first three months.

The queue fills faster than the team can clear it. Sales wants a proposal generator. Finance wants a contract reviewer. Customer success wants call summarisation. Legal wants an intake tool. All of them are urgent to whoever sent the request. The team is four people. They start triaging. They start saying no. Teams that were excited stop believing the platform will get to their problem. Adoption stalls.

The reaction is decentralisation. Let each team build their own things. Free people to solve their own problems.

This also works until it doesn't.

Six months in, sales has built a call analysis tool. So has enablement. So has customer success. Three teams have independently solved the same problem without knowing the others exist. They've also independently hit the same MCP configuration issue, the same rate limit, the same data access gap, solving it three separate times over. The learnings don't compound. The platform gaps are papered over with team-specific workarounds that break on the next model upgrade.

Most organisations discover the right structure only after failing both ways.

How the Hub-and-Spoke AI Team Model Works

The central team owns the platform. That means model routing, data connectors, the authentication layer, governance policy, evals infrastructure, and the skill-sharing mechanism. The expensive things to build once that become more expensive to rebuild twelve times across twelve teams.

It also means enablement: the tooling that lets non-engineers build and share workflows without needing to understand the underlying infrastructure.

Functional teams build on top. A risk analyst who wants to automate financial modelling. A sales ops lead who wants to replace a spreadsheet-based comp process. A finance manager who needs a contract reviewer. They use the platform's connectors and tools, build workflows specific to their function, and pull engineering in when the work is ready for production (if it ever needs to reach production at all).

That last point matters. Many of the most valuable internal AI tools never touch the production codebase. A workflow that saves one person sixteen hours a month, running on the internal platform, is worth building without a single engineering ticket.

The AI-native team design principles cover the hiring and autonomy norms that make functional builders effective once the platform exists.

Why the AI Platform Feedback Loop Runs Both Ways

Describing this as "central platform serves functional teams" misses something. Functional builders aren't just consuming the platform. They're stress-testing it.

When the risk analyst's workflow breaks because the data connector has a gap, that's a platform priority. When the legal team's tool produces unreliable output on specific clause types, that's an evals problem the central team needs to address. When someone can build a training simulator in fifteen minutes, that tells you the skill authoring workflow is working. When someone can't build what they need without three days of setup, that's friction the platform should eliminate.

The spokes are the best signal source for the hub's roadmap. Organisations that treat the relationship as one-directional (central team pushes capabilities, functional teams consume them) end up building the wrong things and discovering it slowly.

I've seen the one-directional version play out at scale. At Cotality, centralised product capability meant the teams closest to customer problems had the least direct route to building solutions. The queue existed. The priorities were always contested. The people with the most context on the problem had the least authority to act on it. The hub-and-spoke model doesn't eliminate that tension. It gives functional people a legitimate path to building, and keeps that signal flowing back to whoever maintains the platform.

What Hub-and-Spoke Produces: Real Outcomes from Non-Engineers

When Ramp landed on hub-and-spoke after trying both alternatives, the outcomes were specific:

• A risk analyst automated 16 hours per month of manual financial modelling
• A sales ops lead replaced a spreadsheet comp model across three organisations in 48 hours
• Someone in finance built a contract reviewer saving 45 minutes per contract
• An L&D lead built a training simulator in 15 minutes

None of them are engineers. None of them filed a ticket to get started.

The platform made it possible. The functional context made it useful. Neither alone produces the same result.

What the Central Team Owns vs. What Functional Teams Own

Central team owns: Model routing, data connectors, authentication, governance policy, evals infrastructure, the skill publishing layer. The expensive things to build once rather than rebuild twelve times.

Functional teams own: Domain-specific workflows, the skills they publish, feedback to the central team's roadmap, and the judgment about when a prototype is ready for broader use.

Neither team owns: The adoption decision. Hub-and-spoke creates the conditions. Whether people actually build is a management problem, not a platform problem.

The constraint-based adoption approach works alongside this structure: functional teams that are deliberately under-resourced with headcount and over-resourced with tokens are the ones that build the most on top of the platform. Abundance of people removes the forcing function. Constraint forces the reach for AI solutions.

The org design question for internal AI isn't "centralise or decentralise". It's "what does the platform own, and what do the builders own?" Get that split right, keep the feedback loop running both ways, and capability starts to compound across the organisation.

Getting the org model right is the first step. The second is making the platform frictionless enough that non-engineers actually use it: that's a product design problem, not a training problem.

Most orgs land on this answer on the third try. Worth knowing before you start, so the first two failures are shorter.