Now we know what an agent is. The next question is where it physically lives, and why "local vs. cloud" hides two questions in one word. Market data, a three-layer anatomy, and the definitional detour you actually need.
- The split nobody benchmarks
- The agents we're actually talking about
- The three-layer anatomy of any agent
- "But what counts as 'local'?"
1. The split nobody benchmarks
Most of the AI-agent conversation in 2026 is about capability, which model, which framework, which orchestration layer. The conversation that actually decides cost, security, latency, and legal risk is the one fewer people are having: where do they run?
The market has settled into a rough 62 / 28 / 10 split, cloud, on-prem, edge. Hybrid and edge deployments are the fastest-growing slice at a 37.8% CAGR through 2030 (MarketsandMarkets). That number isn't a rounding error, it's the part of the market actively choosing not to answer the question with "just cloud".
2. The agents we're actually talking about
This isn't only about coding copilots. The agents driving real budget in 2026 are outward-facing, they touch the public internet, third-party APIs, and customer-facing workflows: outbound SDRs, research and scraping agents, support and inbox agents, RPA and internal automation, and the browser agents that log in and click through real sites.
3. Quick primer, the three-layer anatomy of any agent
Before the tradeoffs make sense, the three-layer stack underneath every agent. Every agent has three layers, software, network, and infrastructure. Each layer can live in your building, or in someone else's. The combination decides everything that comes next.
Two things worth saying plainly:
- "Infra" doesn't mean a data center. Your laptop is infra. A Raspberry Pi under your desk is infra. The Mac mini in the closet running 24/7 is infra. Infra is just the physical thing running the code, it can be your computer, or a rented server on the other side of the world.
- The three layers don't have to match. You can run an open-source model (local software) on a cloud GPU (cloud infra) routed through a residential proxy (mixed network). The mix is the point, and the source of almost every tradeoff in the rest of this series.
4. "But what counts as 'local'?"
A colleague pushed back on the framing with a good question: "If I use the OpenAI service, but the workload runs on my infra, and my infra is virtual, on the cloud, is that still 'cloud'?"
This trips up almost everyone. "Local vs. cloud" is one word for two things people usually mean at the same time:
- Physical location. Is the metal in your building, or in someone else's data center?
- Trust boundary. Is the workload inside your tenant / your VPC / your account, or sitting in a vendor's multi-tenant pool?
These two axes look the same at the extremes (a Mac mini under your desk is your-building and your-tenant; OpenAI's chat API is their-building and their-tenant). They split apart everywhere in between, and modern agent deployments live almost entirely in the middle.
So how does the OpenAI-sandbox case score? OpenAI's model API runs in their data center, multi-tenant. The tool-execution sandbox runs in your AWS account, single-tenant. The whole thing sits on a foundation you don't own but are a named tenant of.
The honest answer: it's a hybrid, but it inherits cloud properties on five of the six axes that matter. The one place it acts "local" is data sovereignty, and only for the data that never touches the model provider.
The cleaner mental model: don't ask "is it cloud or local?" Ask "which layer am I asking about, and whose trust boundary is it inside?"
From here on, "local" = your hardware, your network, your tenant. Anywhere that breaks down, the layer that matters will be flagged.
