Security & performance, the two lenses everyone benchmarks

§1 The security lens

Cloud-hosted agents (Anthropic Managed Agents, Replit Agent, Vercel AI Cloud) hand the infra-hardening problem to the vendor. Good. They also hand you a new one: machine-identity sprawl. The Cloud Security Alliance reports machine-to-human identity ratios already hit 100:1 in enterprise environments.

Local agents flip the burden. Data sovereignty is solved by definition, nothing leaves your box. But Prelude Security's 2026 risk inventory lists what most teams underestimate: unsafe model loading (torch.load), prompt injection from local files the agent has been told to trust, and no policy layer between tool call and execution.

88% of organizations reported confirmed or suspected AI-agent security incidents in the past 12 months. Location doesn't fix governance, it changes which gaps you inherit.

Lowering the security risk

Concrete moves that change the math, not the marketing.

If you go cloud:

• Single-tenant or VPC-deployed agent runtime, no shared compute with other customers
• Short-lived credentials per agent (OIDC, IAM Identity Center) instead of long-lived API keys
• Network segmentation, agent VPC isolated from prod data, egress through a single inspectable proxy
• Policy layer in front of tool calls (Cisco AI Defense, Zenity, Lakera), score each action before it runs
• Audit log streaming to your SIEM, not the vendor's dashboard

If you go local:

• Run the agent in a container or VM, not bare-metal, limit the blast radius of prompt injection
• Read-only filesystem mounts for anything the model doesn't strictly need to write
• Never load .pt / .bin weights from untrusted sources (use safetensors)
• Outbound allowlist on the firewall, agent can hit Stripe but not your finance DB
• Endpoint security agent on every host running local agents (treat them like employee laptops, because that's what they are)

Security is the lens executives ask about first. Performance is the one that decides whether the agent feels usable on day two.

§2 The performance lens

Cloud wins where raw model quality matters, frontier models stay cloud-only for the foreseeable future. Elastic compute, no thermal throttle, no local-hardware ceiling. Local wins on three vectors cloud can't beat: latency, rate-limit headroom, and unit cost at scale.

Round-trip latency for a single tool call is where the gap shows up first. Three places this matters most:

• Latency. A 100×+ gap between local and cloud round-trips compounds across every tool call. An agent that takes 5 hops to complete a task feels instant locally and sluggish in the cloud.
• Rate-limit headroom. Your local box doesn't queue behind 50 million other API calls at peak. Anthropic's rate limits have silently degraded production agents more than once.
• Cost at scale. A local model amortizes to near-zero per call after the hardware is paid off. Cloud-hosted agents meter every session.

Squeezing more speed from each option

Where the latency or throughput ceiling actually sits, and how to lift it.

If you go cloud:

• Pick the region closest to your tool endpoints, not closest to you
• Provisioned-throughput tier (Anthropic, OpenAI) for predictable latency under load
• Streaming + speculative decoding to mask round-trip lag
• Cache repeated context (prompt caching), first call slow, every subsequent call cheap
• Circuit breakers + retry-with-jitter so one rate-limited call doesn't stall the whole agent

If you go local:

• Quantize aggressively (Q4/Q5), 70B model on a single 24GB GPU is now realistic
• vLLM or TGI as the serving layer, not raw transformers (5–10× throughput)
• Hybrid routing: small/fast local model for cheap steps, cloud frontier model for hard ones
• Local KV-cache reuse, agent loops with shared context get near-instant follow-ups
• Watchdog + auto-restart for the model server; agents die silently when the GPU OOMs

Fast is good. Easy is what keeps it in production a year from now.