← All posts

When local isn't a choice, it's the only option

On-premise server secured by a shield

For RPA, internal automation, and any regulated workload, cloud isn't a tradeoff, it's structurally impossible. The on-prem ERP, the legacy Java app, the patient data, the classified system, none of them have a public endpoint, and they shouldn't.

In this part
  • RPA and internal automation can't reach into the cloud
  • Regulation often forecloses cloud entirely
  • Making mandatory-local actually work

The previous parts framed local mostly as a reputation play for outbound traffic. That's only one slice of the agent world. A big chunk of real production deployments are internal, and for those, cloud isn't a tradeoff, it's structurally impossible.

RPA and internal automation can't reach into the cloud

The agent that reconciles invoices in your on-prem ERP, moves files between two desktops in the accounting department, fills in fields in a 1998-vintage Java application that has no API, or watches a network share for new PDFs and drops them into SharePoint, that agent has to physically run inside the network where those systems live. The data sources don't have public endpoints. Even if they did, opening them up would defeat the security model that put them on-prem in the first place.

This is the entire reason UiPath, Automation Anywhere, and Blue Prism shipped on-prem orchestrators a decade ago. AI agents now have the same constraint, only with stronger reasoning attached. A cloud agent can't open the legacy Citrix session, can't read the local Excel file with the macro, can't drive the desktop app that the finance team uses every morning. A local agent can.

Why internal and regulated workloads must run local

Regulation often forecloses cloud entirely

For a meaningful slice of the economy, "deploy it in AWS" is not a legal option:

  • Healthcare (HIPAA, EU MDR, NHS DSP). Patient data can leave the hospital only under tightly-defined BAAs. Many systems forbid US-hosted inference entirely; EU sovereignty rules push toward on-prem or sovereign-cloud only.
  • Financial services (PCI-DSS, MiFID II, FFIEC, DORA). Trading systems, KYC pipelines, and core banking can't expose data to a third-party LLM provider without explicit auditor sign-off, which most cloud agent platforms don't yet have.
  • Government & defense (FedRAMP High, IL5, NATO classifications). Most public agent platforms don't hold the clearances. Anything touching classified or controlled-unclassified information runs in air-gapped environments by default.
  • EU AI Act (in force Aug 2026). "High-risk" AI systems carry recordkeeping, human-oversight, and data-governance requirements that cloud-only deployments struggle to satisfy, especially when the model provider is non-EU.
  • Sectoral data-residency laws. Germany, France, China, India, Saudi Arabia, Brazil all have rules that effectively pin certain workloads inside national borders. Generic US-hosted Claude or GPT isn't a path forward there.
For internal automation and regulated workloads, "local" isn't a reputation strategy. It's the only deployment that ships.

Making mandatory-local actually work

When cloud isn't on the table, the discipline shifts to network design and audit. The hard part isn't running the model, small open-weight models on a workstation are well-understood in 2026. The hard part is governance: who can call which tool, what gets logged, who reviews destructive actions, how the regulator sees it.

Two patterns that actually work:

RPA / internal automation

  • Run agents on a dedicated VM inside the same network segment as the target systems
  • Use service accounts with least-privilege, not human credentials
  • Pair a small local model for sensitive steps + cloud frontier model only for reasoning that genuinely needs it (with PII stripped before the call)
  • Treat the agent like an RPA bot, record what it did, gate destructive actions behind human approval
  • Network-level isolation: the agent VM has only the routes it needs, nothing more

Regulated industries

  • Sovereign cloud or air-gapped on-prem, confirm provider holds the right certifications (BAA, FedRAMP, IL5) before designing
  • Self-hosted open-source models (Llama 3, Mistral, Qwen) on hardware you own, no inference call leaves the perimeter
  • Document data flows for the regulator before you ship, not after audit
  • For EU AI Act "high-risk" systems: human-in-the-loop checkpoints, full audit log, written model card
  • Quarterly model-card refresh with bias/fairness eval, regulators are starting to ask

Frequently asked questions

When is local AI deployment mandatory?

Local deployment becomes necessary when AI agents need access to internal systems, legacy applications, on-prem infrastructure, or data that cannot leave a controlled environment.

Why can't cloud AI agents handle some automation tasks?

Many business-critical systems, such as legacy ERP platforms, desktop applications, file shares, and internal networks, are not accessible from the public internet. Cloud agents simply cannot reach them.

Which industries are most likely to require local AI agents?

Healthcare, financial services, government, defense, and organizations operating under strict data residency or compliance requirements are common examples.

Can organizations combine local and cloud AI models?

Yes. Many deployments use local models for sensitive data and internal workflows while selectively using cloud models for advanced reasoning or non-sensitive tasks.

What is the biggest challenge of running AI agents locally?

The challenge is usually governance rather than technology. Organizations need clear controls around permissions, audit logging, security, compliance, and human oversight.

About the author

Accountable access for agents.

Metro Fabric gives every agent a verifiable identity and a signed access record, wherever it runs.

Join the waitlist