On-premises sovereign AI: a global view and an India guide
On-premises sovereign AI means running AI models inside infrastructure you control, under your jurisdiction, with your own people able to say where every request and every byte goes. This guide explains what sovereignty covers, the options (on-premises, sovereign cloud, air-gapped), what to put inside the walls, and a separate section on India: the Digital Personal Data Protection Act, 2023, in-country deployment and what changes for an Indian organisation. It is general information, not legal advice.
What "sovereign AI" means
The phrase is used by governments, regulators and vendors, and it does not have one fixed definition. For a buyer, a useful way to read it is as five questions. For each one, can you give a firm answer?
- Data. Where do prompts, documents and outputs live, and who can read them?
- Models. Do you hold the model weights, or do you rent access to a model you cannot inspect?
- Infrastructure. Who owns and controls the GPUs, the network and the building?
- Operations. Who runs the system day to day, from where, and with what access?
- Jurisdiction. Which country's courts and laws apply to the data and to the people who run the service?
A system is sovereign to the extent that you can answer all five with "us" or with a party you chose and can hold to an agreement. The more questions you answer with "a provider abroad", the less sovereign it is.
The options, side by side
| Option | Where it runs | Control | Internet needed | Typical fit |
|---|---|---|---|---|
| Public AI API | The provider's cloud | Provider | Yes | Non-sensitive data and fast starts |
| Dedicated hosted endpoint | Provider's data centre | Shared, by contract | Yes | Work that may leave your site but needs reserved capacity |
| Sovereign or in-country cloud | A provider's data centre inside your country | Provider, within your jurisdiction | Usually | Data that must stay in-country but may be hosted |
| On-premises AI | Your data centre | You | Optional | Regulated and sensitive workloads |
| Air-gapped AI | Your data centre, no outside connection | You | No | Defence, public sector, critical infrastructure |
The categories overlap, and some organisations use more than one. The important step is to decide, per workload, which rung it needs.
Why organisations bring AI on premises
- Regulated or confidential data. Customer records, health data, legal files, source code and designs are exactly what people want to give to a model, and often exactly what rules say must not leave the building.
- Audit and accountability. An auditor can inspect a system you control. It is much harder to inspect one that belongs to someone else.
- Continuity. An on-premises system keeps working when a link to the outside is cut or a provider changes terms.
- Operational technology and critical systems. Where AI sits close to systems that cannot stop, many operators want a human to approve every change and an audit trail they own.
- Cost predictability. A fixed, agreed capacity does not change with the number of tokens an agent uses.
None of these reasons is universal. Plenty of workloads are better served from a hosted endpoint. On-premises is for the ones that cannot go anywhere else.
What goes inside the walls
A complete on-premises AI system has more parts than GPUs. Our stack has four layers:
- Your tools. Any OpenAI- or Anthropic-compatible client, agent, IDE or script.
- Gateway. Keys, sessions, admission and queueing, and routing, on your domain.
- Serving. Open-weight models, with parallelism across GPUs, speculative decoding and prefix cache reuse.
- GPU fleet. Hosted in your data centre.
Around the stack you need an operations layer: observability, change control, security scanning and an audit trail. Without it, a sovereign system is a sovereign system that nobody can explain. We describe this as the Day 2 stage of an AI factory.
Open-weight models and the supply chain
Because the weights are open, they can be loaded inside the site, inspected and served without a call to a third party. That is a practical reason sovereign deployments favour open-weight models. It also makes the software supply chain a first-class concern: you should know what is in the software you run and which cryptographic algorithms it uses. Software and cryptography bills of materials (SBOM and CBOM) are the standard way to record this.
Air-gapped AI
"Air-gapped" means the system has no connection to the internet or to any outside network. It is the strictest form of on-premises.
It changes the design, not just the firewall rules.
- Models, software and updates are brought in through controlled, audited routes and not pulled from the internet.
- Observability must run inside the site. A tool that reports to a cloud service cannot be used.
- Remediation needs human approval inside the site, with an identity for every action and a record that cannot be altered.
We build for this first. Every product has to work with no internet connection before it is allowed to use one. Our operations stack observes from the hardware up, explains faults with the evidence cited, and waits for a named person to approve any change. The open EGAProtocol, which we authored, puts identity, approval and an immutable audit trail into the protocol itself.
Sovereign AI in India
India is a large and fast-growing market for AI, and many of its largest buyers are regulated: banks and financial institutions, insurers, telecom operators, government bodies, and operators of critical infrastructure. This section sets out the points that matter to them. It is general information, not legal advice. Check your own obligations with qualified counsel and with your regulator.
The Digital Personal Data Protection Act, 2023
The Digital Personal Data Protection (DPDP) Act, 2023 is India's law on the processing of digital personal data. In general terms:
- It applies to personal data that is processed in digital form, and it sets duties for the organisations that decide why and how that data is processed.
- It lets the central government restrict the transfer of personal data to countries it specifies, so cross-border transfers are not unconditional.
- It allows for stricter treatment of some organisations, and for other laws and sector rules to apply alongside it.
The Act does not, on its own, say that every AI system must run inside India or on premises. What it does is raise the cost of not knowing where personal data goes. If prompts contain personal data, and a hosted model processes them abroad, you need to be able to explain and justify that. Running inference inside your own data centre in India makes the answer simple.
Sector rules and customer contracts
Beyond the Act, regulators in some sectors set their own requirements on where data is kept and how systems are audited. Customers, particularly public-sector and financial-sector customers, often add contract terms that require in-country processing. Treat these as separate from the Act and check each one.
In-country deployment
For an Indian organisation, the practical choices are:
- Your data centre in India. An AI factory on your premises, on your hardware or ours, serving open-weight models to your people and systems. Nothing leaves your walls.
- A sovereign data centre operator. A data centre operator that wants to offer AI capacity can build, run and keep an AI factory in-country.
MIRASTACK LABS PRIVATE LIMITED is headquartered in Hyderabad, Telangana. Agreements are with an Indian company, invoiced with GST.
A short checklist for Indian buyers
- Does the workload include personal data, and whose?
- Which sector regulator's rules apply, and do they set data-location expectations?
- Do customer contracts require in-country processing?
- Which rung of the ladder above does each workload need?
- Who will operate the system, and where do they sit?
- If the system is air-gapped, how will models and updates be brought in?
Sovereign AI beyond India
The same five questions apply anywhere. Each country has its own data protection law, its own sector regulators and its own view of cross-border transfer. The design stays the same: put the compute inside the jurisdiction, keep the keys and the audit trail with the owner, and run operations in a way the regulator can inspect.
How we deliver it
We design, build and run the system as three stages.
- Day 0, design. Workload sizing, hardware (yours or ours), power, cooling and network, model selection, and the security and compliance requirements the system has to meet.
- Day 1, build and go live. Installation, the serving stack, the gateway and keys on your domain, and acceptance testing.
- Day 2, run it. Operations with observability, explainable root cause analysis, human-approved remediation, release risk checks, and supply-chain and cryptography scans.
For the details, see the AI Factory page, the Day 2 operations page and our short page on on-premises sovereign AI.
Questions to settle first
- Which workloads truly cannot leave the site, and which can?
- Who signs off the data boundary: legal, security or the business?
- Will the hardware be yours or supplied?
- How much of Day 2 do you want to run yourselves, and how much do you want operated for you?
Talk to us
If you are deciding where an AI workload may run, send us three facts: the workload, the number of people or systems it serves, and where it must run. Talk to us or write to hello@mirastacklabs.ai.