Sovereign AI Comes of Age: When to Run On-Premises AI on Your Own Infrastructure

On-premises AI infrastructure illustrated through a concrete data vault containing servers and documents, with a hand inserting a blue data module to represent secure, locally controlled AI systems.

On-premises AI used to mean a trade: keep your data in-house, and accept a weaker model. The announcements in the first week of October 2026 narrowed that trade considerably.

Mistral previewed Large 4 on 6 October, with downloadable weights due by the end of October and a European deployment it runs end to end. The same week, PwC and Cohere announced an alliance offering private cloud, on-premises and air-gapped deployment, and Dell expanded an AI data platform that keeps data in customer data centres.

Taken together, the point is simple. Capable open-weight models and private deployment options now make keeping data in-house a realistic choice. For regulated sectors such as healthcare and pharma, the question has shifted from "can we?" to "when is it worth it?". Answering it means weighing what the headlines leave out: what self-hosting really costs, what you give up compared with a hosted API, and what your regulators actually require.

This article is for leaders in regulated or data-sensitive sectors: healthcare, pharma, finance, legal and the public sector. It covers what changed in the week of 5 October and why open weights matter, the trade-offs between a hosted API, private cloud and on-premises deployment, the hidden costs of self-hosting a very large model, and a decision framework based on data sensitivity, volume and regulatory exposure.

What changed in the week of 5 October, and why open weights matter for control and auditability

Three announcements landed within two days of each other, on 5 and 6 October 2026. Each makes a different part of the in-house option more practical.

Mistral Large 4: a trillion-parameter model you will be able to download

Mistral opened a public preview of Mistral Large 4 on 6 October. Mistral describes it as a natively multimodal model with 1 trillion parameters, of which about 52 billion are active on any given task. BetaNews notes that Mistral's documentation lists 1.05 trillion, and that earlier reports gave 49 billion active. The preview runs through an API on Mistral Studio, and Mistral says it will release the weights by the end of the month.

Two deployment details matter here. Mistral says the model can be deployed from Europe on infrastructure it operates end to end, "independently of other digital service providers and under European law", and that it will be able to run on private cloud or on-premises. Mistral describes Large 4 as state of the art among open models on cybersecurity, finance and law. As BetaNews points out, every benchmark figure released so far comes from Mistral and remains preliminary.

PwC and Cohere: private cloud, on-premises and air-gapped deployment

On 5 October, PwC and Cohere announced a global alliance, launching first in Canada, that pairs Cohere's North agentic AI platform with PwC's risk and regulatory work. It supports private-cloud, on-premises and air-gapped deployments for organisations that need more control over sensitive data. Aidan Gomez, Cohere's co-founder and chief executive, put the case plainly: "Enterprises need AI that can deliver real capability without forcing them to give up control of their data."

Dell: keeping the data layer in your own data centre

On 6 October, Dell expanded its AI Data Platform with a Unified Semantic Layer, an Enterprise Knowledge Graph and Knowledge Agents. Solutions Review's weekly round-up describes the aim as keeping sensitive context inside customer data centres, and IT Brief UK reports that these elements will remain within the customer's own data centre. They are due in the first half of 2027, so this is a direction of travel rather than something you can deploy today.

Why open weights matter for control and auditability

An open-weight model is one whose trained parameters are published for anyone to download and run. That is not the same as open source. The licence decides what you may do with the weights, and Mistral has not yet said which licence Large 4 will carry, so check it before you plan around the model.

For a regulated organisation, downloadable weights change three things:

  • Version controlA hosted model can be updated or retired by its provider. A downloaded checkpoint stays exactly as it was until you choose to change it, so the model you validated is the model you run.
  • Logs stay with youEvery prompt, output and system event can be recorded in your own environment, under your own retention rules. That is the evidence an auditor or regulator will ask for.
  • Testing before exposureYou can evaluate the model on your own data, inside your own perimeter, before any real record reaches it.

Hosted API, private cloud or on-premises: the trade-offs in cost, capability and compliance

The three options are not a ladder from worse to better. Each buys a different mix of cost, capability and control.

  • Hosted API: you send prompts to the provider's servers and pay per token.
  • Private cloud: the model runs on capacity dedicated to you, in a cloud region you choose.
  • On-premises: the model runs on hardware you own, in your own data centre, and can be air-gapped from outside networks.
Trade-offs between a hosted API, private cloud and on-premises AI
Factor Hosted API Private cloud On-premises
Cost Pay per token, with no upfront investment Pay for dedicated capacity whether or not it is busy Buy, house, power and staff the hardware yourself; cost is largely fixed
Capability Access to the most capable closed models, which cannot be downloaded, and to provider updates as they ship Open-weight models and vendor models licensed for private deployment Open-weight models and vendor models licensed for on-premises or air-gapped use; you decide when to upgrade
Compliance Data leaves your environment; you rely on the provider's contract, data processing terms and any transfer safeguards Data stays in your tenancy and chosen region, but the cloud provider remains in your supply chain Data never leaves your perimeter; you carry the full security and evidence burden yourself

Cost

The hosted price is the number any self-hosted plan has to beat. Mistral's standard preview price for Large 4 is $1.36 per million input tokens and $4.18 per million output tokens, and BetaNews reports a launch discount that currently halves both. A hosted API turns cost into a variable that tracks usage. Self-hosting turns it into a fixed commitment that only pays back if the hardware stays busy.

Open weights shift the economics even for teams that never self-host. As we argued in our analysis of LLM pricing, hosted prices move through repricing and expiring introductory rates, and a credible self-hosted alternative gives buyers leverage.

Capability

The capability gap between open and closed models is narrowing, but the evidence is still mostly the vendors' own. Mistral says Large 4 beat OpenAI's GPT-6 Astra on each legal and financial task in a third-party evaluation by Vals AI, and those results are preliminary. There is also a structural difference: a hosted model improves as its provider ships updates, while a self-hosted checkpoint improves only when you upgrade it.

Compliance

Keeping data in-house removes one category of risk: sending data to a third party. It does not remove your obligations. A self-hosted model still needs a lawful basis for processing, access controls, logging and security testing, and you can no longer point to a provider's certifications for any of it.

The hidden costs of self-hosting a very large model: hardware, skills and maintenance

The weights are free to download. Running them is not.

Hardware

Mistral has not published hardware requirements for running Large 4, and the precision of the released weights is not yet known. The parameter count still sets the floor. At 8-bit precision, each parameter takes one byte, so a trillion-parameter model needs roughly 1 TB of GPU memory for the weights alone. At 16-bit precision it needs roughly 2 TB. That is before the memory needed to hold the context of the requests it is serving.

For scale, an NVIDIA DGX B200, an eight-GPU server built for this kind of work, has 1,440 GB of GPU memory in total, draws up to about 14.3 kW and occupies 10 rack units. One such system could hold the weights at 8-bit precision; at 16-bit you would need two. A production service also needs capacity for failover, which means more than one system, each with power, cooling and rack space on that scale.

Skills

Self-hosting moves work from the provider to your team. Someone has to run the GPU servers, operate the serving software, choose and test quantisation settings, measure output quality, patch the stack and respond to incidents. With a hosted API, the provider does all of this. In-house, those are permanent roles or a contracted service, not a one-off project. As we noted in our guide to AI model alternatives for business, self-hosting brings responsibilities for access management, monitoring, maintenance and security updates.

Maintenance

A self-hosted model does not stay current on its own. Mistral says reinforcement learning on Large 4 is still running, and BetaNews reports that it expects further gains over the coming weeks and months. Hosted users get those gains automatically. Self-hosted users get them only by downloading a new checkpoint, re-running their evaluations and redeploying.

That is the price of version control. Every upgrade becomes a change you have to test, document and approve, which is exactly what an auditor wants to see and exactly what costs time.

A decision framework based on data sensitivity, volume and regulatory exposure

No single factor should decide the deployment model.

The trap

Deciding that sensitive data means on-premises, then finding the workload is too small or too variable to keep expensive hardware busy.

The fix

Score each workload on all three factors, and choose the least expensive option that meets the most demanding of them.

1Data sensitivity

Ask what happens if the prompts and outputs are seen by a third party. Routine internal text is low risk. Special category health data, legally privileged material, unpublished clinical trial results and market-sensitive information are high risk. High sensitivity points away from a public API and towards private cloud or on-premises.

2Volume

Ask how much the workload will use the model, and how steadily. Low or unpredictable volume favours per-token pricing, because you pay only for what you use. High, sustained volume can justify fixed-cost infrastructure, because idle hardware is the expensive part of self-hosting.

3Regulatory exposure

Ask what your regulators and contracts actually require, rather than assuming. In the UK, the rules are less restrictive than many teams expect:

  • UK GDPR does not require personal data to be stored in the UK. It controls transfers abroad through adequacy regulations and safeguards such as the International Data Transfer Agreement, as the ICO's guidance on international transfers sets out.
  • NHS England's guidance says health and care organisations should use cloud computing services for hosting NHS data, with location rules and senior sign-off for hosting outside the UK.
  • The PRA's outsourcing statement, SS2/21, asks banks and insurers to take a risk-based approach to data location, and to secure access and audit rights over material outsourced services.
  • For SECRET information, the Government Security Classifications Policy calls for secure networks on "secured dedicated physical infrastructure". That is where dedicated or air-gapped deployment stops being a choice.

In our view, the binding constraint in legal and pharma work is often contractual rather than statutory: client confidentiality terms or a sponsor's data agreement can rule out a third-party API even where the law would allow it. Public-sector suppliers should also expect, as we noted in our piece on UK AI regulation, sharper questions in tenders about where models are hosted and where data sits.

Putting the three factors together

The table below is our framework, not regulatory guidance. Use it to shortlist an option, then cost it properly.

How data sensitivity, volume and regulatory exposure point to a deployment model
Workload profile Likely fit Why
Low sensitivity, any volume Hosted API Lowest cost to start and access to the most capable models; nothing in the data justifies the overhead
High sensitivity, low or variable volume Private cloud Data stays in your tenancy and chosen region without paying for hardware that sits idle
High sensitivity, high and steady volume Private cloud or on-premises Sustained use can keep dedicated hardware busy enough to justify it; compare total cost against private cloud capacity
A requirement for no third-party access, such as SECRET information or an air-gap obligation On-premises, air-gapped The requirement decides it; cost becomes a budgeting question rather than a deciding one

Most organisations will end up with more than one answer: a hosted API for low-risk work, and private or on-premises deployment for the workloads where sensitivity and regulatory exposure justify the cost.

Where those factors do line up, the case for keeping AI on your premises is now stronger than it has been. A model Mistral calls its most capable will be downloadable, vendors such as Cohere offer air-gapped deployment, and running the model yourself gives you control over versions, logs and data that a hosted API cannot give you in full.

Sovereign AI has come of age. Is it worth it for your workloads?

Before committing to hardware, score each workload on data sensitivity, volume and regulatory exposure, and price the hosted option it has to beat. If on-premises AI still comes out ahead, plan for the hardware, the skills and the upgrades from the start.

At Imobisoft, we design and build AI solutions around the systems, data and compliance obligations our clients already have, and we have completed three consecutive years of ISO 27001 accreditation.

Talk to us about private AI deployment

Frequently asked questions

What is on-premises AI?

On-premises AI means running AI models on hardware an organisation owns and operates in its own data centre, rather than sending data to a provider's cloud. It can also be air-gapped, with no connection to outside networks.

What is sovereign AI?

Sovereign AI means control over where AI runs, who can access it and which laws apply. For governments it is about national capability. For an organisation, it means keeping models and data under its own control, in a jurisdiction it chooses.

What are open-weight models?

Open-weight models are AI models whose trained parameters are published for anyone to download and run. The licence still governs how they may be used, so open weights are not always open source.

When is on-premises AI worth it?

When high data sensitivity, high and steady volume, and regulatory or contractual exposure all point the same way. If volume is low or variable, private cloud usually meets the same data requirements at lower cost.

Does UK GDPR require AI data to stay in the UK?

No. UK GDPR does not require personal data to be stored in the UK. It controls transfers abroad through adequacy regulations and safeguards such as the International Data Transfer Agreement.

When will Mistral Large 4's weights be released?

Mistral says it will release the weights by the end of October 2026. It previewed the model on 6 October and has not yet named the licence the weights will carry.

Sources

Have a project in mind? Let’s get to work.

Let’s chat about how we can help you. Fill in the details and we’ll get back to you as soon we can.