Sovereign AI for UK businesses: when a private LLM beats an API

Why sovereignty is back on the agenda

Data residency has been a procurement checkbox for years. What changed in late October is that two different actors made the same argument from opposite directions, and neither was talking primarily about privacy.

Canada's case was about national capability. Prime Minister Mark Carney framed the Cohere investment in terms that would have sounded overwrought two years ago: "Today, true sovereignty depends on our ability to build, adopt, and govern AI on our own terms." The deal is staged rather than paid up front, and in exchange Cohere committed to creating at least 1,000 Canadian technical jobs by 2030, spending at least C$1.7 billion on computing infrastructure in Canada by 2030, and keeping its Canadian incorporation and headquarters.

Mistral's case was about continuity. Pierre Stock, speaking to TechCrunch about the Large 4 release, put it as "access that can't be cut off by definition because the model is open-weighted". That is a different risk from the one most data protection officers have been managing. It is not about where the data sits. It is about whether the capability you have built a workflow around can be withdrawn, repriced or deprecated by a supplier whose roadmap you do not control.

The sovereignty argument has quietly shifted from "where is our data" to "can this be taken away from us". The second question has a different answer, and a different architecture.

What is a private LLM?

A private LLM is a large language model that runs inside a boundary you control, so that prompts and outputs are not processed on a third party's shared infrastructure. In practice it covers three quite different arrangements: a model deployed on your own servers, a model running in your own cloud tenancy, and a dedicated instance provided by a vendor but isolated from other customers. The governance and cost consequences of those three are not interchangeable, which is why the decision matrix below separates them.

Open weights is not open source

This distinction is routinely blurred in coverage, and Large 4 is a clean example of why it matters. Open weights means the parameters can be downloaded and run. Open source implies a licence granting broad rights to use, modify and redistribute. A model can be the first without being the second.

Mistral released Large 4 under what it calls the Mistral Ecosystem Open-Weight License, which its own text describes as a modified version of the MIT licence. Hugging Face tags it simply as "other". For most organisations the terms are permissive: commercial use is allowed, internal deployment is explicitly exempt from further obligations, and the grant includes the right to sublicense and sell copies provided the copyright notice travels with them.

The restriction sits in one place. If you operate what the licence calls a Model as a Service business, you must enter a separate commercial agreement with Mistral before any commercial use. The licence defines that as giving third parties inference or post-training access in a way that lets them exercise meaningful control over inputs, parameters or training data. It explicitly excludes end-user products where model capabilities are embedded in specific features, and excludes simply relaying requests to models hosted elsewhere.

In plain terms: deploying Large 4 inside your business is permitted. Building a product with it is permitted. Reselling access to the raw model is a conversation with Mistral's legal team. That is a question your commercial director should answer before your engineers download anything.

The real cost of self-hosting a frontier model

The attraction of a self hosted LLM is obvious: no per-token bill, no data leaving the estate, no supplier able to deprecate the model under you. The costs are less obvious because they do not arrive as an invoice with a line item.

Start with the figure that actually governs hardware, because it is not the one in the headline. Large 4 is a granular mixture-of-experts model with 128 routed experts, of which four are active for any given token, plus one shared expert. The result is roughly 52 billion active parameters against a one-trillion total. The repository name says it outright: Mistral-Large-4-1T-A52B. A sparse model of this shape demands memory closer to its active footprint than its headline count, which is why a trillion-parameter model is deployable at all.

Mistral publishes no minimum VRAM or GPU count. What it does publish is working serving configurations, and every one of them runs tensor-parallel across eight GPUs. The main weights are fp8, there is a separate NVFP4 build targeting Blackwell hardware, and a GGUF conversion exists for llama.cpp. Treat eight accelerators as the shape of a production deployment rather than a specification, and size your own from the quantisation you choose.

The rest of the cost is structural, and differs from an API in a way that catches finance teams out:

• Capacity is paid for, not consumption. An API charges for tokens processed. Eight GPUs cost the same whether saturated or idle overnight. Utilisation, not volume, is what decides whether the swap saves money.
• Redundancy multiplies the hardware. A single inference node is a single point of failure. Production availability means duplicating that eight-GPU footprint, which doubles the part of the bill people remember to budget for.
• The engineering is continuous, not one-off. Serving infrastructure, quantisation decisions, throughput tuning, monitoring and a path to the next model version are an ongoing function, not a project with an end date.
• Evaluation becomes your job. With a hosted model, the provider absorbs regression testing between versions. Self-hosted, you own the evaluation harness that proves the model still does what your business needs after any change.

The right way to size this is the same discipline we argued for in our piece on taking AI from proof of concept to production: establish your own baseline first. Your monthly token volume, your peak-to-average ratio and your latency tolerance are the three numbers that will decide this, and no vendor case study can supply them for you.

A low-volume, spiky workload on eight idle GPUs is the most common way to make a private LLM cost more than the API it replaced.

A decision matrix: hosted API, private cloud or private LLM

Three deployment models, assessed against what actually differs between them. The question is not which is best, it is which fits a given workload.

Three rules of thumb fall out of that table. A high-volume, steady, latency-tolerant workload on sensitive data is the clearest case for a private LLM. A low-volume or unpredictable workload on non-sensitive data belongs on an API, whatever the sovereignty mood in the boardroom. And a regulated workload that is genuinely low volume usually belongs in a private cloud deployment, which buys most of the residency argument without the hardware.

Note what the capability row does and does not say. If your workload needs the best available reasoning, the closed frontier is still ahead. If it needs competent reasoning on data that cannot leave your estate, that gap may not matter.

This is a different question from whether to buy software or build it, which we covered separately in the case for and against custom development. Here the software may be identical. What changes is where it runs and who carries the risk.

Hybrid architectures that keep options open

Most organisations that get this right do not choose. They route.

The pattern is to classify workloads by data sensitivity and volume, then send each class to the appropriate deployment, behind a single internal interface. Customer records and anything carrying special category data under UK GDPR go to the private deployment. Public-facing drafting, summarisation of already-public material and experimental work go to the API, where the latest capability lives and the bill scales down when nobody is using it.

Large 4's one-million-token context window makes this split sharper than it used to be, because the workloads that most want a very long context, whole case files, full contract sets, entire codebases, are frequently the ones carrying the most sensitive material.

Six design decisions make that routing possible rather than theoretical:

• A model-agnostic interface, so swapping a backend is a configuration change
• Prompts and evaluation suites held as portable assets, not embedded per vendor
• Classification applied at the data layer, not left to the person writing the prompt
• Logging that records which model handled which request, for audit
• A documented exit path from each provider, costed before signing
• Security boundaries defined per deployment, not once for the whole estate

That last point deserves emphasis. A private LLM removes a confidentiality risk and introduces an operational one, because an internally hosted model with access to internal systems is a new security boundary inside your network. The failure modes we described in our analysis of AI agent security after SalesBleed do not disappear when the model is yours. In some respects they get harder, because no vendor security team is watching it for you.

The integration layer is where this is won or lost. An organisation that has wired one provider's SDK directly into forty services has not chosen an API; it has chosen it permanently.

How Imobisoft approaches private LLM deployment

We start from the workload rather than the model. Volume, sensitivity, latency tolerance and utilisation profile decide the deployment, and in most estates the honest answer is that different workloads deserve different answers. Our work across multiple industries follows this pattern: classify first, route second, and keep the interface between the application and the model loose enough that the decision can be revisited when the market moves again, which on the evidence of the last fortnight it will.

Where that leads to a bespoke deployment, the work sits in AI solution design. Where the API is the right answer, we will say so, which is usually cheaper advice than the alternative.

Frequently asked questions