AI Self-Regulation: Should Big AI Be Allowed to Police Itself?

AI self-regulation illustrated by a faceless AI figure holding a gavel above a gated system of AI models, questioning whether technology companies should oversee themselves.

AI self-regulation should not mean allowing leading AI companies to police the entire field. Leading AI companies should share safety information and test one another's models.

But they should not be allowed to define the rules, judge compliance and determine which AI systems reach the public. That would not be neutral regulation. It would turn existing technological power into private governing authority.

Elon Musk has proposed that leading AI companies review one another's most advanced models before release. He argues that rival developers possess the technical knowledge and commercial incentive to identify risks that conventional regulators may miss.

Musk set out the idea in an interview with The Economist, published on 23 July 2026. He suggested the leading labs hold a call "once every few weeks" to discuss safety and security, and give competitors a week or two of early access to each major new model. If a company failed to address a serious concern, he said, "that would be the moment for government to step in and take action", Reuters reported. The interview appeared in the same week that OpenAI disclosed two of its models had escaped a sandboxed test environment, an incident we examined in our piece on the July 2026 sandbox escape.

Musk is not proposing that AI companies formally govern the entire field. The danger arises if peer review becomes the primary mechanism through which they define safety and control market access.

The proposal sounds practical. The companies building frontier AI understand their systems better than almost anyone else.

That is also the problem.

If the same companies build the models, identify the risks, write the standards and decide whether competitors have passed, they become more than developers. They become inspectors, judges and gatekeepers.

The central danger is that a small group of dominant laboratories could end up governing AI on behalf of everyone else.

Part of that future is already the US government's approach. On 29 September 2026, six companies (Google, Anthropic, Meta, OpenAI, Nvidia and Musk's own xAI) signed the White House Accord on Super Intelligence. It asks each company to run internal controls, use an independent external auditor and give a board committee oversight, and its signatories agreed to meet regularly to set standards and best practices. The accord "creates no new regulatory requirements", Infosecurity Magazine reported. President Trump called it "morally binding" and said self-regulation was "very important".

It names no penalties for a company that falls short, and it does not say who will choose or pay the auditors, as the Atlantic Council has pointed out.

What does AI self-regulation look like today?

AI self-regulation means AI companies setting, applying and checking their own safety rules through voluntary commitments, shared standards and peer review, rather than following rules enforced by an independent regulator. It is not a future proposal. Frontier AI developers have been making voluntary AI commitments since 2023, and AI peer review between rival labs has already been tried.

Voluntary AI self-regulation measures from 2023 to 2026
When What happened Penalties built in?
July 2023 Seven companies, including Amazon, Anthropic, Google, Meta, Microsoft and OpenAI, agreed to voluntary safeguards announced by the White House. No
May 2024 At the AI Seoul Summit, 16 companies, including xAI, signed the Frontier AI Safety Commitments, promising to publish safety frameworks and set thresholds for intolerable risk. The list has since grown to 20. No
August 2025 OpenAI and Anthropic each ran their own safety evaluations on the other's publicly released models. No
July 2026 Musk proposed that rival labs review each other's frontier models before release, with government as a last resort. Proposal only
September 2026 Six companies signed the White House Accord on Super Intelligence, covering internal controls, external auditors and board oversight. No

Each step has widened what companies say they will do. None of them carries a penalty for a company that falls short.

Why does AI self-regulation sound so convincing?

Every technically complex industry can make a persuasive case for self-regulation.

Its companies employ the specialists. They possess confidential information. They understand how their products behave. External institutions often move more slowly and lack comparable expertise.

Musk's proposal draws strength from this knowledge gap. A rival AI laboratory may recognise a dangerous model capability faster than an outsider. Regular safety discussions could also prevent companies from discovering the same vulnerabilities separately.

But expertise answers only one question: who understands the technology?

It does not answer the more important questions: whose interests will shape the standards, what information will be disclosed and who will be excluded by the rules?

A company can understand a danger perfectly while still having strong reasons to minimise it, redefine it or delay acknowledging it. Its survival may depend on releasing a model before competitors, satisfying investors and protecting its market position.

Technical knowledge does not remove that conflict. It makes the conflict harder for outsiders to detect.

Amba Kak, co-executive director of the AI Now Institute, has stated the problem directly:

"A closed-door deliberation with corporate actors resulting in voluntary safeguards isn't enough."

Her criticism matters because voluntary commitments tend to address problems companies can accept without changing the foundations of their business models. AI self-regulation may therefore recognise certain technical dangers while leaving questions of competition, discrimination, privacy and corporate power outside the room.

What does history tell us about industries policing themselves?

History does not show that every private standard fails. It shows that voluntary oversight becomes unreliable when public welfare conflicts with the industry's commercial interests.

The early internet provides a direct technology example. For years, companies argued that voluntary privacy policies could protect users without slowing digital innovation. Yet a 1998 Federal Trade Commission review found that only 14% of surveyed commercial websites disclosed their information-collection practices, while roughly 2% offered a comprehensive privacy policy. The FTC concluded that voluntary industry efforts had "fallen short of what is needed to protect consumers."

Credit-rating agencies reveal a second danger: private reviewers can become market gatekeepers. Before the 2008 financial crisis, issuers paid agencies to rate the securities they wanted to sell. Former SEC Commissioner Caroline Crenshaw has called this a "fundamental conflict of interest." Agencies had incentives to satisfy paying clients, while investors and the wider economy relied on their judgments.

The lesson is not that AI will reproduce either case exactly. It is that self-regulation changes the position of private companies. They cease to be only market participants and begin deciding what counts as trustworthy, compliant or safe.

Once their approval becomes necessary, commercial judgment begins to function like law.

Could AI safety standards become tools of market control?

The greatest risk is not simply that AI companies might overlook an unsafe model. It is that they could use safety standards to control the direction of the entire industry.

A group of frontier laboratories could decide which evaluations are mandatory, how much testing is sufficient and what level of computing security every developer must maintain. Those requirements may appear neutral while favouring companies that already possess enormous budgets, specialised staff and access to advanced infrastructure.

A large laboratory can afford extensive evaluations, legal teams and secure data centres. A university, start-up or open-source research group may not.

When dominant developers define what "safe" means, they can create standards that smaller companies and independent researchers cannot afford to meet. Coordination then begins to resemble market control.

Economists have a name for rules that end up serving the industry they are meant to restrain: regulatory capture. Self-regulation skips a step, because the industry writes the rules in the first place.

The credit-rating precedent matters here. Rating agencies did not manufacture the financial products they assessed, yet their approval became a commercial passport. An industry-controlled AI safety process could become even more conflicted because the laboratories issuing judgments would also compete against the companies being judged.

A rival could identify a genuine safety problem. It could also benefit commercially when another developer's release is delayed, restricted or discredited.

Both things can be true at once.

Are we sleepwalking into a technological oligarchy?

Frontier AI is not developing in an open field where every company possesses equal power.

The largest laboratories are closely tied to major cloud providers. In January 2025, an FTC study of partnerships involving Microsoft and OpenAI, Amazon and Anthropic, and Google and Anthropic identified more than $20 billion in cumulative investment. It found that these relationships could affect access to computing resources and engineering talent, increase switching costs and give major cloud companies privileged access to sensitive technical information.

Britain's competition regulator had reached a similar view. In April 2024, the Competition and Markets Authority described an "interconnected web" of more than 90 partnerships and strategic investments involving Google, Apple, Microsoft, Meta, Amazon and Nvidia. Its chief executive, Sarah Cardell, said the CMA had "real concerns" about how the market was developing.

These firms already influence several layers of the AI economy: cloud infrastructure, computing capacity, models, distribution and consumer applications.

Giving leading companies authority over safety standards would add another layer: the power to define legitimate AI development.

Former FTC Chair Lina Khan framed the central choice directly:

"[W]ill a handful of dominant firms concentrate control over these key tools, locking us into a future of their choosing?"

An AI oligarchy would not require a secret agreement between technology executives. It could emerge through an apparently responsible process.

A few companies would meet regularly. They would develop shared standards. Their evaluations would become trusted by investors, businesses and public institutions. Over time, their approval would determine which products entered the market and which developers were considered irresponsible.

The first two of those steps are already written into the White House accord.

The system might still contain competition. But it would be competition managed by the most powerful competitors.

Would consumers receive the best AI or only what AI gatekeepers permit?

AI self-regulation is often presented as a way to give consumers safe products without blocking innovation.

But safety is not the only value at stake. Consumers also benefit from competition, affordability, openness, privacy, choice and the ability to move between providers.

A private AI-governance system could weaken those benefits.

Dominant laboratories may prefer closed models because closed systems protect intellectual property and recurring revenue. They may oppose forms of open development that allow users and smaller companies to run, modify or study models independently. They may design safety conditions that keep customers inside proprietary platforms.

The result could still look innovative. Consumers might receive faster assistants, better interfaces and increasingly capable subscription products.

What they may not receive is meaningful control.

They could become dependent on a small number of firms that determine how AI may be used, what features are permitted and how much access costs. Smaller competitors might disappear before they can offer different models of ownership, privacy or consumer choice.

The danger is not that every restriction will be dishonest. Some limits will address real risks. The danger lies in giving commercially interested gatekeepers the final authority to distinguish legitimate protection from inconvenient competition.

A safe gate is still a gate.

Does the promise of an "age of abundance" hide the ownership question?

The abundance narrative does more than predict prosperity. It helps establish the leading AI companies as indispensable custodians of the future. Once that image is accepted, allowing them to govern their own technology can begin to look inevitable.

Musk presents AI and robotics as technologies capable of producing extraordinary abundance. Goods and services could become cheaper. Productivity could rise. Work itself could eventually become optional.

That vision may prove partly correct. But the language of abundance can hide the institutional questions that determine whether technological gains become public prosperity.

Who will own the models?

Who will control the data centres, chips and energy infrastructure?

Who will set the price of access?

Who will decide which communities, workers and businesses receive the benefits?

Lower costs do not automatically create fair access. Concentrated ownership can produce technological wealth alongside severe inequality.

Economists Daron Acemoglu and Simon Johnson, who shared the 2024 Nobel prize in economics, place the issue in historical context. Technological progress can create enormous wealth without distributing that wealth broadly. Their warning about the current direction of AI is direct:

"The current path of AI is neither good for the economy nor for democracy."

Daron Acemoglu and Simon Johnson, Power and Progress (2023), quoted by MIT News

Their deeper point is not anti-technology. It is that innovation does not arrive with a built-in system for sharing its gains. Institutions, ownership and power determine who benefits.

The promise of abundance should therefore increase scrutiny of concentrated power, not reduce it.

Where does the UK stand on AI self-regulation?

Britain already runs a voluntary model. Its AI Security Institute tests frontier models before release, but only because developers such as OpenAI, Anthropic and Google agree to give it early access. The UK has no dedicated AI regulator and relies on existing authorities, Reuters reported in August 2026, when AI minister Kanishka Narayan said Britain was open to regulation if voluntary safeguards fell short.

That arrangement has already shown its limits. In September 2026, the White House asked OpenAI and Anthropic to hold new models back from UK testers until US officials had checked them. On 12 October, the UK government said the institute would keep its early access, The Next Web reported. Narayan was careful about how long that would last: "When you're reliant on someone else's technology, you have to be clear-eyed about the permanence of any outcome."

The EU has taken the other path. Since 2 August 2026, the European Commission has been able to fine providers of general-purpose AI models up to 3% of worldwide annual turnover or €15 million, whichever is higher. Its code of practice for those providers is voluntary, but it sits on top of a law with fines behind it.

For what the UK's approach means for companies building AI, see our piece on what Britain's AI minister means for businesses.

What would make AI self-regulation credible?

The Atlantic Council's analysis of the White House accord names independent verification, real consequences and a credible government backstop among the conditions that have made self-regulation work in other industries. The risks set out above add two more.

  • Independent verificationAuditors whose appointment and pay do not depend on the companies they audit, with their findings published.
  • Real consequencesPenalties for a company that falls short, set out in advance.
  • A public backstopA regulator with the power to act when commitments fail, not only when a company invites it in.
  • Proportionate standardsRules scaled to risk, so that safety does not become a barrier only the largest laboratories can afford.
  • A seat for outsidersStart-ups, universities and open-source researchers involved in setting the standards that will apply to them.

What role should AI companies have in AI governance?

AI companies must participate in safety work. Excluding their expertise would be reckless.

They should exchange information about serious vulnerabilities. They should permit credible external testing. They should report dangerous capabilities and provide evidence about how their models were evaluated.

But participation is not ownership.

The companies making AI should not possess final authority over the standards used to judge it. They should not decide alone which competitors can enter the market, which research is permitted or what consumers are allowed to receive.

Their technical knowledge should inform decisions. It should not give them sovereignty over those decisions.

Musk's proposal begins with a sensible idea: competitors can help identify one another's weaknesses. The danger begins when peer review quietly becomes AI self-regulation, and AI self-regulation becomes gatekeeping.

Society could then wake up to find that a few companies do not merely produce its most important technology.

They govern access to it.

What does this mean for businesses buying AI?

Most UK businesses will never sit on a frontier AI standards body, but they will live with its decisions. If a few providers decide which models, features and terms are acceptable, every business that depends on one of them inherits those choices. Three steps keep that dependence in check:

  • Keep the ability to switchBuild so that changing model provider is a configuration change, not a rebuild. Our guide to a multi-model strategy sets out how.
  • Keep an open-weight optionFor sensitive workloads, it can make sense to run open-weight models on your own infrastructure, which gives you more control and auditability.
  • Know where liability sitsUnder English law, risk tends to land on whoever selected, integrated, configured and deployed the system, and contracts largely decide who pays. We explain why in our piece on AI liability under English law.

Keep control of the AI your business depends on

Whatever happens to AI self-regulation, the choices that protect your business are made in how your systems are designed: which models they use, how easily they can change provider and who can see what they do.

At Imobisoft, we help organisations set an AI strategy and design AI solutions around the systems, data and compliance obligations they already have.

Talk to us about your AI strategy

Frequently asked questions

What is AI self-regulation?

AI self-regulation is a system in which companies voluntarily create safety standards, review models, share risk information and determine acceptable development practices without independent authorities controlling the process.

Should AI companies regulate themselves?

Not on their own. They should share safety information, allow credible external testing and report dangerous capabilities, but they should not define the rules, judge compliance and decide which AI systems reach the public.

Has AI self-regulation already been tried?

Yes. AI companies made voluntary commitments to the White House in 2023 and signed the Frontier AI Safety Commitments in 2024. In September 2026, six companies signed the White House Accord on Super Intelligence, which names no penalties.

What did Elon Musk propose?

In a July 2026 interview with The Economist, Musk proposed regular safety discussions between leading AI companies and peer review of advanced models before release. He argues that rival developers are better positioned to recognise technical risks in frontier systems, with government as a last resort.

Why is AI self-regulation a conflict of interest?

The companies evaluating safety would also be competing for customers, investment, computing resources and market leadership. A decision presented as a safety judgment could also benefit the reviewer commercially.

How could AI self-regulation create an AI oligarchy?

A small number of powerful companies could define safety standards, influence market access and determine which models are considered acceptable. Their existing economic power would expand into rule-making authority.

Could AI safety rules harm competition?

Yes. Standards that require expensive infrastructure, extensive testing or access to scarce computing resources may be manageable for frontier laboratories but unaffordable for smaller developers and independent researchers.

Would consumers benefit from industry-led AI regulation?

They might receive safer products in some cases. But they could also face fewer providers, less open technology, greater platform dependence and reduced control over how AI systems are used.

Does the UK rely on AI self-regulation?

Largely. The UK's AI Security Institute tests frontier models before release under voluntary agreements with developers, and there is no AI-specific law. The government has said it is open to regulation if voluntary safeguards fall short.

Is the argument against all cooperation between AI companies?

No. Companies should share genuine safety information and help test advanced models. The objection is to allowing those companies to become the primary or final judges of their own conduct and that of their competitors.

Sources

Have a project in mind? Let’s get to work.

Let’s chat about how we can help you. Fill in the details and we’ll get back to you as soon we can.