Paolo Benanti, a priest who advises the Catholic Church on artificial intelligence, is challenging the idea that frontier laboratories should define the rules for the technology they build. In an interview with Wired, he argues that public debate about governance is being displaced by dramatic claims about superintelligence.
His concern is not that advanced systems should escape scrutiny. It is that the companies warning about extreme future risks also have economic and strategic incentives that shape how those risks are framed. Standards, in his view, require democratic legitimacy beyond corporate boards.
Existential Warnings Can Narrow the Debate
Warnings from researchers at major AI labs increasingly focus on models that could surpass human capabilities and potentially cause catastrophic harm. Benanti says this framing can make the problem appear so complex that only the laboratories themselves are qualified to manage it, excluding wider participation.
He directs attention toward who gets to decide rather than toward a single forecast about superintelligence. Ethics, he argues, belongs in the public square. His use of the word “cartel” describes a situation in which a small group of companies both dominates a market and sets the rules under which it operates.
That is Benanti’s assessment, not a finding by a competition authority. Anthropic declined Wired’s request for comment, while OpenAI and Google DeepMind did not respond. The article also notes that leading labs have supported some calls for slower development while preparing for public listings and competing for a market worth billions of dollars.
The rivalry between the United States and China adds another incentive. The most advanced models are being developed in those two countries, and geopolitical competition can encourage both speed and secrecy. Benanti argues that this overlap of commercial, political and personal motives makes sweeping predictions about future capability difficult to evaluate.
Human Control Matters More Than Banning Methods
Benanti is less interested in prohibiting specific development techniques, which he fears could suppress innovation. He instead favors standards intended to prevent models from slipping beyond human control. The important condition is that those standards should not be designed solely by the companies subject to them.
That position differs from proposals to ban recursive self-improvement or restrict model distillation. Anthropic chief executive Dario Amodei has called for mandatory third-party evaluations before new models are released and for limits on rivals training systems from proprietary model outputs. Benanti’s emphasis is institutional: independent governance should determine acceptable boundaries.
He also rejects language that treats superintelligence as a near-divine force. In his account, AI remains statistical computation rather than a new being. That distinction matters because describing technology as incomprehensible or omnipotent can discourage ordinary political oversight and make corporate claims harder to challenge.
His argument does not dismiss technical expertise. It separates expertise from authority: laboratories may understand their systems best, but that knowledge alone does not give them a democratic mandate to determine what risks society must accept.
The Church Is Entering the Governance Argument
Benanti sees AI as a subject that can give religion a renewed role in public debate, particularly around the line between people and machines and the effects of technology on future generations. His position connects technical governance to questions about human value rather than treating performance as the only objective.
Pope Leo made AI the subject of his first encyclical in May. According to Wired, the letter warned against an industry logic that treats efficiency as the ultimate measure and people as objects to optimize. It also called for cooperation among rival governments and companies instead of a race for geopolitical or commercial dominance.
The Church’s intervention does not supply a detailed regulatory architecture. Benanti does not specify the institution that should write standards, the tests models should pass or the enforcement mechanism. His contribution is a boundary around the process: powerful developers can provide expertise, but they should not have exclusive authority over the rules that constrain them.
