A moral crusade against AI takes shape

The pope takes on AI, chatbots abet a mental health crisis, and per MIT, generative AI use impairs learning. The Critical AI report, June 22nd edition.

Greetings all,

Hope everyone’s hanging in there during [whatever fresh horror has most recently arisen as you are reading this]. Seeing as how we now teeter on the brink of all-out war, I almost gave up on publishing this edition altogether. But there’s just so much going on in the AI world, and serious new currents that bear examination.

To wit: The New Pope has come out swinging against AI, just as more disturbing and detailed stories of chatbots exacerbating mental health crises are surfacing, and a new scientific paper that, if it checks out, confirms some of our worst fears about AI’s impacts on human cognition. Is Leo beginning a moral crusade against AI—should he? We’ll get to it all in this week’s Critical AI report.

A quick note: The full post here is for paying subscribers. I’ve been light on the paid-only stuff lately, mostly because I’ve been focusing on longer-form reportage, AI Killed My Job interviews, and some other stuff in the works. Thanks to everyone for bearing with, and many, many thanks to all who chip in to support this work; I’m truly grateful to all of you. If you find value in it, and you read regularly—and you’re able— consider upgrading, too. Simply put, paid subscribers make all this work possible.

How the new pope came to take on AI

The first-ever American pope dubbed himself Leo, after the previous Leo, XIII, who got himself nicknamed “Pope of the Workers” for advocating for labor unions and critiquing industrial capitalism back in the 19th century. The American Pope Leo XIV, meanwhile, claimed the mantle to show his intent to advocate for human and worker rights in the age of AI.

The Wall Street Journal published a deep dive into how Leo XIV arrived here, how he’s in many ways following a lead staked out by Francis, and why he is, essentially, our first day one papal tech critic.

From the story:

VATICAN CITY—Two days into his reign, the new American pope spoke softly to a hall full of red-capped cardinals and invoked the digital-age challenge to human dignity he intended to address with the power of his 2,000-year-old office: artificial intelligence.

The princes of the Catholic Church listened intently as Pope Leo XIV laid out his priorities for the first time, revealing that he had chosen his papal name because of the tech revolution. As he explained, his namesake Leo XIII stood up for the rights of factory workers during the Gilded Age, when industrial robber barons presided over rapid change and extreme inequality.

“Today, the church offers its trove of social teaching to respond to another industrial revolution and to innovations in the field of artificial intelligence that pose challenges to human dignity, justice and labor,” Leo XIV told the College of Cardinals, who stood and cheered for their new pontiff and his unlikely cause.

Interesting details abound here, both about the history of Leo XIII, who really did inveigh against the excesses of industrial capitalism and make crucial strides in driving the church to advocate for suffering workers, and about Francis’s evolution in his thinking about Silicon Valley and modern tech.

In a groundbreaking 1891 encyclical called Rerum Novarum—Latin for “Of New Things”—Leo XIII backed calls for labor unions, living wages and safer working conditions. But he also affirmed the right to private enterprise and property.

“The hiring of labor and the conduct of trade are concentrated in the hands of comparatively few; so that a small number of very rich men have been able to lay upon the teeming masses of the laboring poor a yoke little better than that of slavery itself,” Leo XIII wrote.

By invoking Leo—about as radical an icon on the labor front as a pope can get, it seems—the new American Leo appears to be preparing to affix his legacy to addressing AI. And that’s at least in part because his predecessor, Pope Francis, had soured so deeply on Silicon Valley and big tech. Francis started out his term sanguine, embracing tech like Twitter and Snapchat and welcoming Valley honchos to the Vatican. But as the Valley’s excesses of became clearer—the mass slaughter of Rohingas in Myanmar, spurred on by viral Facebook posts that the company declined to address, seemed to be a tipping point—Francis outlook darkened accordingly, and he began calling for governments to step in.

In one meeting, Francis compared AI’s dependency on web content produced in a few languages—largely English—to the story of the Tower of Babel, in which overconfident humans, speaking a single tongue, attempted to build a tower to heaven, prompting God to scatter them and sabotage their creation.

During another meeting, the pope warned tech leaders against believing that they understood humans just because their data could predict human behavior: “You miss their humanity because you cannot reduce the human being to its data,” Father Salobir recalled…. Later that year, Francis warned of a “technological dictatorship” and called on governments to develop a legally binding international treaty to regulate AI.

So the new pope names himself Leo, after a staunch critic of industrialization and a proponent of worker’s rights. (Influenced indirectly or otherwise, perhaps, by the legacy of the Luddites.) If Francis built his reputation on advocating for the poor and calling for action to address climate change, it seems Leo is poised to make a similar move with regards to AI. Time will tell. Regardless, tech leaders are now desperately trying to set meetings with the Vatican to plead their case to the pope, in hopes that they can limit the chances that he’ll mount any kind of sustained campaign against them or their most-hyped product category.

So far, Leo is holding fast. On Friday, June 20th, the Vatican News published Leo’s comments on AI delivered at the second annual Rome Conference on Artificial Intelligence:

"All of us, I am sure, are concerned for children and young people, and the possible consequences of the use of AI on their intellectual and neurological development."

"Our youth," he insisted, "must be helped, and not hindered, in their journey towards maturity and true responsibility," underscoring they "are our hope for the future."

"Society’s well-being," he also underscored, "depends upon their being given the ability to develop their God-given gifts and capabilities, and to respond to the demands of the times and the needs of others with a free and generous spirit."

These comments could not be more timely, given the fresh round of reports of the malign impact AI is having on young people, education, and cognitive development. More on that in a second.

To me, the pope’s comments throw into relief the relative lack of institutional moral leadership over AI’s risks, harms, and excesses. Our most mainstream news orgs are all but cheerleading the ascent of companies like OpenAI and their products. Higher education is too often content to blithely charge ahead—the California State University system rushed to ink a deal to become the “nation’s first and largest AI-empowered university system.” Colleges like Ohio State have followed suit. US Congress is trying to pass a law that will *ban* states from regulating or legislating AI at all. The White House wants no guardrails at all, and has vowed to see the American AI industry dominate.

All of this while the full scale of the moral crisis that AI stands to beget could not be clearer….