Will AI ever outsmart humans? Philosophers weigh in

As AI becomes increasingly capable of performing human tasks, philosophers are left with new questions about the nature of humanity. At the top of their list: Can AI overtake human intelligence? What happens if it does?

For Ken Archer, philosopher and head of responsible AI at Microsoft, a capable AI can never equal an intelligent one.

“There’s been this vision from AI from the beginning … that we would be able to have all knowledge in such clear and distinct form that we could mechanize it,” Archer told the audience packing the fifth-floor conference room at the Berkman-Klein Center for Internet and Society.

But that vision overlooks the very nature of thinking, Archer continued. “To be human is to be defined by our own finitude. Our finitude is central to how we think, to what it means to be rational, to be thinking, because we realize that there’s always more to know.”

At a September 14 talk, co-hosted by the Division of Arts & Humanities’ Public Culture Project and the Berkman-Klein Center, Archer explored the distinctions between human and artificial intelligence with Jackson Kernion ’12, philosopher and technical staffer at Anthropic. Moira Weigel, assistant professor of comparative literature, moderated the discussion.

Archer argued that humans contain lived experiences in the world we are “thrown” into, to cite Martin Heidegger’s philosophical concept. Large language models can produce fluent text only because their source material — human language — contains a richness that comes from lived experience, Archer said. LLMs don’t inherit the lived experience itself.

Kernion agreed that humans carry the context of personal lived experience in ways AI does not.

“These language models, from their perspective, they pop into existence from the first token of the context window until they’re done generating,” Kernion said. “This is very much unlike us. Every moment of my life, I’m bringing with it all my experiences that I’ve accrued all along. I’m not being reset every conversation.”

Kernion, who said he views AI as a tool or collaborator whose purpose is to serve humans rather than surpass them, said he hopes for a “merge” scenario, in which humans and AI become more capable together.

“Ultimately, the models are built to do our bidding and within constraints,” Kernion said. “I’m on team ‘AIs are the Dogs,’ and we should be holding the leash.”

Archer said he does not believe AI will surpass humans in superintelligence, since current AI systems don’t demonstrate the same general intelligence humans do. He described AI as operating within a “closed world” defined by rules, similar to a chess program. Humans, on the other hand, operate in an “open world” where anything can happen, and because of this, we’re constantly re-evaluating our knowledge.

“As the world frustrates us and disappoints our expectations constantly, we step back from the world and clarify our ideas, clarify our plans,” Archer said. “When presented with an impossible task, does [AI] step back from the world, and say, ‘Well, what is the significance of this task that I have been given?’ like we would, like is constitutive of general intelligence? No.”

When asked about AI rendering people obsolete in the job market, Kernion suggested AI would eliminate some fields, including math. But he doesn’t see that as a negative. Rather, Kernion explained that job displacement could open new intellectual possibilities for people to focus on, like philosophy.

“I think AI is going to make us look in the mirror and say, ‘All this intellectual labor. What were we doing all along?’” Kernion said. “Why were we having all these meetings? Why were we writing these memos? What were we doing at work before?”

Archer strongly disagreed that AI will eliminate the need for people to do math.

“When you look at the history of math, what you see are great acts of creativity,” he said. “These involve a reflection on our fundamental lived experience of the world, and then idealizing that experience in totally new, creative ways. That is a capability unique to humans.”

Weigel asked both philosophers to explain what worries them most about the future of AI.

Archer answered that he fears people will surrender their own judgments and responsibilities.

“Were [AI] to tempt us to displace our perspective, our responsibility onto models — that is the real crisis. That’s what we want to avoid,” he said.

AI gaining autonomy is more worrisome, he added, than leaps in capability, because capable models are only dangerous without sufficient controls and monitoring.

Kernion said he is concerned about misuse of AI, since the same capabilities that can protect systems can also attack them. He believes more work should be done to develop robust guardrails.

The talk, which was titled “‘What Rough Beast…?’ On AI’s Potential to Surpass Humanity” in reference to the 1919 W.B. Yeats poem “The Second Coming,” also touched on the long history of scholars grappling with rapidly changing societies.

Kernion said we should think of the AI boom as similar to past technology shocks, including radio, electricity, and the internet. 

“Those technologies did change human civilization, so I think this will change us,” he said. “We will have to adapt.”

Archer agreed that regardless of how we choose to use the tools, change is on the horizon.

“Technology can structure the way we experience the world in the first place,” he said. “Writing absolutely did that, definitely radio did that, and AI is going to do the same.”