Solving an urgent math problem: How to educate grad students amid AI disruptions

Mathematics is built upon thousands of years of scholarship. But in recent months, the foundations of this ancient discipline have been shaken by an epochal disruption: the rise of artificial intelligence.

Seeking to adapt to the rapidly changing landscape, Harvard faculty convened a two-day summit this fall on revamping doctoral math education for the age of AI. The two dozen prominent mathematicians in attendance produced a report recommending urgent changes: Written work by itself — even a brilliant dissertation — is no longer sufficient proof of mastery. In evaluating students, oral questioning must become more rigorous and frequent. Human-to-human communication — far from being obsolete — will become more important than ever. Students should carefully adopt AI tools to supplement their own brainpower, not to supplant it.

“The incredible pace of advancement in the capabilities of AI tools in mathematics is likely to transform the field of mathematics over the next few years in ways that are much bigger than the total change in the field over a hundred years,” said summit co-organizer Melanie Matchett Wood, William Caspar Graustein Professor of Mathematics and director of graduate studies for the Department of Mathematics.

One key aspect of that transformation will be training future mathematicians. At the summit, seven Harvard faculty were joined by 17 colleagues from institutions such as the University of California, Berkeley; Stanford; Brown; MIT; Princeton; the University of Texas at Austin; and Johns Hopkins University. Summit co-organizer Dan Freed, Shiing-Shen Chern Professor of Mathematics and director of the Center of Mathematical Sciences and Applications, said the agenda included anticipating changes and ensuring the discipline continues to attract young talent. The group aimed to produce “proactive recommendations to guide Ph.D. students toward the next chapter in the long and continuing story of mathematics,” Freed said.

‘Rigorous in-person assessment’

The summit report drew on the experiences of a handful of leading math programs, but aimed at a wider audience of universities everywhere.

In mathematics, the centerpiece of the doctoral education long has been the dissertation — a written document that exhibits mastery of some area of the field, while usually serving as the first demonstration of a mathematical statement. For generations, it was taken at face value that the student produced the original work. Not anymore.

“AI can produce things that could have been acceptable Ph.D. theses a couple of years ago,” said Wood, who also serves on a new Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study.

The dissertation will remain a key component but will be supplemented by more frequent and probing oral questioning. The report recommended that doctoral programs require “rigorous in-person assessment” every year by multiple faculty who provide detailed feedback. Until now, the dissertation defense has been a largely ceremonial exercise conducted shortly before graduation. But in the future, there should be more exacting verbal examination to verify mastery.

As before, the topic should remain an original contribution at the frontiers of mathematics, but the dissertation need not be the first proof of a statement. The change reflects the new reality of AI-generated math. Sometimes, LLMs can quickly produce solutions nobody understands, so a student can still demonstrate mastery and make a novel contribution by producing the first properly presented, comprehensible proof.

Homework remains essential for learning, but it should play a minimal role in assessment because it, too, can be done by AI. The take-home exam is largely obsolete.

At the Harvard Kenneth C. Griffin Graduate School of Arts and Sciences, the math program enrolls about 60 Ph.D. candidates. The experience of current students is likely to be very different from those who graduated only a few years ago.

“Right now, we are facing two challenges,” observed summit participant Lauren K. Williams, Dwight Parker Robinson Professor of Mathematics. “One challenge is to figure out how to adapt our teaching and advising in light of AI tools. The other challenge is to deal with the perceptions of the public, and especially of young people.

“Many people are misunderstanding what the advances in AI capabilities mean for mathematics,” Williams continued. “Some people have this idea that now there are fewer interesting problems to work on. But the field of math is infinite, and we have a new powerful tool to explore it. So, if anything, we need more mathematicians to go into the field of research.”

At the mid-September summit, participants discussed how to respond to anxieties present in some parts of their community, particularly among aspiring mathematicians who worry about career opportunities in the age of AI. In one session, a panel of postdoctoral fellows and current Harvard Griffin GSAS students described how they had integrated the new tools into their own math research.

“Their deep thoughtfulness and well-considered approach are precisely what is needed to carry the longstanding traditions of mathematics forward and incorporate new possibilities opened up by AI,” Freed recalled. “After engaging with them, we all emerged inspired and optimistic for the future.”

Ultimately, the summit report struck a note of optimism. It reads: “If we can harness the power of AI tools to increase our mathematical understanding and creativity — in addition to using them to solve problems — then the future is very bright.”

Chalk and talk

One recent morning, Harvard Griffin GSAS student Jack Miller walked into his weekly meeting with Wood, his adviser.

In one hand, the second-year Ph.D. math candidate carried two pages of handwritten notes summarizing recent progress on his research project — an intriguing problem at the intersection of number theory and knot theory. In the other hand, he toted a coffee mug filled not with caffeinated stimulus but multicolored chalk.

Miller stepped into Wood’s office and started writing on the blackboard. Giving his presentation what he called a “provocative title” — “Topological Statistics” — he sketched a trefoil knot, a shape with interwoven arcs. For the next 45 minutes, Miller outlined his work while Wood peppered him with questions and offered suggestions.

Miller has been working on the problem for months. He sought out professors for advice, adopting one mathematical vocabulary to talk with number theorists and another to chat with knot theorists. He has spent hours pondering drawings of doughnut-shaped spaces and seeks to create a new synthesis of ideas.

At times, he has relied on AI to perform tasks like data analysis, numerical simulation, or literature review. But for the big conceptual questions, he prefers to think quietly at his desk.

“Technology cannot replace the way we meditate on ideas,” he said.

Indeed, the summit report emphasized this point. The models can be beneficial for tasks such as making figures, literature searches, proofreading, or rapidly obtaining overviews of unfamiliar topics. But the report cautions against using AI in ways that stunt intellectual growth.

“Students should be advised to be extremely careful before offloading anything that will help them learn how to think,” warned the report. “In particular, students should develop the capacity to think for long periods about hard problems.”

Doctoral programs also were urged to place more emphasis on communication — particularly the ability to succinctly express essential mathematical concepts in language comprehensible to varied audiences. This will allow mathematicians to share information between subfields or scientists from different disciplines and extract pearls of wisdom from the glut of computer-generated math.

Although the deluge of AI has reshaped the landscape, Wood and her colleagues are quick to emphasize one essential message: The guide stars of the ancient science remain unchanged.

“We want as much human understanding of mathematical truth as we can get,” Wood said. “That is still the goal, but the mechanics of our profession, and how you demonstrate you have understood something, might be very different.”

Envisioning something new

Mathematics requires scholars to contemplate abstract concepts. The challenge now includes envisioning an uncertain future.

As an aspiring academic, Miller has gone through moments of questioning his career path. Early this fall, he was stunned when OpenAI summarily released solutions to hundreds of problems — including some he had hoped to pursue.

“It was not doomsday, but with 700 papers being released all at once, it was just overwhelming,” he said, adding that his adviser and other mentors provided reassurance.

“Over time,” he continued, “the answer became more clear: There will be new problems — more interesting problems, more beautiful ones, and we will just shift gears. I am pretty early in my Ph.D., and this actually encourages me to dream big.”

And so he did, exploring analogies between fields such as topology, algebraic geometry, and number theory, using some AI but mostly the old tools of brainpower, chalk, and talk.

Miller works in arithmetic statistics, or searching for subtle patterns in prime numbers and polynomials. Between the possibilities of an arithmetic property being true or false lies a gray area of probabilities — and Miller is discovering that his “magical” probability formulas hint at deeper mathematical structures.

By the end of his presentation, he had filled the blackboard in his adviser’s office and left her intrigued.

“I’m excited,” Wood said. “I know I was telling you not to get distracted by tricolorability, but I agree — this is cool! You should push this through.”