Tutorials · Guide · Artifical intelligence

LLMs for the working mathematician

A practical guide for incorporating LLMs into your mathematical research.

Introduction

Large language models (LLMs) are quickly being incorporated into research workflows of scientists from all fields. If you've ever wondered how to do this yourself but are unsure where to start, this tutorial might just be for you!

Popular LLM providers spaced evenly around a circle with lines pointing to the center of the circle toward a large question mark.

LLMs (huh), what are they good for?

LLMs are particularly well-suited for tasks that do not require absolute precision:

  • Literature searches
  • Summarizing
  • Drafting and editing text (if you don't mind LLM idiosyncrasies in your text)
  • Code generation (if you don't mind slightly unhygienic coding practices)
  • Counterexample searches via generate-and-run code
  • Problem reformulation (with a knowledgeable human to check the correctness)
  • Refereeing an article

LLMs are currently not well-suited for:

  • Exact arithmetic
  • Generating novel or creative proofs of mathematical conjectures
  • Tasks where the provenance or output needs to be directly checkable

The goal of this tutorial is to encourage mathematicians to move from casual chatters to deliberate users of AI in their math research.

Chatbots vs. AI coding agents

One thing to clear up before we get started is the difference between a chatbot (e.g., ChatGPT, Gemini, Claude, Deepseek) and an AI coding agent (e.g., Claude Code, Codex, OpenCode). Both of these have their place in a workflow, but knowing the difference is important. For a quick rule-of-thumb, if your task involves multiple files, generating and running code, or iterating on an idea until a goal is reached, lean towards an AI coding agent.

A chat is a conversation with an LLM that is used for drafting text, question-and-answer, or editing a single file. This is how most individuals interact with LLMs.

An AI coding agent is more specialized and involves a LLM reading and editing files on your machine. These tools can write and execute code that runs on your computer or in a protected sandbox.

Practical tips for using LLMs in your research

General tips:

  1. Input source files, not PDFs.
    • While LLMs can read a PDF, they may make mistakes. It is usually best to upload to upload your files in their raw form: .tex, .bib, .docx, .xlsx, .py, .r, etc. This way the LLM can see everything instead of inferring/translating from the PDF.
    • For example, if you want a high-quality referee report on a math paper, you should upload your .tex and .bib files into a chat with the LLM and write a prompt along the lines of:

Please write a referee report on the attached preprint that I recently wrote. Check line-by-line, and tell me only about things that should be changed, small or large. Write the response to a file.

  1. Manage your context.
    • If you choose to include something in your prompt, the LLM will not easily forget it. On the flip side, if you leave something important out, the LLM has no way to read your mind. Precision is key.
    • Don't be afraid to start a fresh conversation or coding session. A long back-and-forth with an LLM can degrade the quality of the responses, and an error introduced early will carry through a conversation.
  2. Don't expect your LLM to be honest.
    • LLM hallucinations are a well-documented phenomenon. The LLM is not designed to be an unbiased judge that delivers absolute truth; it is designed to always give you an answer, right or wrong. It is up to the user to be weary of the output of an LLM. A good motto is "trust, but verify."

Math-specific tips:

  1. Goal-directed prompts will usually save you time, and they avoid a lot of back-and-forth.
    • In Claude Code, for example, the /goal slash command is an example of this.
    • You should state the current goal, any constraints you might have, and what success looks like upfront. You can also ask for code that will achieve the goal if you are comfortable running the code yourself. Here are some examples of prompts that follow this advice:

I want to determine whether every integer n with 2 <= n <= 10^6 is expressible as p + 2k^2 for a prime p and integer k. I want you to write and run Python code with exact integer arithmetic. The output should be a complete list of exceptions in this range, or a confirmation message that there are no exceptions.

Test whether F(t) = sum_{k=1}^{100} cos(k^2 t)/k^2 ever drops below -1.2 anywhere on [0, 2pi]. Write some Mathematica code to do this which implements dense sampling and local refinement near candidate minima. Make sure the code produces a plot and reports the minimizing t and value of F(t) to 6 digits.

Find and list all of the hypohamiltonian connected, simple graphs on 15 vertices.

  1. Consider using a chat conversation or AI coding agent session in place of parts of your pre-existing workflow.

    • An example workflow before LLMs may look something like: curiosity, literature search, conjecture formulation, numerical checks/(counter)example hunting, pattern analysis, proof sketching, and drafting.
    • With the help of LLMs, the workflow can be augmented to something like: human curiosity, chatbot searches the literature, human conjecture formulation, AI coding agent runs numerical checks/searches for (counter)examples, human analyses patterns, human writes proof, human drafts preprint, chatbot referees preprint for mistakes.
    • Despite having more steps, the second workflow will be much faster than the first!

  2. LLMs are a jacks-of-all-trades, masters of none.

    • Your domain specific knowledge will usually outclass an LLMs performance in that area. The entirety of math knowledge is not contained on the arXiv; in particular, LLMs do not have access to all of the failures that came before the successes. These come up conversationally in offices and seminars, or privately in your own head. Trust your own judgement on what is likely to be correct.
    • That being said, it can be helpful to lean on the output of an LLM when you have little or no domain specific knowledge. If you want to write a run-of-the-mill program but do not know how to code, asking an AI coding agent to do it for you in an hour is much more efficient than teaching yourself how to do it over the course of a few days.

Conclusion

While there are many nuances to incorporating LLMs into your workflow (model selection, cost considerations, privacy concerns, output verification, environmental impacts, etc.), there is a lot to gain by using the tools that have been developed. It is helpful to think of a LLM as a fast, knowledgeable, overconfident collaborator. They can produce a lot of useful information very quickly, but it needs to be verified by an intelligence that is not artificial.

← All tutorials