Working with AI

Version 1October 2026The positions and policy on this page are subject to change.

This page sets out my position on AI in research and the evidence on AI and learning behind it. It also states my AI policy for written work I assess at Leiden University, theses included. It is written for students, but other faculty and administrators may remix and reuse it.

Jump to the policy

Definitions Terms used on this page Key terms and definitions used here.

• Generative AI. AI that produces new text, images, code, or other content in response to a prompt. The Faculty’s guidelines call it GenAI. Legally, it is one kind of “AI system” as defined in the EU AI Act, Art. 3(1).
• Large language model (LLM). The model behind most generative AI tools, trained on vast amounts of text to predict and generate language (Anthropic glossary; OpenAI Academy).
• Chatbot or AI assistant. An app for conversing with an LLM, such as ChatGPT, Claude, or Gemini.
• AI agent. A system in which an LLM directs its own steps and tools to complete a multi-step task, such as running and checking code (Anthropic, “Building Effective AI Agents”). Claude Code and Codex are agentic coding systems.
• AI tools. Any of the above, where the distinction does not matter.

---

A new moment

Generative AI tools can now write, code, search, and, as AI agents, carry out multi-step tasks on their own. Their quality is still jagged across models and tasks, but they are already good enough to change how knowledge work is done, and they are improving faster than universities can adapt. Depending on the period measured, the length of software and research tasks that frontier systems can complete on their own has doubled every four to seven months for several years (Kwa et al. 2025). In formal mathematics, AI has produced machine-verified solutions to several open Erdős problems that professional mathematicians accept as novel, and it reached an officially graded gold-medal performance at the 2025 International Mathematical Olympiad. The tools you use for your thesis this year are likely the worst you will ever use. Even if development stopped today, the models already available are powerful enough to transform how research is done.

Living through a watershed moment is disorienting, and it calls for grace and humility. None of us, students or supervisors, knows yet what good scholarship with AI looks like, and anyone who claims otherwise is masking uncertainty with bravado. That said, humility should not translate into passivity, which a university cannot afford. We have to take this seriously, especially when it comes to assessing written work and protecting the learning that has long defined writing a thesis. When we award a degree, we vouch that its holder can ask a good question and build an argument of their own from evidence they have read closely. If we can no longer say that credibly, our degrees will lose much, if not all, of their value, and we all pay for that.

---

Where I stand

I hold three positions.

First, the technology is remarkable, and banning it would be a mistake. I use AI every day in research and administration, for searching the literature, building and checking corpora, writing and testing code, and verifying what sources actually say. I used it to help build this site. I am also building a separate site, AI for Research, with tools, demonstrations, and guidance for students and faculty. You can and should find ways to use these tools, but you have to go about it carefully.

Second, it is unclear whether the benefits for learning persist. The research section below summarizes the current evidence. In general, AI appears to improve output on a task, usually at some cost. When the same people are tested without AI, most of that improvement is lost, and some even become less skilled. Durable gains went to people who already had domain expertise and used AI to augment it rather than substitute for it. For students who are still developing foundational skills, offloading that development is something to avoid at all costs.

Third, and following from that, you, the human, must stay “in the loop,” and the work you turn in, from a term paper to a thesis, has to be yours. That includes the question you ask, the reading you do, the evidence you marshal, and the analysis and arguments you build from it. The machine may be a useful collaborator in parts of this process and can help improve your work. But do not outsource either the thinking or the writing. The writing, especially, has to be yours. Even if an idea came out of extensive discussion with ChatGPT, you need to be able to explain it on your own, without assistance.

I am not against AI help with writing, such as grammar and spelling checks. It is plainly a great technology for non-native English speakers, and denying that is unrealistic. Its greatest potential lies in letting you do something you otherwise could not do, or do much better something you can only do in a limited way. That might mean building a larger and more sophisticated corpus of documents, or analyzing those documents with resources that would otherwise be out of reach or too time-consuming (supercomputers, for example). The aim is to do better than the counterfactual in which you had far fewer documents, or none at all.

Ultimately, the question is what remains scarce as intelligence becomes cheaper. Do not offload your thinking to the point where you can no longer think critically or make an argument persuasively. You also need to be able to read and understand your sources, which takes focus. These abilities will likely be scarce, and university is where you develop them. They are worth protecting and worth augmenting. The university’s task now is to produce students who can ask better questions about more worthwhile problems and work on more ambitious projects, while explaining what they are doing clearly and persuasively to other people.

---

What does the research say?

The literature on AI and learning is young, but there is enough of it to draw one conclusion. Working with AI does not necessarily help you learn, even when it produces better work. If you are a student considering AI, the best current studies should make you wary of assuming that the benefits you feel are real. We also do not yet know what AI does to your ability to write a thesis-length piece of research under supervision, or to perform a year later without it. Full references are at the bottom of the page.

Performance versus learning

AI tends to improve the work, much less the learning

How you use AI determines whether it helps or harms the development of your skills.

One of the most consistent findings so far is a gap between performance and learning. In experiments, people who use AI on a task often do much better than a control group without it. When they then do the same or a similar task without AI, the gain tends to disappear, and sometimes reverses. Two meta-analyses of AI and student learning found large effects on performance, about two-thirds of a standard deviation (Deng et al. 2025; Wu et al. 2026). These estimates do not reflect durable learning. In 9 of the 62 studies Deng et al. reviewed, students had ChatGPT during the post-test, and 33 studies did not report whether they did. Much of the literature claiming that “AI improves learning” shows only that “AI improves performance on tasks done with AI.”

One of the cleanest demonstrations comes from a randomized experiment with high-school students solving math problems (Bastani et al. 2025). About a thousand students were randomly assigned to practice with (i) unrestricted GPT-4, (ii) a “guardrailed” GPT-4 that gave hints but not answers, or (iii) no AI. They were then tested on a math exam without AI. Compared with (iii), students in (i) did better in practice but worse on the exam. Students in (ii) did even better in practice and no worse on the exam. The model was the same in (i) and (ii). The only difference was how the tool was set up, giving answers versus only hints.

Show the values as a table

Condition	Practice with AI (pp)	Exam without AI (pp)
Unrestricted ChatGPT (GPT Base)	+13.7 (SE 3.1)	−5.4 (SE 2.2)
Guardrailed tutor (GPT Tutor)	+36.1 (SE 3.2)	−0.4 (SE 1.3)

The control group averaged 28.4% on practice and 32.1% on the exam. The unrestricted group scored 48% higher in practice and 17% lower on the exam than the control. The tutor group scored 127% higher in practice with no measurable exam penalty. One selective school in Turkey, autumn 2023, one subject.

Other studies find the same gap. Students in an introductory college programming course did much better on an exercise with AI but gained no more knowledge than other students (Bassner et al. 2026). University students revising essays with ChatGPT produced better revisions than any other group, but they gained no more knowledge and transferred it no better (Fan et al. 2025). Across seven experiments with more than 10,000 participants, people who learned a new topic from an LLM synthesis ended up with shallower knowledge than people who learned from search results, despite learning the same facts, and afterwards gave worse advice on the topic (Melumad and Yun 2025).

How you use AI matters, as the Bastani experiment shows, and a systematic review of AI use in classrooms points the same way (Li, Cui and Hagedorn 2026). Use AI boldly, below, sets out three modes of use.

Who benefits

Experts are positioned to gain the most from AI

Durable learning gains go to those who have prior domain expertise and know how to work with AI and check its output.

Studies of professional work help show who gains. In a field experiment, Autor et al. (2026) had patent lawyers draft patents with an AI assistant for 90 days and then asked them to review patents without it.

Show the values as a table

Group	Drafting with AI (SD)	Reviewing without AI (SD)
Junior (under 7 years)	+0.60 (SE 0.26)	−0.03 (SE 0.20)
Senior (7 or more years)	+0.30 (SE 0.17)	+0.45 (SE 0.19)

Expert ratings. The junior/senior difference on unaided review is a subgroup pattern and is not itself statistically significant. The pooled unaided effect is 0.32 SD.

I include this study, caveats and all, because it matches what I see in supervision. The people who get a durable boost from AI already know enough about the domain to direct the tool and catch its errors.

The strongest counter-evidence comes from a study of customer-support agents. A generative AI assistant raised productivity most for the least experienced agents, and some of the gain persisted when the software was briefly offline (Brynjolfsson, Li and Raymond 2025). But the task was heavily scripted and the outages were brief. Much of what the tool did was codify the knowledge of the best agents. The study shows that a well-designed assistant can teach a routine. It does not show that unrestricted use teaches you to think.

Other workplace experiments find that AI helps less experienced workers more while they are using it. College-educated professionals finished writing tasks 40% faster and with 18% better quality using ChatGPT, and the gap between stronger and weaker writers narrowed (Noy and Zhang 2023). BCG consultants completed 12% more tasks, faster and better, when the tasks were inside AI’s jagged frontier, but were 19 percentage points less likely to get a task right when it was outside (Dell’Acqua et al. 2026). These findings do not contradict Autor’s, because they measure performance while people are using AI. Autor also measures what the lawyers could do once the AI was taken away. For a student, that second question is the one that matters, and there the evidence favors those who already know enough to judge what the tool produces.

Self-assessment

It is hard to know when AI is helping you

People who think AI is helping them often fail to notice when it is hurting them. This holds for workers and students alike.

In a study by Becker et al. (2025), experienced open-source developers working on real tasks in their own code bases were randomly allowed or not allowed to use AI. They took 19% longer with AI. Beforehand they expected to be 24% faster, and afterwards they believed they had been 20% faster.

Show the values as a table

Measure	Change in completion time
Developers' forecast before the study	−24% (faster)
Developers' belief after the study	−20% (faster)
Measured effect	+19% (slower)

Sixteen experienced maintainers, 246 real issues, early-2025 tools. The authors are explicit that this is a snapshot of one demanding setting and not a claim that AI slows most programmers. The point here is the gap between belief and measurement, from people whose other forecasts were well calibrated.

Students show the same pattern. In a preprint, Eleftheriou et al. (2026) found that the condition with the highest self-reported understanding produced the lowest measured learning. In Bassner’s experiment, the programming students preferred unrestricted use of the tool, which was also the condition in which they learned least. In a survey of knowledge workers, Lee et al. (2025) found that the more people trusted AI, the less effort they reported putting into critical thinking. The measure is self-reported rather than a direct test of critical thinking, but it is suggestive. Expect your sense of whether AI is helping you learn to be unreliable, and set up your workflow accordingly.

---

The policy

My AI use policy builds on the Faculty of Humanities Guidelines for the Use of GenAI in Assessment (27 August 2025). In short, I hold that generative AI (including chatbots, AI agents, and other AI tools) may not be used in work that I assess unless I have explicitly permitted a specific use. This use must be pre-approved and formally disclosed. Records of prompts and outputs must be kept and handed over upon request. Below is more detail on my AI policy.

Applies to all written work I assess (incl. theses)

Four rules

1. 1

   Agree on AI use in advance

   Before you accept AI into your research workflow -- whether a chatbot, an AI agent, or otherwise -- you need to discuss with me which tools you plan to use and for which tasks. I want you to show me that you understand what the tool does and demonstrate why you need to use it. We must agree on this use in writing. No agreement will cover having AI write the prose of your manuscript. For thesis research, the agreement can form a short section of your proposal. Without prior authorization, no AI use in the research and writing process is permitted, which is the baseline Faculty policy.

2. 2

   Disclose your AI use

   Any written manuscript must include an AI-use statement (template below) that identifies all models used and the tasks they were used for. Any additional tools used must also be disclosed. You are encouraged to err on the side of providing too much information rather than too little. The principal concern here is transparency.

3. 3

   Expect a Pangram check

   I will use Pangram to check for AI-generated text. Pangram classifies passages as human, AI-assisted, or AI-generated and reports the share of the document falling into each category. For classes, I will scan a random selection of manuscripts per written assignment. For theses, you can expect all written manuscripts (draft or full) to be checked. AI-generated prose is highly detectable, particularly when a document is predominantly or entirely machine-written (Jabarian and Imas 2025; Van Vlasselaer, Van Droogenbroeck and Spruyt 2026), and watermarking of model output is being deployed at scale (Dathathri et al. 2024). AI-detection software was unreliable in the past, but this is no longer the case. If you are mainly or entirely outsourcing the writing process to the machine, this is accurately and reliably measurable. Pangram's technical report shows extremely high detection rates for fully AI-generated text (Glickenhaus et al. 2026). Pangram also gives each document an overall verdict, but I work from the share of text it flags as AI-generated, which runs from 0 (fully human) to 100 (fully AI-generated).

   My standards are deliberately generous. The percentages refer only to the share of your work that Pangram flags as AI-generated, not including the share flagged as AI-assisted.

   If less than 25% of your work is flagged, nothing happens. This margin allows for grammar and spelling checks, editing tools, and other light uses of AI, including influence on your writing you may not have intended.

   If between 25% and 50% of your work is flagged, I will generally not grade your manuscript as submitted. If you had my permission to use AI tools, we will meet to discuss the nature and extent of your use and how it relates to the flagged material. Depending on that conversation, I might ask you to defend your work orally and then revise and resubmit your manuscript. If you did not inform me about your use of AI tools, your manuscript fails, and you will have to redo it as a resit.

   If between 50% and 75% of your work is flagged, we will meet, and you will have the opportunity to show me drafts, version history or other evidence that you wrote it yourself. Unless you can do that, your manuscript fails, and you will have to redo it as a resit. Informing me in advance does not help you here, since no agreement will allow AI to write your work for you.

   If more than 75% of your work is flagged, I will not grade your manuscript. I will consider it possible research fraud and refer it to the Board of Examiners.

   25% 50% 75%

   No action Meeting, then revise
   Undisclosed: fail, resit Meeting, then resit Not graded, Board of Examiners notified

   On privacy: Pangram does not train on submitted text and does not share or sell it. Text is encrypted, and the service is SOC 2 Type 2 certified (see Pangram's data privacy page). I remove names before uploading. I consider this use consistent with the GDPR and the Faculty guidelines.

4. 4

   I will check your sources

   You are responsible for checking all of your sources. I will use an AI-based research skill to ensure that your reference list and in-text citations are accurate and that there are no fabricated or hallucinated sources. If I find that you cited sources that do not exist, I will consider this research fraud.

   I will, of course, do a careful human check of any problems that are flagged. At my discretion, I will also consider scheduling a meeting with you to discuss any issues found. Otherwise, you should submit a manuscript in which the sources you use are confirmed and accurate. The same logic applies to a quotation, page reference, or other evidence attributed to a source.

---

Disclosure statement

Put a statement like this in your thesis, right before the bibliography. If you used no AI tools, say so; the statement is still required.

Checklist before you submit:

• All references exist and have been checked against the original source.
• All quotations and paraphrases accurately reflect the source.
• My AI-use statement lists every AI tool I used, including any used for grammar or translation.
• I have saved all prompts and outputs from these tools.
• I can explain and defend every major choice and claim in my manuscript, in my own words and without notes.

---

Use AI boldly

Do not let the reasons above put you off AI. Within the rules, I want you to think creatively, even aggressively, about how to use it, and I am excited to see what you do with it, even in strange ways.

When you use AI, work out which of the three modes below you are in. That helps you decide whether to use it at all, and how to use it well if you do.

Mode	Use it when	Examples
Automate	The work is routine and you can automate it guilt-free.	Formatting references, converting files, renaming and sorting sources, cleaning and formatting a spreadsheet or corpus.
Protect	The work builds core skills you should not offload.	Reading your sources, forming the core argument, writing the analysis.
Collaborate	The machine can augment your work.	Finding literature, checking a claim against a source, debugging code, adversarial reviews of analysis.

Here are some ideas I would love to see students try in a paper or thesis.

• Build a knowledge base. Make all your sources available as searchable text files with stable names, then have an agent search and cite them, and even cross-check them. You still need to check its work. See Building a Corpus for the design.
• Self-check. Before submitting, have an agent check each claim in your draft against your sources, then confirm that each reference can be found in a bibliographic index. See the procedures in the Open Science Skills library.
• Scale up. An AI-assisted thesis can work with thousands of documents instead of a dozen and run classifications that would be impossible by hand, which leaves more time for interpretation. See the Methods guide.
• Devil’s advocate. Have an AI model look for evidence against your argument or pick apart your analysis. Then respond to the critique in your own words.
• Something I haven’t thought of. If you can explain what you did and argue that it was an appropriate use of AI, I will likely support it.

I am developing tools, starter prompts, and demonstrations of these ideas at AI for Research. Let me know what you would like to see there.

---

Sources

Full references for studies cited on this page.

1. Autor, David, Tanya Rodchenko, Josh Martin, Zanna Iscenko, Scott Strand, David Pearl, and Melissa Ferere. 2026. "Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting." NBER Working Paper 35720. Working paper, not peer reviewed. nber.org/papers/w35720
2. Bassner, Patrick, Ben Lenk-Ostendorf, Ramona Beinstingel, Tobias Wasner, and Stephan Krusche. 2026. "Less Stress, Better Scores, Same Learning. The Dissociation of Performance and Learning in AI-Supported Programming Education." Computers and Education, Artificial Intelligence 10, 100537. DOI 10.1016/j.caeai.2025.100537
3. Bastani, Hamsa, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, and Rei Mariman. 2025. "Generative AI Without Guardrails Can Harm Learning. Evidence from High School Mathematics." Proceedings of the National Academy of Sciences 122 (26), e2422633122. DOI 10.1073/pnas.2422633122
4. Becker, Joel, Nate Rush, Elizabeth Barnes, and David Rein. 2025. "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." METR. Preprint. arXiv 2507.09089
5. Brynjolfsson, Erik, Danielle Li, and Lindsey Raymond. 2025. "Generative AI at Work." Quarterly Journal of Economics 140 (2), 889–942. DOI 10.1093/qje/qjae044
6. Dathathri, Sumanth, et al. 2024. "Scalable Watermarking for Identifying Large Language Model Outputs." Nature 634, 818–823. DOI 10.1038/s41586-024-08025-4
7. Dell'Acqua, Fabrizio, Edward McFowland III, Ethan Mollick, Hila Lifshitz, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R. Lakhani. 2026. "Navigating the Jagged Technological Frontier. Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality." Organization Science, articles in advance. DOI 10.1287/orsc.2025.21838
8. Deng, Ruiqi, et al. 2025. "Does ChatGPT Enhance Student Learning? A Systematic Review and Meta-Analysis of Experimental Studies." Computers & Education 227, 105224. DOI 10.1016/j.compedu.2024.105224
9. Eleftheriou, Pallis, and Constantinides. 2026. "Confidence Without Competence in AI-Assisted Knowledge Work." Preprint. arXiv 2604.09444
10. Fan, Yizhou, et al. 2025. "Beware of Metacognitive Laziness. Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance." British Journal of Educational Technology 56 (2), 489–530. DOI 10.1111/bjet.13544
11. Glickenhaus, Bradley, Khanh Thai, Jonathan Russell, et al. 2026. "Pangram 4 Technical Report." Vendor technical report. arXiv 2607.27183
12. Jabarian, Brian, and Alex Imas. 2025. "Artificial Writing and Automated Detection." NBER Working Paper 34223. Working paper, not peer reviewed. nber.org/papers/w34223
13. Kwa, Thomas, Ben West, Joel Becker, et al. 2025. "Measuring AI Ability to Complete Long Tasks." METR. Preprint. arXiv 2503.14499
14. Lee, Hao-Ping (Hank), Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. 2025. "The Impact of Generative AI on Critical Thinking. Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." In Proceedings of CHI '25, 1–22. ACM. DOI 10.1145/3706598.3713778
15. Li, C., H. Cui, and L. S. Hagedorn. 2026. "The Cognitive Impact of ChatGPT in Higher Education. A Systematic Review of Critical and Creative Thinking Outcomes." Computers and Education, Artificial Intelligence 10, 100571. DOI 10.1016/j.caeai.2026.100571
16. Melumad, Shiri, and Jin Ho Yun. 2025. "Experimental Evidence of the Effects of Large Language Models versus Web Search on Depth of Learning." PNAS Nexus 4 (10), pgaf316. DOI 10.1093/pnasnexus/pgaf316
17. Noy, Shakked, and Whitney Zhang. 2023. "Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence." Science 381 (6654), 187–192. DOI 10.1126/science.adh2586
18. Van Vlasselaer, M., F. Van Droogenbroeck, and B. Spruyt. 2026. "Who Wrote This? Evaluating the Reliability of AI Detection Tools in Higher Education." International Journal for Educational Integrity 22 (1). DOI 10.1007/s40979-026-00226-w
19. Wu, Xiaoyan, Ping Zhu, Jing Zhang, Min Yin, and Yan Wang. 2026. "ChatGPT's Impact on Student Learning Outcomes. A Meta-Analysis of 35 Experimental Studies." Humanities and Social Sciences Communications 13, 684. DOI 10.1057/s41599-026-07019-z

A note on what is not cited. A widely circulated 2025 meta-analysis of ChatGPT and learning was retracted in April 2026 and is deliberately absent. Claims that “AI has solved a Millennium Prize problem” refer to an unrefereed September 2026 result on a narrower variant of the Navier–Stokes problem and are not relied on here.