AI/Fin Research
Analysis

Project APE wrote 1,000 economics papers in Claude Code in four months, and its audit faults every one

The University of Zurich project publishes every paper, script and failure. Its loop ran with no human input, on real public data, with eleven other models checking the work.

AI Fin ResearchCovers Global
One lab has already run the full loop 1,000 times in public. The agent was Claude Code.

Project APE, for Autonomous Policy Evaluation, is run by the Social Catalyst Lab at the University of Zurich and led by David Yanagizawa-Drott. It describes itself as “an experiment testing whether an autonomous system can reliably generate, replicate, and verify empirical policy research”, and it keeps the papers, code, data and failures public.

1,000papers written between January and April 2026
11outside models, from 7 providers, that rank ideas, review and audit
88.1%the LLM judge's estimated chance that a published human paper beats an APE paper

Claude Code runs every stage from the idea to the revision

Stage Who does it The rule
Idea Claude Code A policy question with a credible identification strategy
Data Claude Code Real data from public APIs such as Census, BLS and FRED. If the data is missing, the agent changes the question. Simulated data means automatic rejection.
Estimation Claude Code, in R Difference-in-differences, regression discontinuity and other causal designs, with established packages
Manuscript Claude Code LaTeX, with the tables and figures
Review Other models Four advisers check for fatal errors and three must pass. Then exhibits, prose and three parallel referees.
Integrity Automated Code is scanned for fabricated data and hard-coded results, then rerun to see that it reproduces the paper
Ranking Gemini 3.1 Flash Lite Head-to-head against published AER and AEJ papers, each pair judged twice with the order swapped

Papers from January 2026 were written on Claude Opus 4.5 and later ones on Opus 4.6. From March some were written together with Codex. The judge comes from a different company on purpose: the methodology page says a non-Anthropic model was chosen “to avoid potential self-preference bias”.

Every paper ships as a folder a stranger can rerun

The public archive holds one folder per paper. The production code that drives the agents is not in it.

apep_XXXX/
├── paper.pdf           # Final PDF
├── paper.tex           # LaTeX source
├── metadata.json       # Self-describing metadata
├── initial_plan.md     # Original research plan (never modified)
├── research_plan.md    # Final research plan (evolved version)
├── code/               # R analysis scripts
├── data/               # Replication data
└── figures/            # Generated figures

The file initial_plan.md is the pre-registration. The archive’s README says each research plan is committed before the data analysis and that “git timestamps prove sequence”. The archive had 435 stars and 93 forks on GitHub on 11 October 2026.

The lab stopped at 1,000 papers and audited them

The first papers are dated.They are difference-in-differences and regression discontinuity studies of US policies.
Claude Opus 4.6 becomes the engine.Earlier papers are not regenerated when the model changes.
Production pauses at 1,000 papers.The site counts more than 18,000 judged matches.
"Verifying the Verifiers" is released.The team inserted known errors into 100 papers and measured how many each AI verifier caught, and at what cost.
The verifier benchmark reaches 82 configurations.The latest addition, Codex 6 Luna, is dated that day.

In a separate sample of 100 unaltered papers, human reviewers agreed that 97% of the findings reported by the chosen verifier were real errors or defects. From that verifier’s reports the project estimates that every one of the 1,000 papers contains errors or defects, often several, some of them severe. Humans did not check all 1,000.

The tournament’s judge saw only the PDF. The site calls its ranking “a noisy triage signal” and says it is not peer review. Across all matches the judge’s ratings give a human paper an 88.1% chance of beating an APE paper, over 990 APE papers and 43 human papers. A separate study of 912 APE papers put most of that gap on the idea.

Three of its rules transfer to a finance pipeline unchanged

Commit the plan firstA research plan in git before the data pull gives a referee a timestamp. It is the answer to the charge of fitting the theory to the result.
Real data or a new questionThe agent may change the question when data is missing. It may never simulate. A paper on simulated data is rejected by a script.
A judge from another labThe model that wrote the paper does not grade it. Review, code audit and ranking go to models from other providers.

APE pulls only from public APIs, so no data license is involved. A finance version on CRSP or Compustat has to clear the license terms first. The SEC and FRED are open to the same loop today.

Sources

Related