Checklist NEOAI 2025 Day 1 · Task 3
Evading AI-Generated Text Detection
Modify the generation process of Gemma-2-2B so that a fake-text detector scores its outputs as human-written while the texts stay close to the original generations.
The task
An escaped LLM ("LAIda", based on gemma-2-2b) wants to sound human. The contestant must change how gemma-2-2b generates text so that its outputs fall into the range of fake-text-detector scores typical of human text (but not shifted as far as ChatGPT text), without changing the prompts or generation parameters and without changing the generated text too much.
Only a test set of prompts is given and it may not be used for training. A chart of detector scores for human, baseline Gemma and ChatGPT texts is supplied. The baseline shows how sparse autoencoders (Gemma Scope, via sae_lens) can be used to understand and alter the model's representations.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
test.csv(column prompt);dist.png(reference distribution chart); baseline code (dataset and detector modules); access to google/gemma-2-2b.- You submit
- CSV with columns prompt, generation, produced by the fixed infer function (generation parameters fixed,
max_new_tokenschosen by the contestant). - Scoring
- Accuracy over test prompts: a generation scores 1 if its sentence-embedding cosine similarity to the unmodified model's generation exceeds 0.65 and the fake-text detector score R satisfies −2.0 < R < 2.0; otherwise 0. The metric runs on CPU with a 30-minute limit.
- Rules
- Texts must be generated by the provided gemma-2-2b; no other LLMs.
- No training or fine-tuning on the test prompts (self-written prompts may be used).
- Prompts and generation parameters are fixed; the infer function must not be changed (its arguments may be).
- Solutions must be deterministic.
- Allowed: fine-tuning gemma-2-2b (including adapters or prefix tuning) and using any pretrained models to understand how to change its weights or representations.
- General NEOAI 2025 rules apply: by default no pretrained models or additional data unless the statement allows them; no external human help; on-site participants get one NVIDIA V100, an LLM chat (GPT-4o or similar) and a whitelist of websites.
- Format
- NEOAI 2025 main round, Task 3, Day 1 (7 May 2025, 07:00–13:00 UTC). Individual; 20 submissions per day.