Checklist NEOAI 2026 Day 2 · Task 5
Creatures of the Static
Train a 5-class image classifier on 50 clean images and apply it to 5,000 test images distorted by an unknown domain shift, without pretrained models.
The task
The training catalogue contains 50 clean RGB images of 32×32 pixels, exactly 10 per class for 5 species. The 5,000 test images (1,000 per class) show the same creatures but "from another dimension": they have passed through a medium that distorts them in an undisclosed way, and discovering how the test distribution differs is part of the task.
For every test image the contestant predicts one of the class labels 0–4. The public leaderboard uses a stratified 50% of the test set (2,500 images) and the private leaderboard the other 2,500.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- train/ and
train.csv(id, label ∈ {0,…,4}); test/ (00000.png…04999.png) andtest.csv(id);sample_submission.csv. - You submit
- CSV with columns id, label; exactly 5,000 rows; rows are merged by id, so order does not matter.
- Scoring
- Micro-F1 = TP_total / (TP_total + 0.5·(FP_total + FN_total)), equivalent to accuracy for this balanced task.
- Rules
- Pretrained models must not be used ("DO NOT USE PRETRAINED MODELS").
- General NEOAI 2026 rules apply.
- Format
- NEOAI 2026 main round, Day 2 (4 May 2026). Individual.