Checklist IOAI 2025 At-Home Round · Task 3
Chameleon
Build a guesser that ranks ten candidate words for a secret, given an ordered sequence of icon hints from a clue-giver.
The task
Chameleon is a word-guessing game in which a clue-giver hints at a secret word or phrase by pointing at an ordered sequence of icons, each with a text description. The first icon usually carries the core idea and later ones add context, so order changes the meaning.
You write the guesser: a function guess_words(hint, choices) that takes a list of icon IDs and a list of candidate answers guaranteed to include the secret, and returns 10 guesses ranked from most to least likely.
The model you use must have fewer than 1 billion parameters and cannot call external model APIs at inference time, although external pretrained models may be used during development. The provided baseline fine-tunes a small sentence-embedding model.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- Icon descriptions (with icon images) and a take-home validation set of hints, options and labels, as Hugging Face datasets; the repo also holds training, validation and test sets in Hugging Face datasets format.
- You submit
- A
guess_words(hint, choices) function returning 10 ranked guesses; the baseline submitssubmission.zipcontainingsubmission_model.pyand the fine-tuned model folder. - Scoring
- Final Score = 0.9 × Hits@10 + 0.1 × NDCG@10, where Hits@10 is 1 if the secret is among the 10 guesses and NDCG@10 = 1/log2(i + 1) for the secret at rank i.
- Rules
- Model must have fewer than 1 billion parameters (about 4 GB of memory)
- No external model APIs at inference time; the submitted solution must be self-contained
- Format
- At-home round (non-graded practice, 1-30 July 2025)