# Sentiment Update

*English translation by SOTA – AI Community of the Hungarian original. Licensed CC BY-NC-SA 4.0, like the original. Organisers who would like this translation removed can email sota.ai.community@gmail.com.*

**Magyar MI Diákolimpia** (Hungarian AI Olympiad)\
**Task description · Summer Camp Selection (Nyári Tábor Válogató)**\
June 2026

*[Figure: see the original one-pager.]*

## 1. Sentiment Update

You are in charge of a sentiment recognition system that processes tens of thousands of messages day after day: customer service feedback, comments on educational platforms, community conversations. Until now the system could distinguish three moods – positive, negative and neutral – and for a long time this was enough. User needs, however, have outgrown it: from now on, the services want to know whether a message is merely dissatisfied or already outright angry, politely appreciative or sincerely enthusiastic.

Your task is to extend the system to **five sentiment levels**, from very negative to very positive, and to assign the correct level to every text in the test set:

- **0**: very negative
- **1**: negative
- **2**: neutral / mixed
- **3**: positive
- **4**: very positive

The extreme cases are easy: bursting enthusiasm and undisguised rejection practically shout out of the text. The real difficulty lies in the in-between tones, where the same word is sincere praise in one sentence and sarcasm in another, and lukewarm disappointment easily blurs into neutrality. For these, it is not enough to pick out a keyword or two: you have to understand the tone of the whole message, and your score depends precisely on how well you cope with these subtle transitions.

## 2. What you receive

Two CSV files. `train.csv` is the labelled training set (`id`, `text`, `label`; `label` is 0–4, **750 rows**, balanced across the classes), and `test_public.csv` is the input of the test set (`id`, `text`, without labels, **275 rows**). You must give a prediction for every row of the latter. Any additional training data (even your own) may be used.

## 3. What you submit

A single CSV file with the header `id,target`, containing all **275** rows of the test set. `id` is the identifier from `test_public.csv`, and `target` is the predicted sentiment class (0–4). The set of submitted `id` values must cover the test set exactly. You do not need to submit a model; the evaluator computes the score from the CSV.

## 4. Scoring

Evaluation is based on the **macro-averaged F1** (`f1_macro`): the arithmetic mean of the F1 scores of the five classes, so all five classes count equally, and correctly recognising the rarer sentiments is just as important as recognising the more common ones.

$$
\mathrm{F1}_c = \frac{2\, P_c\, R_c}{P_c + R_c}, \qquad \text{macro-F1} = \frac{1}{5} \sum_{c=0}^{4} \mathrm{F1}_c,
$$

where $P_c$ and $R_c$ are the precision and recall of class $c$. The task is worth **40 points**. The raw macro-F1 is mapped to points by a **calibration curve**: **below 0.85, 0 points**; **above 0.965, the full 40 points**; linear in between:

$$
\text{Points} = 40 \cdot \operatorname{clip}\!\left( \frac{\text{macro-F1} - 0.85}{0.965 - 0.85},\ 0,\ 1 \right).
$$

The curve is needed because on this task the raw macro-F1 is compressed (most sensible solutions fall between 0.84 and 0.96), so a linear scale would hardly spread out the field.

**Maximum score:** 40 points.

**Public and private test set.** The test set is divided into a public part and a closed (private) part. During the contest you only see the score measured on the public part on the leaderboard; the final ranking is decided by the closed part.

## 5. Constraints and technical information

1. Scoring is carried out automatically, on the server side, by the **DOCK** platform on the basis of the submitted `submission.csv`; you do not need to submit a model.
2. You have **23 upload attempts** in total; **the score of your best submission counts** towards the final result.
3. `test_public.csv` may be used only for prediction; reverse-engineering its labels or labelling it by hand is **not allowed**.
4. You must use the provided `Qwen/Qwen3-Embedding-4B` model; using any other pretrained model is **not allowed**.
5. The data may be used only for the contest and for educational and research purposes; uploading the dataset or any part of it to an external drive, or using it outside the task, is **not allowed**.

## 6. Useful resources

- [Hugging Face Transformers documentation](https://huggingface.co/docs/transformers/index)
- [PyTorch documentation](https://pytorch.org/docs/stable/index.html)
- [pandas documentation](https://pandas.pydata.org/docs/)
- [macro-F1 (scikit-learn)](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html)