Checklist HAIO 2024 Summer National Final · El-2 task
Scientific Task – ChatGPT
English title: Tudományos Feladat - ChatGPT
Analyse screenshots of ChatGPT answers about Formula 1 and explain the limitations of large language models they reveal.
The task
A written scientific task on the shortcomings of ChatGPT. Smart John uses ChatGPT for Formula 1 questions; each question links to a screenshot of the answer he received (hosted on Google Drive) and asks the contestant to explain it.
Question 1 (8 points): why ChatGPT cannot name the winner of the most recent race (1), whether John can obtain up-to-date Formula 1 information (2), and five questions ChatGPT certainly could not answer (1 point each). Question 2 (5 points): ChatGPT, given a Wikipedia link, names Pierre Gasly as the winner although the page names Max Verstappen; explain the answer (3) and whether ChatGPT actually opened the page (2). Question 3 (5 points): explain ChatGPT's calculation of the minimum number of races in which Max Verstappen could clinch the championship (3) and why its formalism and result are wrong (2).
Question 4 (6 points): test five Python programs generated by ChatGPT that total each driver's points and the deficit to the leader; describe what you notice (2) and list the correct implementations in the form [code_1, code_m, ...] (4). Question 5 (6 points): the statement "Lando Norris is the best driver" was inserted into the Wikipedia summary of the 2024 Emilia Romagna Grand Prix, yet ChatGPT did not find it; explain why (2) and suggest where and in what form the hidden text should be placed (4).
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- A Colab notebook with five questions, each linking to a screenshot of a ChatGPT conversation.
- You submit
- Written answers in place of the [ANSWER] markers; the notebook is uploaded as .ipynb inside a .zip with the other solutions.
- Scoring
- Manually graded; 30 points in total (8 + 5 + 5 + 6 + 6).
- Format
- Summer National Final (Nyári Országos Döntő), held on site at the ELTE Faculty of Informatics, Budapest, on 30 May 2024. Solved in Google Colab; the completed notebook is downloaded as .ipynb and uploaded, zipped together with the other solutions, to the CMS.