Recognising Grains (Dry Bean Varieties)

Task 1: Recognising Bean Varieties with Explainable Tree Models

English translation by SOTA – AI Community of the Slovak original, Uloha1_rozpoznavanie_zrn.pdf ("Úloha 1: Rozpoznávanie odrôd fazule pomocou vysvetliteľných stromových modelov"), published among the national-round tasks of the Olympiáda v umelej inteligencii (Slovak Olympiad in Artificial Intelligence), 2025/26. The original task belongs to its authors. Organisers who would like this translation removed can email [email protected].

In agriculture, in the food industry and in the automated sorting of crops, it is important to be able to distinguish different kinds of seeds quickly and reliably. In practice, image-based or geometric characteristics of the objects, obtained from cameras and sensors, are often used for this. In such tasks, however, it is not enough merely to achieve high accuracy. It is also important that the model can explain its decision: which properties were decisive, which rules led to the classification, and where the model makes mistakes.

[Figure: see the original statement.]

In this task you will work with the real Dry Bean dataset from the UCI Machine Learning Repository. The dataset contains 13 611 samples of seven varieties of dry beans. For each bean, 16 numerical features are available, derived from image analysis of the shape and size of the seeds. The dataset was created from images taken with a high-resolution camera and is freely available for download: https://archive.ics.uci.edu/dataset/602/dry%2Bbean%2Bdataset

Your task will be to create and analyse a system that:

• classifies the bean variety using decision trees,
• compares a single tree with groups of trees (ensemble trees),
• uses PCA to visualise and analyse the data,
• and explains as well as possible why the model decides the way it does.

Data preparation

• Download the dataset and load it into your program.
• Check whether the data contain missing or suspicious values.
• Split the data into a training part and a test part, for example in the ratio 80:20.
• Use a stratified split, so that the class proportions remain similar in both parts.
• If appropriate, also prepare a standardised version of the data for PCA.

Data exploration and PCA

• Find out which features (attributes) are strongly related to one another.
• Use the PCA method to reduce the dimension of the data to at least 2 principal components.
• Create a 2D plot of the samples after projection into the PCA space.
• Comment on:
  • which classes are well separated,
  • which classes overlap,
  • how much of the variability the first two or three principal components explain.

Training a decision tree

• Train a classification Decision Tree model.
• Try at least two settings:
  • a simple, shallow tree,
  • a deeper or less constrained tree.
• Compare, for example, the effect of the parameters:
  • max_depth,
  • min_samples_leaf,
  • criterion (gini vs. entropy).
• Evaluate the model using the metrics:
  • accuracy,
  • macro F1-score,
  • confusion matrix.
• Visualise the final tree or a part of it.

Interpreting the tree

Extract as much information as possible from the decision tree. Find at least 5 interesting rules, for example in the style:
If Area > ... and ShapeFactor1 < ..., then it is probably the variety ...

Choose at least 3 misclassified samples and explain:

• which path the tree took,
• at which decision point the model "went wrong",
• which classes were similar to each other.

Compare:

• a simpler tree with lower accuracy but better interpretability,
• a more complex tree with higher accuracy but worse readability.

Groups of trees (ensemble)

Train at least two ensemble tree models, for example:

• Random Forest,
• Gradient Boosting,
• or Bagging with decision trees.

Compare them with a single decision tree in terms of:

• accuracy,
• macro F1-score,
• stability of the results,
• interpretability.

Comment on why groups of trees often achieve better results than a single tree, but are at the same time harder to explain.

PCA vs. tree models

Compare:

• models trained on the original features (original attributes),
• models trained on the PCA components (after applying PCA).

Evaluate:

• whether PCA improved or worsened the results,
• whether the visualisation improved after PCA while the interpretability worsened,
• whether tree models need PCA at all.

An explainable model of bean grains

Imagine that a company wants to build a simple device that can measure only a few properties of a bean, not all 16.
Propose a procedure:

• select only the 3 to 5 most important features,
• train a new tree on them,
• compare its result with the model that uses all the features.

Finally, answer:

• What is the minimum number of measurements that still suffices for reasonable sorting?
• By how much does the accuracy drop?
• Is a simpler and cheaper model worth it?

What to submit

1. The solution code in the form of .py or .ipynb.
2. Brief documentation or comments in the code.
3. Model outputs and plots:
   • the PCA visualisation,
   • the confusion matrix,
   • a comparison of the models' metrics,
   • a visualisation of at least one decision tree.
4. A short written commentary on the results, in which you explain:
   • what was most important for the classification,
   • which classes were confused with each other,
   • what the difference was between a single tree and the groups of trees,
   • what PCA showed.

Scoring

0 – 100 points
The assessment takes into account:

• 30 points – correctness of the data preparation and of the implementation,
• 20 points – quality of the modelling with decision trees,
• 15 points – comparison with ensemble tree models,
• 15 points – meaningful use of PCA,
• 20 points – interpretation of the results, commentary, originality and clarity.