Discord

Checklist Bulgaria selection 2025 IOAI Team Selection · Task 2

CyrillicCross: Recovering Visual Features Across Languages

Bulgarian title: CyrillicCross: Възстановяване на визуални характеристики между различни езици

Train a PyTorch projector that maps corrupted Bulgarian RoBERTa token embeddings to the CLIP text-embedding sequence of the English prompt.

  • NLP
  • Multimodal
  • Embedding translation / representation reconstruction
  • Bulgarian original · English translation
Download the English notebook Open the task notebook (Bulgarian)

The task

In the story, a Bulgarian NLI encoder (RoBERTa-base) turns Bulgarian captions into embeddings used to condition Stable Diffusion. Because of a bug, training-time embeddings had dropout left on (half the elements zeroed) and the tokens circularly shifted; at inference time the embeddings are clean, but only the corrupted ones are available for training.

Contestants implement a single PyTorch nn.Module projector that takes corrupted RoBERTa vectors [batch_size, seq_len, hidden_size], undoes or becomes robust to the corruption, and projects each token into the CLIP embedding space, outputting a fixed-size tensor [batch_size, 77, clip_dim]. The goal is to recover the CLIP embeddings of the equivalent English prompt.

Abridged and translated by SOTA from the official Bulgarian materials. The official statement has the exact rules, and it wins wherever this summary differs.

In English

This task was published in Bulgarian. SOTA translated its 4 files into English on 17 September 2026. Only the words changed in the notebooks: markdown, code comments, messages and printed output. The code, file names and paths are the original's, so a translated notebook runs with the original data.

Read the task notebook in English 503 words and 12 code cells 
%pip install gdown matplotlib
!gdown "1_wv4XFBYoeynXyS8aUb5AH2Cr3fMNbNW"
!gdown "1p50Vn4PGaKlPdPQdooFT0W6GXsz2a0Se"
!gdown "1Civ9LJqf-2mFOqh7o2ozRlRZ9g1jtaWk"
!gdown "1kw27Jzot2aa4VyG7dr_XwzdB42xJsuBw"
%pip install -r requirements.txt

CyrillicCross: Recovering Visual Features Across Languages

image.png

Story A team of researchers was working on the “CyrillicCross” project when a software bug appeared during the training sessions. Your NLI (Natural Language Inference) encoder RoBERTa-base was being trained to convert these Bulgarian captions into embedding vectors, which would then be used to generate images with Stable Diffusion. During the training sessions, however, because of a software bug, dropout remained active — half of the elements of each sequence were randomly zeroed, and at the same time the tokens in the sequence were circularly shifted (every element is moved a certain number of positions to the right, and the elements that fall off the end return to the beginning). As a result, half-corrupted embedding vectors were generated during training.

In the test (inference) sessions, dropout and the circular shift are switched off and the embedding vectors are created correctly — these correct vectors may be used only for testing (when we are not training the projector), but while we train the projector on the reconstruction task, we have only the corrupted training embedding vectors.

Requirements

Within CyrillicCross, implement a single PyTorch nn.Module projector that restores the surviving embedding vectors:

  1. Input

    • RoBERTa vectors with dimensions [batch_size, seq_len, hidden_size], to which dropout and a circular shift have been applied.

  2. Projector module

    • Correcting the corruptions: Think of an architectural solution that reverses these corruptions - the dropout and the circular shift (or that adapts to them?).
    • Projecting the vectors: Project every reconstructed token into the CLIP embedding space of dimension clip_dim.
    • Output: Think about how to output a tensor of fixed size [batch_size, max_seq_length_clip, clip_dim]. We assume that max_seq_length_clip=77.

  3. Constraints

    • Do not use any libraries other than torch for the architecture!
    • Do not modify other files; add code only in the sections marked with #INSERT YOUR CODE HERE!
    • You are not allowed to change the loss functions or to add new ones. Only the training hyperparameters may be changed!
    • The training time must not exceed 20 minutes on an A100 GPU!

Goal

Recover the CLIP visual embeddings corresponding to a prompt in Bulgarian executed in English, thereby restoring the lost visual representations of the Bulgarian language in the global CLIP space. The test metrics will be, respectively, the reconstruction error Mean-Squared Error (MSE) and a Cosine Similarity based error (the MSE between the cosine similarity of every token with every other token in the prediction and, again, the cosine similarity of every token with every other token in the original sequence). The error is computed as MSECosine + MSE.

Submission

Submit the final solution as the notebook's .ipynb file with the EXECUTED CELLS, together with the CHECKPOINT created by the final cell!!!

import torch
from torch import nn, optim
from torch.utils.data import DataLoader
from transformers import CLIPTextModel, CLIPTokenizer, AutoModel, AutoTokenizer
from datasets import load_dataset
import torch.nn.functional as F
import matplotlib.pyplot as plt
%run ./tools.py
Example code for the circular shift and dropout

# Example embedding vector for demonstration
seq_len, hidden_size = 3, 5
x = torch.arange(seq_len * hidden_size, dtype=torch.float32).reshape(seq_len, hidden_size)

# Dropout mask and its application
mask = torch.tensor([[1, 0, 1, 1, 0],
                     [0, 1, 0, 1, 1],
                     [1, 1, 0, 0, 1]], dtype=torch.float32)
x_dropped = x * mask

# Circular shift
k = 2
x_shifted = torch.roll(x_dropped, shifts=k, dims=0)

# Visualisation
for data, title in [
    (x, "Original embedding vector"),
    (x_dropped, "After dropout (applying the mask)"),
    (x_shifted, f"After circular shift (k={k})")
]:
    plt.figure()
    plt.imshow(data.numpy(), aspect='auto')
    plt.colorbar()
    plt.title(title)
    plt.xlabel("hidden_size")
    plt.ylabel("seq_len")

plt.show()
Hyperparameters
BATCH_SIZE = 20
LR = 5e-5
WEIGHT_DECAY = 1e-4
EPOCHS = 20
DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model_id = "sd-legacy/stable-diffusion-v1-5"
Loading the data
ds = load_dataset(
    "json",
    data_files={
        "train": "./train.json",
        "validation": "./val.json"
    }
)

dataset_train = ds["train"]
dataset_val = ds["validation"]
Loading the models
clip_tokenizer = CLIPTokenizer.from_pretrained(model_id,
    subfolder="tokenizer",        # points to the tokenizer files
    use_fast=True  )
clip_text_encoder = CLIPTextModel.from_pretrained(model_id,
    subfolder="text_encoder" ).to(DEVICE)
clip_text_encoder.eval()
bg_tokenizer = AutoTokenizer.from_pretrained('rmihaylov/roberta-base-nli-stsb-theseus-bg')
bg_encoder = AutoModel.from_pretrained('rmihaylov/roberta-base-nli-stsb-theseus-bg').to(DEVICE)
bg_encoder.eval()
with torch.no_grad():
    dummy_bg = bg_tokenizer(['тест'], return_tensors='pt', padding=True)  # "тест" = test (Bulgarian input)
    bg_dim = bg_encoder(**{k: v.to(DEVICE) for k, v in dummy_bg.items()}).last_hidden_state.size(-1)
    dummy_en = clip_tokenizer(['test'], return_tensors='pt', padding=True)
    clip_dim = clip_text_encoder(**{k: v.to(DEVICE) for k, v in dummy_en.items()})[0].size(-1)
Projector architecture (this is where the solution is required)
class Projector(nn.Module):
    # INSERT YOUR CODE HERE
    def __init__(self, bg_dim, clip_dim=512, len_tokens=77):
        super().__init__()
        self.linear = nn.Linear(bg_dim, clip_dim)
        self.len_tokens = len_tokens

    def forward(self, hidden_states, attention_mask):
        last_slice = hidden_states[:, -1:, :]

        if self.len_tokens > hidden_states.size(1):
            tail = last_slice.repeat(1, self.len_tokens-hidden_states.size(1), 1)
            # Repeating the last token and concatenation
            hidden_states = torch.cat([hidden_states, tail], dim=1)
        return self.linear(hidden_states)
# Set radnom seed
random.seed(42)
np.random.seed(42)
torch.cuda.manual_seed(42)
torch.manual_seed(42)

proj = Projector(bg_dim, clip_dim).to(DEVICE)
opt = optim.AdamW(proj.parameters(), lr=LR, weight_decay=WEIGHT_DECAY)
train_loader = DataLoader(
    dataset_train,
    batch_size=BATCH_SIZE,
    shuffle=True,
    collate_fn=lambda batch: (
        bg_tokenizer([x['bg'].lower() for x in batch], padding=True, truncation=True, return_tensors='pt'),
        clip_tokenizer([x['en'].lower() for x in batch], padding=True, truncation=True, return_tensors='pt')
    )
)
val_loader = DataLoader(
    dataset_val,
    batch_size=BATCH_SIZE,
    shuffle=False,
    collate_fn=lambda batch: (
        bg_tokenizer([x['bg'].lower() for x in batch], padding=True, truncation=True, return_tensors='pt'),
        clip_tokenizer([x['en'].lower() for x in batch], padding=True, truncation=True, return_tensors='pt')
    )
)
Training
%%time
train(proj, bg_encoder, clip_text_encoder, train_loader, val_loader, opt, EPOCHS, DEVICE)
Validation
print(f"Validation: {validate(proj, val_loader, bg_encoder, clip_text_encoder, DEVICE)}")
Saving
proj.eval()
torch.save(proj.state_dict(), "./proj_state_dict.pth")

Translated by SOTA. The Bulgarian original is the official version and wins wherever the two differ. The notebooks need tools.py, train.json and val.json from the original ProblemStatement folder; the Bulgarian test string 'тест' in the code is kept and glossed. The three solution notebooks are unattributed and repeat the statement with small differences in wording. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.

At a glance

You get
train.json and val.json with caption pairs, the Bulgarian encoder rmihaylov/roberta-base-nli-stsb-theseus-bg and a CLIP text encoder loaded in the notebook, tools.py and requirements.txt.
You submit
The executed notebook together with the checkpoint saved by its final cell.
Scoring
Reconstruction MSE plus a cosine-similarity error (MSE between the token-to-token cosine-similarity matrices of the prediction and of the target); error = MSECosine + MSE.
Rules
  • Only torch for the architecture.
  • Add code only in the '#INSERT YOUR CODE HERE' sections; do not modify other files.
  • Loss functions may not be changed or added; only training hyperparameters may be changed.
  • Training time at most 20 minutes on an A100 GPU.
Format
Bulgarian IOAI 2025 team selection, Day 1, Task 2. Dates and format are not published in the repository.

In English (SOTA)

Notebook

Model

Solutions

Source files

Details

Year
2025
Round
IOAI Team Selection · Task 2
Language
Bulgarian; English translation by SOTA
License
Not stated by the source