Transfer Learning

The task investigates when and how successfully neural networks transfer knowledge between tasks. Two pretrained ResNet-20 models (CIFAR-10 and CIFAR-100, loaded from torch.hub chenyaofo/pytorch-cifar-models) are provided, together with CIFAR-10 and CIFAR-100 loaders and a fixed random seed (2024).

Task 1: report the test accuracy of both models; replace the last layer of the CIFAR-100 model with that of the CIFAR-10 model and report its CIFAR-10 training and test accuracy, compared with a randomly initialised network given the same output layer (four numbers in total). Task 2: replace only the output layer with a randomly initialised one, freeze the rest, fine-tune on CIFAR-10 and plot CIFAR-10 and CIFAR-100 test accuracy against fine-tuning epochs. Task 3: compute the pairwise L2 distances between the parameters of the random model, the Task 1 model and the Task 2 model as a symmetric 3×3 matrix.

Task 4: also fine-tune the layers up to and including the second residual block and report whether CIFAR-10 test accuracy improves; compare with training only the layers after the second residual block. Task 5: using only 50% of the CIFAR-10 training data, try to match the accuracy of the model trained on the full CIFAR-10 by transfer learning from CIFAR-100, and repeat with 20%.