Cross-Lingual Task Transfer Using Task Vectors

This project investigates the utility of task vectors (TVs) for cross lingual task transfer. Specifically the ability of TVs to encode syntactic vs semantic knowledge, and how linguistic similarity effects the success of transfer cross linguistically.

Multilingual transformers were trained on each target language and then the base transformer was trained to perform a task (NER, POS tagging, NLI, or Dependency Parsing). Success was evaluated across tasks and crosslinguistically. It was hypothesized that task vectors would perform better for semantic tasks than syntactic tasks, and would perform better on languages more similar to the source language.

This project is in progress.

Code for this project can be found here.