mbert-study

CROSS-LINGUAL ABILITY OF MULTILINGUAL BERT: AN EMPIRICAL STUDY

View the Project on GitHub CogComp/mbert-study

Pre-training BERT

Preparation

When creating a google cloud instance, make sure full api access is turned on.
Correctly set GC_BUCKET_NAME in init-gcloud-server.sh to your cloud storage bucket name.
In a google cloud instance, run init-gcloud-server.sh and run.sh will be created.
Executing run.sh and passing it a tpu name initiates bert training.