question on cutoff_len

[BenoitWang]

Hi, first thanks for this awesome work. I'm trying to rewrite the training code for ltu-as while I find that the cutoff_len for stage 1 and 2 is 108 which is rather small. There are still many samples that exceed this length and in this case the outputs are cut off. Is this because of a memory issue please?

[YuanGongND]

Yes, it is mainly due to computational efficiency consideration.

More specifically, we see a chance to fit a batch size, say 4, to our GPU, and then with this batch size, we find the max seq length, which is something around 108.

The model will see the same data with larger cutoff_len in later stages because we still use these data in later stages.

-Yuan