Dynamic batching and dynamic padding

[FlorentRamb]

Hello,

I'd like to deploy a Transformers model and use the dynamic batching feature. However the inputs shape may vary from one request to another (different sequences length).

Is there a way to perform dynamic padding (pad to the longest sequence length) within the dynamic batch created by Triton? What would be the best approach?

Thanks in advance!