Rife V2 pytorch?

[NevermindNilas]

Hi, do you guys by any chance store a pytorch version of the onnx models for rife V2?

Otherwise onnx2torch would be a workaround I guess?

[WolframRhodium]

What's the purpose of the PyTorch version?

[NevermindNilas]

I would like to test it with my project TAS. I am not quite literate in onnxruntime since as far as my understanding goes the inputs must be a numpy array which hinders the performance compared to Pytorch Cuda.

[WolframRhodium]

You can use the RIFE PyTorch code from the official repository.

For onnxruntime, with IOBinding it is possible to pass pytorch cuda tensor into it without data transfer.

[NevermindNilas]

I think the RIFE Pytorch code uses the official 'v1' models. I was interested to find out more about the V2 models and their performances ( quality and speed ) relative to the 'official' models.

🤔 Interesting. Would the IOBinding be persistent or would it need to be reinitialized for every inference ( every frame of a video ) for video usecases.

[WolframRhodium]

v2 simply eliminates some unnecessary data transfers in v1, but with custom implementation you can do better than v2 by transferring only the frame data while caching the metadata like time points.

IOBinding can be either persistent or dynamic, I expect the creation overhead is low.

[NevermindNilas]

Oh Interesting, well I will try out onnxruntime since I was extremely curious about directml for some time now.

Thanks! :)