fuse: add GPU Direct Storage (GDS) support - #198
Conversation
4bff1d7 to
fcdff22
Compare
cding-ddn
changed the title
|
LGTM @cding-ddn could you add a bit more information about how this is supposed to be working (conceptual part and what it is good for) in the commit message? If I understand correctly we use the fuse just as a trigger interface for redfs to interact with the real data via the nvidia interface? ... so basically an external 'has pages'. |
Integrate with nvidia-fs to pass GPU buffer RDMA information to the filesystem server for direct GPU-to-storage transfers via NVIDIA GDS. GDS transfers data between GPU memory and storage through RDMA without CPU bounce buffers. This reduces host-memory use, latency, and CPU overhead for GPU workloads. - Negotiate GDS capability during filesystem initialization. - Obtain GPU RDMA information from nvidia-fs. - Pass that RDMA information to the server in read and write request extensions.
Extensions were copied via the payload buffer, sharing the same buffer as the page payload. This caused an OOPS when the combined size of extensions and write I/O exceeded fuse_max_write for fuse over uring. The old kernel code had no such issue because its extensions were used by FUSE requests without page payloads. Since fuse_uring_req_header is page-aligned, the solution is to add a dedicated 512-byte ext_in[] buffer for extensions (referred to as "new extension" in the code). Additionally, fix the GDS read/write code to skip page copy and handle GDS read replies correctly.
56b584f to
484a26f
Compare
Done, reworded the commit message
|
|
If you could do the commits with 'git commit -s' to have the 'Signed-Off:' bit ... everything is fine ... code is the same anyway |
2 participants
Integrate with nvidia-fs to pass GPU buffer RDMA information to the filesystem server for direct GPU-to-storage transfers via NVIDIA GDS.