bfloat16 (bf16) support in faiss

[mlomeli1]

Many LLMs are trained with bf16, if we want to use the hidden states of LLMs for retrieval, those vectors will be in bf16 dtype. It would be helpful to support bf16 in Faiss so that we can use LLMs as retriever or embedding model.

[mdouze]

Note that we cannot pass through the numpy wrapper because numpy does not support bf16.
Adapting gpu_knn code for pytorch should be easy

faiss/contrib/torch_utils.py

Line 497 in dc55e11

def torch_replacement_knn_gpu(res, xq, xb, k, D=None, I=None, metric=faiss.METRIC_L2, device=-1, use_raft=False):

[mdouze]

• add bfloat16 support here

https://github.com/facebookresearch/faiss/blob/main/faiss/gpu/GpuDistance.h#L76