Unified search layer

[op-hunter]

The function searchBaseLayer and searchBaseLayerST has almost the same code, this pull request unified the two functions.

[op-hunter]

@yurymalkov Can you help me with this pr? Error: "ImportError: cannot import name 'NoReturn'" occurs.

[yurymalkov]

Thanks for the PR! It seems like a python 3.6 error. I can look into it.

From the point of the PR, I wonder if you've benchmarked the performance of the single-threaded version (e.g. check that performance is the same/better)?
The main intention to separate them is to avoid locks/branching in the search code (which isn't really need them unless there are updates at the same time).

[op-hunter]

I did a simple benchmark on my machine with 32G memory.
Index parametes: efConstruction = 200, M = 48, efSearch = 128. nq = 10000
Dataset: sift-1m, dimension = 128.
It seems that there is no much difference.
Test with multithread:

	branch master	unified search layer
build cost	293890	295734
query cost	2140	2053

Test with single thread:

	branch master	unified search layer
build cost	1854470	1849888
query cost	12939	12951

Note that the unit of time is millisecond.

[yurymalkov]

Thank you for the test! I usually test it with low dim (say d=4), so to make evident the bottlenecks outside of the distance function computation.
I'll do my typical tests for the change the following week and will get back to you.

[yurymalkov]

Hey @op-hunter ,
I've finally checked the performance for d=4 and it seems like there is a slowdown by more than 10% in case of large ef values (e.g. searching 1M vectors is 26.5k/s in single thread for ef=128 and only 20.8 k/s for the proposed change).

I think this can be solved by using templates. I wonder if there any way to make conditional lock creation?

[yurymalkov]

Maybe even pointers are fine. Not sure if there is a clean alternative.

[op-hunter]

@yurymalkov How about use std::defer_lock to delay the lock operation?
line 198:

std::unique_lock<std::mutex> lk(link_list_locks_[current_node_id], std::defer_lock);
if (!is_st) lk.lock();

[yurymalkov]

Hm. I can check that.