-
Notifications
You must be signed in to change notification settings - Fork 392
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Linear Set
performance with respect to number of items already in the Cache
#85
Comments
The CPU profile shows that |
Moving forward, is there a way to reduce the |
@thesilentg |
Thanks for the help. Moving to parallel inserts was able to shorten the overall duration. Although this is an improvement, it doesn't appear to change the asymptotic linear latency when calling |
It requires large code modification. |
That makes sense. Would it be ok if I updated the README documentation in a separate PR to explain the write limitations here and link back to this issues? That way new users would be aware that concurrent inserts are helpful when inserting large numbers of keys and that write latency will increase as more items are added. |
@thesilentg |
When switching out a
map[string]string
withfreeCache
in order to reduce garbage collection overhead, I noticed some strange behaviors:freeCache
is about 2x faster forSet
than using amap
, I was actually seeing roughly 2x slowerSet
performance on my benchmarks when compared to the defaultmap
implementationmap
implementation.On (1), I realized that benchmarks here don't preallocate the
map
with any capacity. In my local testing, I saw that preallocating themap
to the expected size increased benchmark performance by roughly2x
. Given that you must preallocate thefreeCache
to a desired size, it feels like the most accurate comparison would attempt to do something similar for themap
. Otherwise, you're basically just testing themap
's performance when dynamically resizing when new data is added, not really how fast it is at adding in new data to an already allocatedmap
.On (2), I did some more performance testing. Code below:
This creates a 12GB
freeCache
, attempts to write 100 million items into it, and measures the time to write each 100 thousand items. It also does the same for the defaultmap
. This produces the following graph:It appears that
freeCache
has roughly linear latency for calls toSet
with respect to the number of items already present in the cache. It certainly explains the benchmark behavior. By default, the benchmarking tests here will never write enough items into the cache to show this latency regression.This will bias the benchmark comparison between
map
andfreeCache
. It is only once ~2 million items have been written into thefreeCache
that the differences between the built inmap
andfreeCache
become apparent. In addition, you can see that writing the first ~1 million items into themap
has very high latency. My guess is that this is almost entirely due to Garbage collection overhead, as until themap
has a lot of items present, the default setting forGCPercent
will force very frequent garbage collection cycles during the benchmark process, as the ratio of live memory to garbage generated is quite high. Once themap
gets more and more populated and begins increasing the size of the live memory, the frequency of garbage collection will decrease quite significantly.I have two open questions:
Set
latency when more items are added?Set
expected? If so, can this be documented in the readme? If not, are there any potential workarounds to have constant (or at least sub-linear)Set
latency with respect to the cache size?The text was updated successfully, but these errors were encountered: