-
Notifications
You must be signed in to change notification settings - Fork 175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add VisitAllEntries method to iterate all entries in cache. #3
base: master
Are you sure you want to change the base?
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3 +/- ##
=========================================
+ Coverage 76.05% 77.2% +1.14%
=========================================
Files 4 4
Lines 497 522 +25
=========================================
+ Hits 378 403 +25
Misses 65 65
Partials 54 54
Continue to review full report at Codecov.
|
Will it be better to filter keys outside fastcache? |
This won't be better, since returned keys for huge number of cached items (typical workload for |
fastcache.go
Outdated
idx += keyLen | ||
value := chunk[idx : idx+valLen] | ||
|
||
if err := f(key, value); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently it is called under per-bucket read lock. This means the bucket contents cannot be modified until all its entries are visited. This may take a lot of time for slow callbacks. I'm unsure whether this is OK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also would lead to deadlock on an attempt to call Cache.Set(key, ...)
inside f
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we mention these possible situations in the docs, do you think users will still ignore them and they will end up in undesired states?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we mention these possible situations in the docs, do you think users will still ignore them and they will end up in undesired states?
I'm absolutely sure :) There are the following choices:
- to document all the restrictions and caveats in the hope users read and adhere the docs
- to harden the implementation against users who don't read docs. I.e. do not hold the lock during
f
call and copykey
,value
args passed tof
. This would slow down the implementation - to fork
fastcache
and implement this in the fork
I'm leaning to the last solution. Let's leave the PR open for a while.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we push on a queue all operations (get, set, del...) while VisitAllEntries is working? And when it finishes, we run all the operations (in the same order, of course).
Retrieves cached keys by regex pattern.
Please don't put everything inside doc only. You can write the limitations, caveats and few basic complete examples on the readme.md Things to look out for should be most visible. There are a lot of nice features and TTL is definitely needed here. Please do make TTL ready at least as soon as possible. I think the format of method can be implemented and set, e.g. SetWithTTL(k,v,t) etc or just SetT(k,v,t) etc can be implemented already. No one will read the doc complete but at least they will definitely read the readme.md AND a wiki section will be nice with FAQs section |
some case TTL is must and cool, but some others cache had. fastcache is perfect in my trial project and running in product now. |
@andreiavrammsd do you need any help to finish this PR? |
@diegobernardes, the PR is finished but not accepted because of performance concerns. Please see the discussions. |
Is there a feature for VisitAllEntries()? How can I code this / add to personal repo? |
Why not develop a function of simple Keys without regex pattern,I think there will be fewer performance problems 😄 |
Iterate all entries in cache with a given callback with signature
f func(k, v []byte) error
. Iteration stops if error is returned from callback.Outdated:
Retrieves cached keys by regex pattern.