Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement stable hashring for node management #30

Merged
merged 51 commits into from
Dec 19, 2023
Merged

feat: implement stable hashring for node management #30

merged 51 commits into from
Dec 19, 2023

Conversation

AmeanAsad
Copy link
Contributor

@AmeanAsad AmeanAsad commented Oct 26, 2023 •
edited
Loading

Description

  • Implements a hashring implementation to dictate which nodes receive requests
  • the hashring should direct requests to the same cid to the same nodes.
  • the hashring supports fallback functionality; it's able to return multiple nodes given a cid path.
  • the nodes in the hashring are weighted, so closer nodes with higher performance are more likely to receive requests.

AmeanAsad and others added 30 commits October 11, 2023 00:17
@AmeanAsad
feat: implement a nodes list for clients
804978c
@AmeanAsad
feat: implement a storage interface using indexedDb
f4d1844
@AmeanAsad
feat: implement a test suite for fallback
894d16d
@AmeanAsad
fix: remove unused code
7009c7a
@AmeanAsad
fix: eslint an jsdoc
3336a87
@AmeanAsad
fix: formatting and consistency
696cd8a
@AmeanAsad
fix: indexDbCheck
a3dec56
@AmeanAsad
Merge branch 'main' into feat/fallback
747d780
@AmeanAsad
chore: change storage implementation
8dec1c7
@AmeanAsad
enhancement: simplify node loading
b0dc673
@AmeanAsad
naive fallback implementation
78651fc
@AmeanAsad
modify fallback
d15c0ad
@AmeanAsad
fix formatting and typos
29e8da1
@AmeanAsad
typos
194ce70
@AmeanAsad
Update .eslintrc
1d97dc9 
Co-authored-by: Diego Rodríguez Baquero <diego@protocol.ai>
@AmeanAsad
enhancement: edit storage impl
84a1b8e
@AmeanAsad
enhancement: deal with overlapping byte chunks
f688406
@AmeanAsad
feat: add fallback test suite
5b7c215
@AmeanAsad
fix: tests running
6068a90
@AmeanAsad
cleanup content fetch with fallback
2c3cc79
@AmeanAsad
add initial origin fetch to fallback
4cf0c00
@AmeanAsad
formatting and file re-org
d81038f
@AmeanAsad @guanzo
feat: merge main into fallback branch (#22)
a1ecd50 
* Abort on error (#19)

* feat: use controller from options if exists. abort fetch if error occurs.

* test: check if external abort controller is used

* build: move build output to dist/ folder

* fix: newline

* 0.1.1

* Build exports (#20)

* chore: rename file

* feat: add new entrypoint with exports.

Switch Saturn to named export

* build: expose entire module instead of just the default export

* docs: update README

* 0.2.0

* feat: include worker scopes when checking for browser runtime (#21)

* 0.3.0

---------

Co-authored-by: Eric Guan <guanzo91@gmail.com>
@AmeanAsad
Merge branch 'main' into feat/fallback
55d56d1
@AmeanAsad
load nodes on first success
d2c5c37
@AmeanAsad
add fallback limit
e0065d8
@AmeanAsad
fix: fallback bug
109019b
@AmeanAsad
put eslint settings in package.json
b84ce4a
@AmeanAsad
add nodesListKey as static
7731fe8
@AmeanAsad
fix: resolve process in browser
e0fcfd8
@AmeanAsad AmeanAsad marked this pull request as ready for review October 26, 2023 20:06
@AmeanAsad AmeanAsad requested a review from guanzo October 26, 2023 20:07
guanzo
guanzo reviewed Oct 30, 2023
View reviewed changes
src/client.js Outdated Show resolved Hide resolved
@AmeanAsad
Copy link
Contributor Author

@guanzo ready for review again

@guanzo
Copy link
Collaborator

guanzo commented Dec 6, 2023

the hashring supports fallback functionality; it's able to return multiple nodes given a cid path.

I'm thinking we should code for the worst case, and allow falling back to ALL available nodes, if the nodes for the cid path all fail.

@AmeanAsad AmeanAsad merged commit cc7a758 into main Dec 19, 2023
1 check passed
@AmeanAsad AmeanAsad deleted the feat/hashring branch December 19, 2023 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guanzo guanzo guanzo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@AmeanAsad @guanzo