Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replication: Eventual and Strong Consistency for SwarmNL #39

Merged
merged 75 commits into from
Dec 10, 2024
Merged

Replication: Eventual and Strong Consistency for SwarmNL #39

merged 75 commits into from
Dec 10, 2024

Conversation

thewoodfish
Copy link
Member

Title:

Replication: Eventual and Strong Consistency for SwarmNL

Description:

This PR introduces robust synchronization models to SwarmNL, providing support for eventual consistency and strong consistency replication. These models enable seamless data synchronization across nodes in various scenarios, improving the system's flexibility and reliability.

Key additions include:

  • Eventual Consistency: Ensures data updates propagate to all nodes asynchronously, achieving consistency over time without blocking operations.
  • Strong Consistency: Guarantees that all nodes agree on the same data state immediately after updates, suitable for scenarios requiring stricter guarantees.
  • Examples and Documentation: Practical usage examples and updated documentation to guide users in integrating and testing these models.

These improvements strengthen SwarmNL's capabilities and provide users with greater control over their data synchronization strategies. Feedback and suggestions are highly welcome!

thewoodfish and others added 30 commits November 21, 2024 08:10
@thewoodfish
fix: removed overflow bug from DataQueue::push()
98008a8
@thewoodfish
fix: import cleanup
3ad25f1
@thewoodfish
chore: impl of data synchronization algorithms
f7df0f6
@thewoodfish
feat: impl of the strong consistency data synchronization model
be9100f
@thewoodfish
feat: strong consistency data sync impl
de2e0dc
@thewoodfish
chore: impl of the eventual consistency sync model
59acbce
@thewoodfish
feat: eventual consistency data sync model impl
5921cdb
@thewoodfish
feat: impl of eventual consistency data sync model
65c581d
@thewoodfish
feat: fault tolerance and recorvery mechanism impl
5d5fbde
@thewoodfish
fix: code refactoring and optimization
a15d509
@thewoodfish
feat: impl of sharding algorithms
a5f4408
@sacha-l
docs: correct punctuation
2aaac36
@sacha-l
chore: move repl_cfg to separate file
aa77fd1
@thewoodfish
chore: impl the Sharding::get() method
9a0c079
@sacha-l
add: buffer queue tests
7f09712
@sacha-l
fmt
c9f5368
@sacha-l
add: update basic tests
fbdc5fa
@thewoodfish
chore: complete the sharding fetch() function
b692e6a
@thewoodfish
fix: changes port number for test function
f614dd7
@thewoodfish
feat: impl of the fetch fn for shard networks
9546d39
@thewoodfish
fix: prevented node from requesting message self published from peers
6c2047a
@thewoodfish
feat: separated sharding operations into its own file module
a76a695
@sacha-l
fix: safe math
4c3da0c
@thewoodfish
chore: complete research.md
d3fa214
@sacha-l
add: sharding unit test
441d36c
@thewoodfish
chore: complete research.md
4d4c5f6
@sacha-l
fix: remove unused files
d282e06
@sacha-l
docs: nits
fc4b105
@thewoodfish
chore: complete research.md
5899610
@thewoodfish
fix: updated the exit_network() fn for sharding
74e9aa6
thewoodfish and others added 29 commits December 8, 2024 02:58
@thewoodfish
fix: path adjustment in cargo.toml
ce6850a
@thewoodfish
fix: direactory cleanup
397a2b9
@sacha-l
fix: add layer communication tests back in
36225dc
@sacha-l
fix: docs for enum fields
cb181ec
@sacha-l
fix: docs
4281216
@sacha-l
fix: docs
bca8afd
@sacha-l
add: tests for repl
5f0779d
@sacha-l
add repl tests in mod
1305a51
@sacha-l
fix test
d92f0f4
@sacha-l
add: test guide for join and exit
efc6a5b
@sacha-l
fix: test rename
4ffcd98
@sacha-l
add: tests
4932dd8
@thewoodfish
fix: debugging and refactoring
f528e01
@thewoodfish
fix: debugged itest
022c9b0
@thewoodfish
fix: itest refactoring
271a875
@sacha-l
add: tests for strong consistency
4aff2f7
@thewoodfish
chore: debug range based sharding example
493ea08
@thewoodfish
fix: resolved deadlock in replication.rs
8746ff4
@sacha-l
fix: consistency test works
397ab4e
@thewoodfish
fix: debugging of the strong consistency model impl
99ee892
@thewoodfish
fix: replication debugging
ebc398f
@sacha-l
add: more testing
ea72e36
@thewoodfish
fix: strong consistency debugging
e848242
@sacha-l
finish consistency tests
85ae132
@sacha-l
fix: improve test setup node fn
ca7fbfe
@thewoodfish
fix: sharding example
444780d
@sacha-l
more tests
e234218
@sacha-l
fmt
5f65d10
@sacha-l
add: finish tests for eventual consistency
0951bc5
@sacha-l sacha-l merged commit cd6a3ef into main Dec 10, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@thewoodfish @sacha-l