Offload signature checking to taskpools (gigagas!) #2927
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
arnetheduck
changed the title
In block processing, depending on the complexity of a transaction and hotness of caches etc, signature checking can actually make up the majority of time needed to process a transaction (60% observed in some randomly sampled block ranges). Fortunately, this is a task that trivially can be offloaded to a task pool similar to how nimbus-eth2 does it. This PR introduces taskpools in the most simple way possible, by performing signature checking concurrently with other TX processing, assigning a taskpool task per TX effectively. With this little trick, we're in gigagas land 🎉 on my laptop! ``` INF 2024-12-10 21:05:35.170+01:00 Imported blocks blockNumber=3874817 b... mgps=1222.707 ... ``` Tests don't use the taskpool for now because it needs manual cleanup and we don't have a good mechanism in place. Future PR:s should address this by creating a common shutdown sequence that also closes and cleans up other resources like the DB.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In block processing, depending on the complexity of a transaction and hotness of caches etc, signature checking can actually make up the majority of time needed to process a transaction (60% observed in some randomly sampled block ranges).
Fortunately, this is a task that trivially can be offloaded to a task pool similar to how nimbus-eth2 does it.
This PR introduces taskpools in the most simple way possible, by performing signature checking concurrently with other TX processing, assigning a taskpool task per TX effectively.
With this little trick, we're in gigagas land 🎉 on my laptop!
Tests don't use the taskpool for now because it needs manual cleanup and we don't have a good mechanism in place. Future PR:s should address this by creating a common shutdown sequence that also closes and cleans up other resources like the DB.