chore: capping mechanism for relay and service connections #3184

darshankabariya · 2024-11-28T06:26:26Z

This PR prevents nodes from becoming isolated from the network by implementing a capping mechanism for service and relay peers. This ensures there is always room available for relay peers at any given time.

close #3163

github-actions · 2024-11-28T06:31:41Z

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:3184

Built from 62d5a1d

SionoiS

LGTM

waku/node/peer_manager/peer_manager.nim

gabrielmer

Looks great, thanks so much!

Added just a couple questions/comments

gabrielmer · 2024-12-10T11:29:11Z

waku/node/peer_manager/peer_manager.nim

@@ -993,6 +999,24 @@ proc new*(
    # Leave by default 20% of connections for service peers
    maxRelayPeersValue = maxConnections - (maxConnections div 5)

+  var maxServicePeersValue = 0
+  var d_low = 4


I think we shouldn't hardcode this and take it directly from WakuRelay

gabrielmer · 2024-12-10T11:31:40Z

waku/node/peer_manager/peer_manager.nim

+      # relay has higher priority then service peers
+  else:
+    maxServicePeersValue =
+      max(maxConnections - maxRelayPeersValue, maxConnections div 5)


what's the logic of this line? Why the maxConnections div 5?

My logic is to reserve 20% for service peers if the user does not provide maxService, as we discuss in nwaku pm we need to change whole logic

Ivansete-status

The PR doesn't compile snif snif :s
Re-thinking about this, I suggest doing the following:
We need to keep maxConnections config param because this needs to be passed to the nim-libp2p-Switch object.
On the other hand, I would only add an additional parameter, named protoConnWeights , a.k.a. proto-conn-weights, with the following logic:

If proto-conn-weights is not passed, then we apply the internal limits of 70% for relay, and the other 30% is evenly distributed for other protocols.

If proto-conn-weights=relay:80;services:20, we apply the internal limits of 80% for relay, and the other 20% is evenly distributed for other protocols.

We need to make sure the relay limit is bigger than DHigh ( we can create a
public Constant in waku_relay/protocol.nim )

nwaku/waku/waku_relay/protocol.nim

Line 73 in 09f05fe

dHigh = 8,

wdyt @NagyZoltanPeter , @gabrielmer , @darshankabariya ?

Ivansete-status · 2024-11-29T09:49:47Z

waku/node/peer_manager/peer_manager.nim

@@ -993,6 +995,20 @@ proc new*(
    # Leave by default 20% of connections for service peers
    maxRelayPeersValue = maxConnections - (maxConnections div 5)

+  var maxServicePeersValue = 0
+  var d_low = 4


We will need to use the d_low value extracted from waku_relay. Perhaps defining a const in waku_relay and use that value

darshankabariya · 2024-12-12T21:38:49Z

The PR doesn't compile snif snif

My bad, I pushed the code without a local build. I’ll update this PR until we finalize it.

darshankabariya · 2024-12-12T21:56:00Z

The PR doesn't compile snif snif :s Re-thinking about this, I suggest doing the following: We need to keep maxConnections config param because this needs to be passed to the nim-libp2p-Switch object. On the other hand, I would only add an additional parameter, named protoConnWeights , a.k.a. proto-conn-weights, with the following logic:

If proto-conn-weights is not passed, then we apply the internal limits of 70% for relay, and the other 30% is evenly distributed for other protocols.

If proto-conn-weights=relay:80;services:20, we apply the internal limits of 80% for relay, and the other 20% is evenly distributed for other protocols.

We need to make sure the relay limit is bigger than DHigh ( we can create a public Constant in waku_relay/protocol.nim )

nwaku/waku/waku_relay/protocol.nim

Line 73 in 09f05fe

dHigh = 8,

wdyt @NagyZoltanPeter , @gabrielmer , @darshankabariya ?

At the moment, we have parameters like maxConnections, maxRelayPeers, maxServicePeers, and another one called protoConnWeight. This can make things more confusing, as users need to ensure that maxConnections is never greater than the sum of maxRelayPeers and maxServicePeers. Additionally, aligning proto-conn-weights with maxRelayPeers and maxServicePeers adds another layer of complexity.

Instead, I believe users generally don’t think in terms of absolute numbers for services and relays. From a psychological perspective, they might find it more intuitive to think in percentages—e.g., 80% for services, 20% for relays.

My suggestion is to simplify this structure. We should keep maxConnections bcz ( represent hardware capacity ? right ) , which users are familiar with, and deprecate maxRelayPeers and maxServicePeers. Instead, we can ask users to specify the percentage they want to allocate for relays and services. This feels more straightforward and user-friendly. maxConnection, relayWeight and serviceWeight will be deadly combination. the user to easily convey their priorities.

If users don’t provide this percentage, we could default to a ratio, as Ivan suggested. I believe this would simplify the experience while maintaining flexibility.

@gabrielmer @NagyZoltanPeter @Ivansete-status @SionoiS

SionoiS · 2024-12-13T12:39:30Z

I like your idea! IMO maxConnection and relayToServiceRatio (float) would be simpler.

I feel we still need hardcoded minimum still so that you can't, by mistake brick your node.

gabrielmer · 2024-12-13T14:39:08Z

Same opinion here, having 2 params, a maximum and a ratio to know how many to allocate for each should make things simple yet customizable :)

darshankabariya · 2024-12-14T23:23:44Z

I like your idea! IMO maxConnection and relayToServiceRatio (float) would be simpler.

I feel we still need hardcoded minimum still so that you can't, by mistake brick your node.

Same opinion here, having 2 params, a maximum and a ratio to know how many to allocate for each should make things simple yet customizable :)

Thanks @gabrielmer @SionoiS for reply ! i also like suggestion.

If this approach looks good to you, I can begin the implementation. Just to confirm, we're only discussing the deprecation of the external configuration parameter, correct? I believe we still need to maxRelayPeer and maxServicePeer within the peerManager.

Regarding your thoughts on the ratio, I have an idea: instead of using a float, we could represent it as a string, like 0.33 becoming 40:60 or 0.43 as 30:70. it's just need calculator 😂for some ratio, Let me know what you guys think, and I’ll implement accordingly.

gabrielmer · 2024-12-16T09:36:54Z

If this approach looks good to you, I can begin the implementation. Just to confirm, we're only discussing the deprecation of the external configuration parameter, correct? I believe we still need to maxRelayPeer and maxServicePeer within the peerManager.

Regarding your thoughts on the ratio, I have an idea: instead of using a float, we could represent it as a string, like 0.33 becoming 40:60 or 0.43 as 30:70. it's just need calculator 😂for some ratio, Let me know what you guys think, and I’ll implement accordingly.

Yes! That would only be for the external configuration, we would then translate it into maxRelayPeers and maxServicePeers when configuring the Peer Manager

Regarding the format, I'm pretty neutral about it - no strong opinions. Would like to hear what others think :))

SionoiS · 2024-12-16T12:29:39Z

Regarding your thoughts on the ratio, I have an idea: instead of using a float, we could represent it as a string, like 0.33 becoming 40:60 or 0.43 as 30:70. it's just need calculator 😂for some ratio, Let me know what you guys think, and I’ll implement accordingly.

If that's simpler go for it! No strong opinion here either.

NagyZoltanPeter

Great effort! I just added some tiny last notice on it if you would like to consider.

NagyZoltanPeter · 2025-01-17T09:16:30Z

waku/common/utils/parse_size_units.nim

@@ -49,3 +49,28 @@ proc parseCorrectMsgSize*(input: string): uint64 =
  let ret = parseMsgSize(input).valueOr:
    return 0
  return ret
+
+proc parseRelayServiceRatio*(ratio: string): Result[(float, float), string] =


Suggested change

proc parseRelayServiceRatio*(ratio: string): Result[(float, float), string] =

proc parseRelayServiceRatio*(ratio: string): Result[(float, float), string] {.raises: [ValueError].}=

Let's not hide the exception, so compiler can help using the function properly.

NagyZoltanPeter · 2025-01-17T09:25:06Z

waku/factory/external_config.nim

+
+    maxRelayPeers* {.
+      desc:
+        "Deprecated. Use relay-service-ratio instead. It represents the maximum allowed number of relay peers.",


I buy in to this idea. All I wanted to emphasize that do nothide the unintended/old usage of maxRelayPeers behind a log warning and a --help or docs.
Maybe it is better to stop node early and notice user about bad config, IDK.

waku/factory/node_factory.nim

Ivansete-status

Thanks for it and the patience! 🙌
I think we still need to keep backward compatibility with the current max-relay-peers param.
On the other hand, my dreamed nwaku only deals with Result types, doesn't use discard and doesn't raise Exception either ;P

Ivansete-status · 2025-01-20T14:58:02Z

waku/common/utils/parse_size_units.nim

+  try:
+    let elements = ratio.split(":")
+    if elements.len != 2:
+      raise newException(ValueError, "expected format 'X:Y', ratio = " & ratio)


Oh, sorry that I didn't mention before. Would it be possible to avoid raising an exception plz? Applies elsewhere. We already have the Resullt returning type to tackle the err conditions, unless I'm missing something :)

Ivansete-status · 2025-01-20T15:05:11Z

waku/factory/node_factory.nim

@@ -101,8 +101,15 @@ proc initNode(
    agentString = some(conf.agentString),
  )
  builder.withColocationLimit(conf.colocationLimit)
+
+  # Backword compatibility for maxRelayPeers
+  if conf.maxRelayPeers.isSome():


If this condition is true, then we need to infer the relayServiceRation based on the given conf.maxConnections and conf.maxRelayPeers. And we need to keep this temporarily within two consecutive versions

Thanks for your review and patience. It’s now backward compatible and parse function without raising exceptions.

Ivansete-status

LGTM! Thanks for it! 💯 🥳

Ivansete-status · 2025-01-20T21:12:43Z

waku/factory/builder.nim

 ) =
-  builder.maxRelayPeers = maxRelayPeers
-  builder.shardAware = shardAware
+  try:


Maybe not needed to handle exception now?

Ivansete-status · 2025-01-20T21:14:10Z

waku/node/peer_manager/peer_manager.nim

+  var relayRatio: float64
+  var serviceRatio: float64
+  try:
+    (relayRatio, serviceRatio) = parseRelayServiceRatio(relayServiceRatio).get()


Awesome thanks! I think now we don't need to handle the exception

github-actions bot assigned darshankabariya Nov 28, 2024

darshankabariya changed the title ~~chore: Capping mechanism for relay and service Connections~~ chore: capping mechanism for relay and service connections Nov 28, 2024

darshankabariya marked this pull request as draft December 3, 2024 10:45

darshankabariya force-pushed the capping_mechanism branch from 2e420e4 to fa97a4f Compare December 4, 2024 11:57

darshankabariya requested review from gabrielmer, Ivansete-status, NagyZoltanPeter and SionoiS December 9, 2024 14:25

darshankabariya marked this pull request as ready for review December 9, 2024 15:00

SionoiS approved these changes Dec 9, 2024

View reviewed changes

waku/node/peer_manager/peer_manager.nim Outdated Show resolved Hide resolved

waku/node/peer_manager/peer_manager.nim Show resolved Hide resolved

gabrielmer reviewed Dec 10, 2024

View reviewed changes

Ivansete-status reviewed Dec 12, 2024

View reviewed changes

darshankabariya removed the request for review from NagyZoltanPeter December 12, 2024 21:37

darshankabariya marked this pull request as draft December 12, 2024 21:37

darshankabariya closed this Dec 15, 2024

darshankabariya force-pushed the capping_mechanism branch from 02fd08c to ab0c1d4 Compare December 15, 2024 16:29

darshankabariya reopened this Dec 15, 2024

darshankabariya marked this pull request as ready for review December 16, 2024 08:15

darshankabariya closed this Dec 16, 2024

darshankabariya force-pushed the capping_mechanism branch from d5a12ef to 049fbea Compare December 16, 2024 19:19

darshankabariya reopened this Dec 16, 2024

darshankabariya requested a review from Ivansete-status January 1, 2025 07:17

darshankabariya force-pushed the capping_mechanism branch from a82c63f to 6edd7cd Compare January 16, 2025 17:44

darshankabariya requested a review from NagyZoltanPeter January 17, 2025 06:14

NagyZoltanPeter approved these changes Jan 17, 2025

View reviewed changes

Ivansete-status requested changes Jan 17, 2025

View reviewed changes

waku/factory/node_factory.nim Outdated Show resolved Hide resolved

darshankabariya force-pushed the capping_mechanism branch 2 times, most recently from 4366e81 to 6d637a9 Compare January 20, 2025 04:59

darshankabariya requested a review from Ivansete-status January 20, 2025 05:04

darshankabariya force-pushed the capping_mechanism branch from 4f141a3 to c9716b2 Compare January 20, 2025 05:40

darshankabariya and others added 16 commits January 20, 2025 13:42

feat: everything in one PR

738c507

feat: reverse all test related things, will handle in seperate PR

9f778af

chore: make it centralized expection

a89d619

chore: only touch test which are failing

7c00a8e

chore: update test

1bb28bd

chore: update test

2d9ec0f

chore: update test

7d1a2a7

chore: update test

6720dfd

chore: update test

8b80dc5

chore: lint update

a2ed692

chore: lint update

4347631

chore: lint update

5c61d77

chore: lint update

42c4dd9

chore: update build

0724f7d

chore: error update

c625d14

chore: lint update

eef0d11

darshankabariya force-pushed the capping_mechanism branch 2 times, most recently from 509fce5 to eef0d11 Compare January 20, 2025 08:18

Ivansete-status reviewed Jan 20, 2025

View reviewed changes

darshankabariya added 2 commits January 20, 2025 21:23

chore: backward compitibility

16caad0

chore: backward compitibility

03609a2

Ivansete-status approved these changes Jan 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: capping mechanism for relay and service connections #3184

chore: capping mechanism for relay and service connections #3184

darshankabariya commented Nov 28, 2024

github-actions bot commented Nov 28, 2024 •

edited

Loading

SionoiS left a comment

gabrielmer left a comment

gabrielmer Dec 10, 2024

gabrielmer Dec 10, 2024

darshankabariya Dec 11, 2024

Ivansete-status left a comment •

edited

Loading

Ivansete-status Nov 29, 2024

darshankabariya commented Dec 12, 2024

darshankabariya commented Dec 12, 2024 •

edited

Loading

SionoiS commented Dec 13, 2024

gabrielmer commented Dec 13, 2024

darshankabariya commented Dec 14, 2024 •

edited

Loading

gabrielmer commented Dec 16, 2024

SionoiS commented Dec 16, 2024

NagyZoltanPeter left a comment

NagyZoltanPeter Jan 17, 2025

NagyZoltanPeter Jan 17, 2025

Ivansete-status left a comment

Ivansete-status Jan 20, 2025

darshankabariya Jan 20, 2025

Ivansete-status Jan 20, 2025

darshankabariya Jan 20, 2025

Ivansete-status left a comment

Ivansete-status Jan 20, 2025

Ivansete-status Jan 20, 2025

	proc parseRelayServiceRatio*(ratio: string): Result[(float, float), string] =
	proc parseRelayServiceRatio*(ratio: string): Result[(float, float), string] {.raises: [ValueError].}=

chore: capping mechanism for relay and service connections #3184

Are you sure you want to change the base?

chore: capping mechanism for relay and service connections #3184

Conversation

darshankabariya commented Nov 28, 2024

github-actions bot commented Nov 28, 2024 • edited Loading

SionoiS left a comment

Choose a reason for hiding this comment

gabrielmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivansete-status left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darshankabariya commented Dec 12, 2024

darshankabariya commented Dec 12, 2024 • edited Loading

SionoiS commented Dec 13, 2024

gabrielmer commented Dec 13, 2024

darshankabariya commented Dec 14, 2024 • edited Loading

gabrielmer commented Dec 16, 2024

SionoiS commented Dec 16, 2024

NagyZoltanPeter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivansete-status left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivansete-status left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 28, 2024 •

edited

Loading

Ivansete-status left a comment •

edited

Loading

darshankabariya commented Dec 12, 2024 •

edited

Loading

darshankabariya commented Dec 14, 2024 •

edited

Loading