feat: change nonReapeater and add maxRepetitions as configurable

.github/workflows/ci-charts-artifact.yaml

@@ -0,0 +1,35 @@
+name: create-charts-zip
+on:
+  pull_request:
+    branches:
+      - "main"
+      - "develop"
+      - "next"
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      packages: write
+    strategy:
+      matrix:
+        python-version:
+          - 3.9
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+      - name: Zip charts directory and upload it
+        run: |
+          zip -r charts.zip charts 
+          VERSION=$(echo $GITHUB_REF | cut -d / -f 3)
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Create artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ env.ARTIFACT_NAME }}
+          path: charts.zip
+        env:
+          ARTIFACT_NAME: charts

charts/splunk-connect-for-snmp/values.yaml

@@ -224,10 +224,14 @@ poller:
   # https://github:com/splunk/splunk-connect-for-snmp/blob/main/splunk_connect_for_snmp/profiles/base.yaml
   pollBaseProfiles: true
 
-  # Sometimes SNMP Agent cannot accept more than X OIDs per once, so if the error "TooBig" is visible in logs,
+  # Sometimes SNMP Agent cannot accept more than X OIDs per once for SNMP GetRequest, so if the error "TooBig" is visible in logs,
   # decrease the number
   maxOidToProcess: 70
 
+  # maxRepetitions is a number of varbinds requested in response for each of the MIB variables in the request
+  # If this number is too high it can cause issues with SNMP response especially for old SNMP devices.
+  maxRepetitions: 10
+
   # list of kubernetes secrets name that will be used for polling
   # https://splunk.github.io/splunk-connect-for-snmp/main/microk8s/configuration/poller-configuration/#define-usernamesecrets
   usernameSecrets: []

docs/microk8s/configuration/poller-configuration.md

@@ -3,7 +3,7 @@
 Poller is a service which is responsible for querying 
 SNMP devices using the SNMP GET and WALK functionalities. Poller executes two main types of tasks:
 
-- The Walk task executes SNMP walk. SNMP walk is an SNMP application that uses SNMP GETNEXT requests to 
+- The Walk task executes SNMP walk. SNMP walk is an SNMP application that uses SNMP GETBULK requests to 
 collect SNMP data from the network and infrastructure of SNMP-enabled devices, such as switches and routers. It is a time-consuming task,
 which may overload the SNMP device when executed too often. It is used by the SC4SNMP to collect and push all OID values, which the provided ACL has access to. 
 
@@ -41,6 +41,32 @@ The default value is `INFO`.
 ### Define usernameSecrets
 Secrets are required to run SNMPv3 polling. To add v3 authentication details, create the k8s Secret object: [SNMPv3 Configuration](snmpv3-configuration.md), and put its name in `poller.usernameSecrets`.
 
+### Define maxRepetitions
+The maxRepetitions is a parameter used in SNMP GetBulk call. It is responsible for controlling the
+amount of variables in one request. 
+```yaml
+poller:
+  maxRepetitions: 10
+```
+`maxRepetitions` variable is the amount of requested next oids in response for each of varbinds in one request sent.
+
+For example:
+
+The configured variables:
+```yaml
+poller:
+  maxRepetitions: 2
+```
+The requested varbinds in one getBulk call:
+```
+IP-MIB.ipNetToMediaNetAddress
+```
+
+[![PDU Request Example](../images/request_pdu_flow.png)](../images/request_pdu_flow.png)
+
+After third ResponsePDU the returned oids are out of scope for requested table, so the call is stopped. 
+It can be spotted on diagram that response for `IP-MIB.ipNetToMediaNetAddress` includes 2 oids as `maxRepetition` was set to 3.
+
 ### Append OID index part to the metrics
 
 Not every SNMP metric object is structured with its index as a one of the field values. We can append the index part of OID with:

rendered/manifests/tests/splunk-connect-for-snmp/templates/worker/poller/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests/splunk-connect-for-snmp/templates/worker/trap/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_autoscaling_enabled/splunk-connect-for-snmp/templates/worker/poller/deployment.yaml

@@ -67,6 +67,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_autoscaling_enabled/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -67,6 +67,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_autoscaling_enabled/splunk-connect-for-snmp/templates/worker/trap/deployment.yaml

@@ -67,6 +67,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_autoscaling_enabled_deprecated/splunk-connect-for-snmp/templates/worker/poller/deployment.yaml

@@ -67,6 +67,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_autoscaling_enabled_deprecated/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -67,6 +67,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_autoscaling_enabled_deprecated/splunk-connect-for-snmp/templates/worker/trap/deployment.yaml

@@ -67,6 +67,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_enable_ui/splunk-connect-for-snmp/templates/worker/poller/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_enable_ui/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_enable_ui/splunk-connect-for-snmp/templates/worker/trap/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_only_polling/splunk-connect-for-snmp/templates/worker/poller/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_only_polling/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_only_traps/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_only_traps/splunk-connect-for-snmp/templates/worker/trap/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_probes_enabled/splunk-connect-for-snmp/templates/worker/poller/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_probes_enabled/splunk-connect-for-snmp/templates/worker/sender/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

rendered/manifests/tests_probes_enabled/splunk-connect-for-snmp/templates/worker/trap/deployment.yaml

@@ -68,6 +68,8 @@ spec:
               value: "70"
             - name: PYSNMP_DEBUG
               value: ""
+            - name: MAX_REPETITIONS
+              value: "10"
             - name: PROFILES_RELOAD_DELAY
               value: "60"
             - name: MIB_SOURCES

splunk_connect_for_snmp/snmp/manager.py

@@ -64,6 +64,7 @@
 UDP_CONNECTION_TIMEOUT = int(os.getenv("UDP_CONNECTION_TIMEOUT", 3))
 MAX_OID_TO_PROCESS = int(os.getenv("MAX_OID_TO_PROCESS", 70))
 PYSNMP_DEBUG = os.getenv("PYSNMP_DEBUG", "")
+MAX_REPETITIONS = int(os.getenv("MAX_REPETITIONS", 10))
 
 DEFAULT_STANDARD_MIBS = [
     "HOST-RESOURCES-MIB",
@@ -430,8 +431,8 @@ def run_bulk_request(
             auth_data,
             transport,
             context_data,
-            1,
-            10,
+            0,
+            MAX_REPETITIONS,
             *varbinds_bulk,
             lexicographicMode=False,
             ignoreNonIncreasingOid=is_increasing_oids_ignored(ir.address, ir.port),