-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: allow running versioned experiments (#451)
This PR allows us to filter the experiments we want to run in a
declarative way. Experiments must now be explicitly defined in the [new
manifest file
embedded](go-chaos/internal/chaos-experiments/camunda-cloud/manifest.yml)
in the chaos worker.
The manifest looks like this:
```yaml
experiments:
- path: broker-dataloss/experiment.json
clusterPlans:
- production-s
- path: deployment-distribution/experiment.json
clusterPlans:
- production-s
# only for testing
- path: test/experiment.json
clusterPlans:
- test
- path: test/version-experiment.json
clusterPlans:
- test
minVersion: 8.3
maxVersion: 8.4
```
> [!Note]
> For version bounds, we only check the major and minor version. So 8.3,
8.4, etc. If the target cluster is 8.4.0-SNAPSHOT and the bound is 8.4,
then it will match.
The logic when reading the manifest is as follows:
- Given the current target cluster plan X, only experiments who have `X`
in their `clusterPlans` list will be picked.
- Given the current target cluster version X, only experiments whose
`minVersion >= X` `maxVersion <= X` are picked.
- The `minVersion` and `maxVersion` parameters are completely optional.
If you specify none, then the experiment runs on any version.
- If we somehow fail to get the target cluster version, any experiment
with version bound is omitted.
The target version is obtained by getting the topology of the target
cluster, either from a given namespace, or from the current namespace of
the kubeconfig if no namespace was provided.- Loading branch information
Showing
19 changed files
with
317 additions
and
43 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
50 changes: 50 additions & 0 deletions
50
go-chaos/internal/chaos-experiments/camunda-cloud/manifest.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # Lists all the experiments that the chaos-worker can run. | ||
| # Each experiment must use the following structure: | ||
| # | ||
| # - path: path to the JSON file relative to the manifest | ||
| # clusterPlans: a list of cluster plans the experiment can run on | ||
| # minVersion: the minimum (inclusive) version of the target cluster the experiment can run against | ||
| # maxVersion: the maximum (inclusive) version of the target cluster the experiment can run against | ||
| # | ||
| # Both minVersion and maxVersion should only be the major and minor versions, e.g. 8.3, 8.4, etc. | ||
| # minVersion and maxVersion are both optional. | ||
| experiments: | ||
| - path: broker-dataloss/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: deployment-distribution/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: follower-restart/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: follower-terminate/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: leader-restart/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: leader-terminate/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: msg-correlation/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: multiple-leader-restart/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: stress-cpu-on-broker/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| - path: worker-restart/experiment.json | ||
| clusterPlans: | ||
| - production-s | ||
| # only for testing | ||
| - path: test/experiment.json | ||
| clusterPlans: | ||
| - test | ||
| - path: test/version-experiment.json | ||
| clusterPlans: | ||
| - test | ||
| minVersion: 8.3 | ||
| maxVersion: 8.4 |
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
39 changes: 39 additions & 0 deletions
39
go-chaos/internal/chaos-experiments/camunda-cloud/test/version-experiment.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| { | ||
| "version": "0.1.0", | ||
| "title": "Versioned Test Experiment", | ||
| "description": "This fake experiment is just to test the integration with Zeebe and zbchaos workers", | ||
| "contributions": { | ||
| "reliability": "high", | ||
| "availability": "high" | ||
| }, | ||
| "steady-state-hypothesis": { | ||
| "title": "Zeebe is alive", | ||
| "probes": [ | ||
| { | ||
| "name": "Show version", | ||
| "type": "probe", | ||
| "tolerance": 0, | ||
| "provider": { | ||
| "type": "process", | ||
| "path": "zbchaos", | ||
| "arguments": ["version"], | ||
| "timeout": 900 | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| "method": [ | ||
| { | ||
| "type": "action", | ||
| "name": "Show again the version", | ||
| "tolerance": 0, | ||
| "provider": { | ||
| "type": "process", | ||
| "path": "zbchaos", | ||
| "arguments": ["version"], | ||
| "timeout": 900 | ||
| } | ||
| } | ||
| ], | ||
| "rollbacks": [] | ||
| } |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.