From e7daef42b26147e2caf27420953829bc4fb91432 Mon Sep 17 00:00:00 2001 From: jvoravong Date: Thu, 16 Jan 2025 09:30:55 -0700 Subject: [PATCH 01/19] Update docs and values.shema.json for operatorcrds migration --- .../move-operator-crd-install-method.yaml | 9 ++- docs/auto-instrumentation-install.md | 43 ++++++++++++- ...rator-and-auto-instrumentation-values.yaml | 2 + .../otel-demo-dotnet.md | 4 +- .../otel-demo-nodejs.md | 4 +- .../spring-petclinic-java.md | 4 +- helm-charts/splunk-otel-collector/Chart.yaml | 2 +- .../splunk-otel-collector/values.schema.json | 64 +++++++++++++------ helm-charts/splunk-otel-collector/values.yaml | 13 ++-- 9 files changed, 105 insertions(+), 40 deletions(-) diff --git a/.chloggen/move-operator-crd-install-method.yaml b/.chloggen/move-operator-crd-install-method.yaml index 5a69759ed5..cbe1b87d60 100644 --- a/.chloggen/move-operator-crd-install-method.yaml +++ b/.chloggen/move-operator-crd-install-method.yaml @@ -1,12 +1,15 @@ # One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' -change_type: bug_fix +change_type: breaking # The name of the component, or a single word describing the area of concern, (e.g. agent, clusterReceiver, gateway, operator, chart, other) component: operator # A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). note: Move operator CRD installation to the crds/ folder via a subchart to resolve Helm install ordering issues # One or more tracking issues related to the change -issues: [1561] +issues: [1561,1619] # (Optional) One or more lines of additional information to render under the primary note. # These lines will be padded with 2 spaces and then inserted directly into the document. # Use pipe (|) for multiline entries. -subtext: +subtext: | + - Users enabling the operator (`.Values.operator.enabled=true`) must now set `operatorcrds.install=true` in Helm values or [manually manage CRD installation](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/auto-instrumentation-install.md#crd-management). + - Previously, CRDs were installed using templates (`operator.crds.create=true`), which could cause race conditions and install failures. + - CRD installation is now handled via Helm's native `crds/` directory for better stability, using a [localized subchart](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds). diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index 7a07132e4e..4737f9f6c7 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -26,6 +26,8 @@ these frameworks often have pre-built instrumentation capabilities already avail ### 1. Deploy the Helm Chart with the Operator enabled - **Operator Deployment (Required)** + - `operatorcrds.install`: Set to `true` to install the CRDs required by the operator. + - **Required**: Must be set unless CRDs are pre-installed manually. - `operator.enabled`: Set to `true` to enable deploying the operator. - **Required**: This configuration is necessary for the operator's deployment within your cluster. @@ -70,7 +72,7 @@ these frameworks often have pre-built instrumentation capabilities already avail kubectl get pods -l app=cert-manager --all-namespaces # If cert-manager is not deployed, make sure to add certmanager.enabled=true to the list of values to set -helm install splunk-otel-collector -f ./my_values.yaml --set operator.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector +helm install splunk-otel-collector -f ./my_values.yaml --set operatorcrds.install=true,operator.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector ``` ### 2. Verify all the OpenTelemetry resources (collector, operator, webhook, instrumentation) are deployed successfully @@ -284,6 +286,11 @@ in a Kubernetes environment. An operator is a method of packaging, deploying, an In the context of setting up observability in a Kubernetes environment, an operator simplifies the management of application auto-instrumentation, making it easier to gain valuable insights into application performance. +The OpenTelemetry operator relies on +[Custom Resource Definitions (CRDs)](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds) +to manage auto-instrumentation configurations in Kubernetes. +Ensure the required CRDs are deployed before the operator (by configuring `operatorcrds.install=true`). + With this Splunk OTel Collector chart, the [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator#opentelemetry-auto-instrumentation-injection) can be deployed (by configuring `operator.enabled=true`) to your cluster and start auto-instrumenting your applications. @@ -403,6 +410,40 @@ provides best effort support with issues related to native OpenTelemetry instrum | apache-httpd | OpenTelemetry | Available | Needs Validation | | [Link](https://github.com/open-telemetry/opentelemetry-apache-httpd-instrumentation) | ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-apache-httpd | | nginx | OpenTelemetry | Available | Needs Validation | | [Link](https://github.com/open-telemetry/opentelemetry-apache-httpd-instrumentation) | ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-apache-httpd | +### CRD Management + +When deploying the operator, the required Custom Resource Definitions (CRDs) must be deployed beforehand. + +#### Recommended Approach: Automated CRD Deployment + +Set the Helm chart value `operatorcrds.install=true` to allow the chart to handle CRD deployment automatically. +_This option deploys the CRDs using a local subchart, available at https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds._ + +#### Alternative Approach: Manual CRD Deployment + +If you prefer to manage CRD deployment manually, apply the CRDs using the commands below before installing the Helm chart: + +```bash +curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_opentelemetrycollectors.yaml | kubectl apply -f - +curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_opampbridges.yaml | kubectl apply -f - +curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_instrumentations.yaml | kubectl apply -f - +``` + +#### CRD Updates + +With Helm v3.0 and later, CRDs created by this chart are not updated automatically. To update CRDs, you must apply the updated CRD definitions manually. +Refer to the [Helm Documentation on CRDs](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/) for more details. + +#### CRD Cleanup + +When uninstalling this chart, the OpenTelemetry CRDs are not removed automatically. To delete them manually, use the following commands: + +```bash +kubectl delete crd opentelemetrycollectors.opentelemetry.io +kubectl delete crd opampbridges.opentelemetry.io +kubectl delete crd instrumentations.opentelemetry.io +``` + ### Documentation Resources - https://developers.redhat.com/devnation/tech-talks/using-opentelemetry-on-kubernetes diff --git a/examples/enable-operator-and-auto-instrumentation/enable-operator-and-auto-instrumentation-values.yaml b/examples/enable-operator-and-auto-instrumentation/enable-operator-and-auto-instrumentation-values.yaml index bad14c1eb6..def8c7d41c 100644 --- a/examples/enable-operator-and-auto-instrumentation/enable-operator-and-auto-instrumentation-values.yaml +++ b/examples/enable-operator-and-auto-instrumentation/enable-operator-and-auto-instrumentation-values.yaml @@ -7,6 +7,8 @@ splunkObservability: clusterName: CHANGEME environment: CHANGEME +operatorcrds: + install: true operator: enabled: true certmanager: diff --git a/examples/enable-operator-and-auto-instrumentation/otel-demo-dotnet.md b/examples/enable-operator-and-auto-instrumentation/otel-demo-dotnet.md index 3eb6a695b9..d28b6b2a41 100644 --- a/examples/enable-operator-and-auto-instrumentation/otel-demo-dotnet.md +++ b/examples/enable-operator-and-auto-instrumentation/otel-demo-dotnet.md @@ -24,7 +24,7 @@ curl https://raw.githubusercontent.com/signalfx/splunk-otel-collector-chart/main #### 2.1 Deploy the Helm Chart with the Operator enabled To install the chart with operator in an existing cluster, make sure you have cert-manager installed and available. -Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set certmanager.enabled=true,operator.enabled=true`. +Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set operatorcrds.install=true,operator.enabled=true,certmanager.enabled=true`. These helm install commands will deploy the chart to the current namespace for this example. ```bash @@ -32,7 +32,7 @@ These helm install commands will deploy the chart to the current namespace for t kubectl get pods -l app=cert-manager --all-namespaces # If cert-manager is deployed, make sure to remove certmanager.enabled=true to the list of values to set -helm install splunk-otel-collector -f ./my_values.yaml --set operator.enabled=true,certmanager.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector +helm install splunk-otel-collector -f ./my_values.yaml --set operatorcrds.install=true,operator.enabled=true,certmanager.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector ``` #### 2.2 Verify all the OpenTelemetry resources (collector, operator, webhook, instrumentation) are deployed successfully diff --git a/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md b/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md index 3185e554c2..c835e69d61 100644 --- a/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md +++ b/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md @@ -20,7 +20,7 @@ curl https://raw.githubusercontent.com/signalfx/splunk-otel-collector-chart/main #### 2.1 Deploy the Helm Chart with the Operator enabled To install the chart with operator in an existing cluster, make sure you have cert-manager installed and available. -Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set certmanager.enabled=true,operator.enabled=true`. +Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set operatorcrds.instal=true,operator.enabled=true,certmanager.enabled=true`. These helm install commands will deploy the chart to the current namespace for this example. ```bash @@ -28,7 +28,7 @@ These helm install commands will deploy the chart to the current namespace for t kubectl get pods -l app=cert-manager --all-namespaces # If cert-manager is deployed, make sure to remove certmanager.enabled=true to the list of values to set -helm install splunk-otel-collector -f ./my_values.yaml --set operator.enabled=true,certmanager.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector +helm install splunk-otel-collector -f ./my_values.yaml --set operatorcrds.install=true,operator.enabled=true,certmanager.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector ``` #### 2.2 Verify all the OpenTelemetry resources (collector, operator, webhook, instrumentation) are deployed successfully diff --git a/examples/enable-operator-and-auto-instrumentation/spring-petclinic-java.md b/examples/enable-operator-and-auto-instrumentation/spring-petclinic-java.md index d918e6be6b..fb5d93b640 100644 --- a/examples/enable-operator-and-auto-instrumentation/spring-petclinic-java.md +++ b/examples/enable-operator-and-auto-instrumentation/spring-petclinic-java.md @@ -20,7 +20,7 @@ curl https://raw.githubusercontent.com/signalfx/splunk-otel-collector-chart/main #### 2.1 Deploy the Helm Chart with the Operator enabled To install the chart with operator in an existing cluster, make sure you have cert-manager installed and available. -Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set certmanager.enabled=true,operator.enabled=true`. +Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set operatorcrds.install=true,operator.enabled=true,certmanager.enabled=true`. These helm install commands will deploy the chart to the current namespace for this example. ```bash @@ -28,7 +28,7 @@ These helm install commands will deploy the chart to the current namespace for t kubectl get pods -l app=cert-manager --all-namespaces # If cert-manager is deployed, make sure to remove certmanager.enabled=true to the list of values to set -helm install splunk-otel-collector -f ./my_values.yaml --set operator.enabled=true,certmanager.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector +helm install splunk-otel-collector -f ./my_values.yaml --set operatorcrds.install=true,operator.enabled=true,certmanager.enabled=true,environment=dev splunk-otel-collector-chart/splunk-otel-collector ``` #### 2.2 Verify all the OpenTelemetry resources (collector, operator, webhook, instrumentation) are deployed successfully diff --git a/helm-charts/splunk-otel-collector/Chart.yaml b/helm-charts/splunk-otel-collector/Chart.yaml index 9a9bd97dd3..93a9d37107 100644 --- a/helm-charts/splunk-otel-collector/Chart.yaml +++ b/helm-charts/splunk-otel-collector/Chart.yaml @@ -28,7 +28,7 @@ dependencies: repository: https://charts.jetstack.io condition: certmanager.enabled - name: opentelemetry-operator-crds - version: "0.0.1" + version: 0.0.1 alias: operatorcrds condition: operatorcrds.install - name: opentelemetry-operator diff --git a/helm-charts/splunk-otel-collector/values.schema.json b/helm-charts/splunk-otel-collector/values.schema.json index 9d4c549b86..176736b03d 100644 --- a/helm-charts/splunk-otel-collector/values.schema.json +++ b/helm-charts/splunk-otel-collector/values.schema.json @@ -1424,15 +1424,57 @@ "type": "boolean", "deprecated": true }, + "operatorcrds": { + "description": "Deploys Operator related CRDs from https://github.com/open-telemetry/opentelemetry-operator/tree/main/config/crd via a Helm subchart and using the chart crds/ directory.", + "type": "object", + "properties": { + "install": { + "type": "boolean" + } + } + }, + "operator-crds": { + "description": "Configuration for operator-crds (legacy). Prefer using 'operatorcrds' for new installations. Deploys Operator related CRDs from https://github.com/open-telemetry/opentelemetry-operator/tree/main/config/crd via a Helm subchart and using the chart crds/ directory.", + "type": "object", + "properties": { + "install": { + "type": "boolean" + } + } + }, "operator": { "description": "OpenTelemetry Operator configuration. A subchart that is used to install the operator, see https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-operator/values.schema.json for more info.", "type": "object", - "additionalProperties": true + "additionalProperties": true, + "properties": { + "crds": { + "type": "object", + "properties": { + "create": { + "description": "This value must always be false to avoid race condition errors during Helm deployments.", + "type": "boolean", + "const": false + } + } + } + } }, "opentelemetry-operator": { "description": "OpenTelemetry Operator configuration (legacy). Prefer using 'operator' for new installations. This field is provided for backward compatibility with older Helm versions or custom distributions. A subchart that is used to install the operator, see https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-operator/values.schema.json for more info.", "type": "object", - "additionalProperties": true + "additionalProperties": true, + "properties": { + "crds": { + "type": "object", + "properties": { + "create": { + "description": "This value must always be false to avoid race condition errors during Helm deployments.", + "type": "boolean", + "const": false + } + } + } + } }, "instrumentation": { "type": "object", @@ -1684,24 +1726,6 @@ } } }, - "operatorcrds": { - "description": "Deploys Operator related CRDs from https://github.com/open-telemetry/opentelemetry-operator/tree/main/config/crd via a Helm subchart and using the chart crds/ directory.", - "type": "object", - "properties": { - "install": { - "type": "boolean" - } - } - }, - "operator-crds": { - "description": "Configuration for operator-crds (legacy). Prefer using 'operatorcrds' for new installations. Deploys Operator related CRDs from https://github.com/open-telemetry/opentelemetry-operator/tree/main/config/crd via a Helm subchart and using the chart crds/ directory.", - "type": "object", - "properties": { - "install": { - "type": "boolean" - } - } - }, "certmanager": { "description": "cert-manager adds certificates and certificate issuers as resource types in Kubernetes clusters, and simplifies the process of obtaining, renewing and using those certificates.", "type": "object", diff --git a/helm-charts/splunk-otel-collector/values.yaml b/helm-charts/splunk-otel-collector/values.yaml index 97d3d98584..e0500b2bfb 100644 --- a/helm-charts/splunk-otel-collector/values.yaml +++ b/helm-charts/splunk-otel-collector/values.yaml @@ -1174,21 +1174,16 @@ service: # Full list of Helm value configurations: https://artifacthub.io/packages/helm/opentelemetry-helm/opentelemetry-operator?modal=values ################################################################################ -# Should the CRDs be installed by a chart's crd/ directory or by using templates. +# Specify whether the chart should install CRDs automatically. +# Related Documentation: https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/auto-instrumentation-install.md#crd-management operatorcrds: - # Should the CRDs be installed + # Set to true to install CRDs automatically, or false to manage them manually. install: false operator: enabled: false - # This is disabled by default in favor of using `operatorcrds.install=true`, as doing so creates - # a race condition with helm. - # See: https://github.com/open-telemetry/opentelemetry-helm-charts/issues/677 - # Users of this chart should _never_ set this to be true. If a user wishes - # to install the CRDs through the opentelemetry-operator chart, it is recommended - # to install the opentelemetry-operator chart separately and prior to the installation - # of this chart. crds: + # This value is set to false and cannot be changed; use `operatorcrds.install` instead. create: false admissionWebhooks: certManager: From 70c13944cf843da159899eb9ea39293d75fe4f6a Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Thu, 16 Jan 2025 17:32:28 -0700 Subject: [PATCH 02/19] Update .chloggen/move-operator-crd-install-method.yaml Co-authored-by: Curtis Robert --- .chloggen/move-operator-crd-install-method.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.chloggen/move-operator-crd-install-method.yaml b/.chloggen/move-operator-crd-install-method.yaml index cbe1b87d60..199edde999 100644 --- a/.chloggen/move-operator-crd-install-method.yaml +++ b/.chloggen/move-operator-crd-install-method.yaml @@ -11,5 +11,5 @@ issues: [1561,1619] # Use pipe (|) for multiline entries. subtext: | - Users enabling the operator (`.Values.operator.enabled=true`) must now set `operatorcrds.install=true` in Helm values or [manually manage CRD installation](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/auto-instrumentation-install.md#crd-management). - - Previously, CRDs were installed using templates (`operator.crds.create=true`), which could cause race conditions and install failures. + - Previously, CRDs were installed using templates (`operator.crds.create=true`), which could cause race conditions and installation failures. - CRD installation is now handled via Helm's native `crds/` directory for better stability, using a [localized subchart](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds). From cefcf2a1c0a8bae1f75c6fa7c7d2b97dda60cad8 Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Thu, 16 Jan 2025 17:32:35 -0700 Subject: [PATCH 03/19] Update docs/auto-instrumentation-install.md Co-authored-by: Curtis Robert --- docs/auto-instrumentation-install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index 4737f9f6c7..bdd6a06963 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -289,7 +289,7 @@ application auto-instrumentation, making it easier to gain valuable insights int The OpenTelemetry operator relies on [Custom Resource Definitions (CRDs)](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds) to manage auto-instrumentation configurations in Kubernetes. -Ensure the required CRDs are deployed before the operator (by configuring `operatorcrds.install=true`). +Ensure the required CRDs are deployed before the operator by configuring `operatorcrds.install=true`. With this Splunk OTel Collector chart, the [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator#opentelemetry-auto-instrumentation-injection) From 66c23fc77cb03e34f1f27e87b3cac031020f139b Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Thu, 16 Jan 2025 17:32:47 -0700 Subject: [PATCH 04/19] Update docs/auto-instrumentation-install.md Co-authored-by: Curtis Robert --- docs/auto-instrumentation-install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index bdd6a06963..1d4bb77d96 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -417,7 +417,7 @@ When deploying the operator, the required Custom Resource Definitions (CRDs) mus #### Recommended Approach: Automated CRD Deployment Set the Helm chart value `operatorcrds.install=true` to allow the chart to handle CRD deployment automatically. -_This option deploys the CRDs using a local subchart, available at https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds._ +_This option deploys the CRDs using a local subchart, available at [opentelemetry-operator-crds](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds)._ #### Alternative Approach: Manual CRD Deployment From f2cf017071b9a933574dca6dd034ff039caef9c0 Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 07:56:54 -0700 Subject: [PATCH 05/19] Add upgrading notes for operator crd migration --- .../move-operator-crd-install-method.yaml | 1 + UPGRADING.md | 81 ++++++++++++++----- 2 files changed, 62 insertions(+), 20 deletions(-) diff --git a/.chloggen/move-operator-crd-install-method.yaml b/.chloggen/move-operator-crd-install-method.yaml index 199edde999..5d9e1e2344 100644 --- a/.chloggen/move-operator-crd-install-method.yaml +++ b/.chloggen/move-operator-crd-install-method.yaml @@ -13,3 +13,4 @@ subtext: | - Users enabling the operator (`.Values.operator.enabled=true`) must now set `operatorcrds.install=true` in Helm values or [manually manage CRD installation](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/auto-instrumentation-install.md#crd-management). - Previously, CRDs were installed using templates (`operator.crds.create=true`), which could cause race conditions and installation failures. - CRD installation is now handled via Helm's native `crds/` directory for better stability, using a [localized subchart](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds). + - If you use `operator.create=ture` you may have to follow some migration steps, please see the [Upgrade guidelines](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/UPGRADING.md#0113-to-0116) diff --git a/UPGRADING.md b/UPGRADING.md index 41f83040dd..78cdabcddd 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -1,35 +1,76 @@ # Upgrade guidelines -## 0.105.5 to 0.108.0 +## 0.113.0 to 0.116.0 -We've simplified the Helm chart configuration for `operator` auto-instrumentation. -The values previously under `.Values.operator.instrumentation.spec.*` have been moved to `.Values.instrumentation.*`. +This guide provides steps for new users, transitioning users, and those maintaining previous operator CRD configurations: +- New users: No migration for CRDs is required. +- Previous users: Migration may be needed if using `operator.enabled=true`. -- **No Action Needed**: If you have no customizations under `.Values.operator.instrumentation.spec.*`, no migration is required. -- **Action Required**: Continuing to use the old values path will result in a Helm install or upgrade error, blocking the process. +CRD deployment has evolved over chart versions: +- Before 0.110.0: CRDs were deployed via a crds/ directory (upstream default). +- 0.110.0 to 1.113.0: CRDs were deployed using Helm templates (upstream default), which had reported issues. +- 0.116.0 and later: Users must now explicitly configure their preferred CRD deployment method or deploy the + [CRDs manually](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/auto-instrumentation-install.md#crd-management) + to avoid potential issues. Users can deploy CRDs via a crds/ directory again by enabling a newly added value. -Migration Steps: +### **New Users** -1. **Find** any references to `.Values.operator.instrumentation.spec.*` in your Helm values with custom values. -2. **Migrate** them from `.Values.operator.instrumentation.spec.*` to `.Values.instrumentation.*`. +New users are advised to deploy CRDs via the `crds/` directory. For a fresh installation, use the following Helm values: -Example Migration: +```yaml +operatorcrds.install: true +operator.enabled: true +``` -Before (Deprecated Path): +To install the chart: -```yaml -operator: - instrumentation: - spec: - endpoint: XXX - ... +```bash +helm install splunk-otel-collector --set operatorcrds.install=true,operator.enabled=true ``` -After (Updated Path): +### **Current Users (Recommended Migration to `crds/` Directory)** + +If you're using chart versions 0.110.0 to 1.113.0, CRDs are likely deployed via Helm templates. To migrate to the recommended `crds/` directory deployment: + +#### Step 1: Delete the Existing Chart + +Remove the chart to prepare for a fresh installation: +```bash +helm delete +``` + +#### Step 2: Verify or Remove Existing CRDs +Check if the following CRDs are present and delete them if necessary: + +```bash +kubectl get crds | grep opentelemetry +``` + +```bash +kubectl delete crd opentelemetrycollectors.opentelemetry.io +kubectl delete crd opampbridges.opentelemetry.io +kubectl delete crd instrumentations.opentelemetry.io +``` + +#### Step 3: Reinstall with Recommended Values +Reinstall the chart with the updated configuration: +```bash +helm install splunk-otel-collector --set operatorcrds.install=true,operator.enabled=true +``` + +### **Previous Users (Maintaining Legacy Helm Templates)** + +If you're using chart versions 0.110.0 to 1.113.0 and prefer to continue deploying CRDs via Helm templates (not recommended), you can do so with the following values: + ```yaml -instrumentation: - endpoint: XXX - ... +operator.enabled: true +operator.crds.create: true +``` + +**Warning**: This method may cause race conditions during installation or upgrades, leading to errors like: +```plaintext +ERROR: INSTALLATION FAILED: failed post-install: warning: Hook post-install splunk-otel-collector/templates/operator/instrumentation.yaml failed: 1 error occurred: +* Internal error occurred: failed calling webhook "minstrumentation.kb.io": failed to call webhook: Post "https://splunk-otel-collector-operator-webhook.default.svc:443/mutate-opentelemetry-io-v1alpha1-instrumentation?timeout=10s": dial tcp X.X.X.X:443: connect: connection refused ``` ## 0.105.3 to 0.105.4 From 82adc99a7d297a1d991d2877fca8b6bd72928325 Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 14:58:16 -0700 Subject: [PATCH 06/19] Reverse preventing deploying operator crds via helm templates (operator.crds.create) --- .../splunk-otel-collector/values.schema.json | 28 ++----------------- helm-charts/splunk-otel-collector/values.yaml | 8 +++++- 2 files changed, 9 insertions(+), 27 deletions(-) diff --git a/helm-charts/splunk-otel-collector/values.schema.json b/helm-charts/splunk-otel-collector/values.schema.json index 176736b03d..399fbe6660 100644 --- a/helm-charts/splunk-otel-collector/values.schema.json +++ b/helm-charts/splunk-otel-collector/values.schema.json @@ -1445,36 +1445,12 @@ "operator": { "description": "OpenTelemetry Operator configuration. A subchart that is used to install the operator, see https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-operator/values.schema.json for more info.", "type": "object", - "additionalProperties": true, - "properties": { - "crds": { - "type": "object", - "properties": { - "create": { - "description": "This value must always be false to avoid race condition errors during Helm deployments.", - "type": "boolean", - "const": false - } - } - } - } + "additionalProperties": true }, "opentelemetry-operator": { "description": "OpenTelemetry Operator configuration (legacy). Prefer using 'operator' for new installations. This field is provided for backward compatibility with older Helm versions or custom distributions. A subchart that is used to install the operator, see https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-operator/values.schema.json for more info.", "type": "object", - "additionalProperties": true, - "properties": { - "crds": { - "type": "object", - "properties": { - "create": { - "description": "This value must always be false to avoid race condition errors during Helm deployments.", - "type": "boolean", - "const": false - } - } - } - } + "additionalProperties": true }, "instrumentation": { "type": "object", diff --git a/helm-charts/splunk-otel-collector/values.yaml b/helm-charts/splunk-otel-collector/values.yaml index e0500b2bfb..35e0060f10 100644 --- a/helm-charts/splunk-otel-collector/values.yaml +++ b/helm-charts/splunk-otel-collector/values.yaml @@ -1182,8 +1182,14 @@ operatorcrds: operator: enabled: false + # This is disabled by default in favor of using `operatorcrds.install=true`, as doing so creates + # a race condition with helm. + # See: https://github.com/open-telemetry/opentelemetry-helm-charts/issues/677 + # Users of this chart should _never_ set this to be true. If a user wishes + # to install the CRDs through the opentelemetry-operator chart, it is recommended + # to install the opentelemetry-operator chart separately and prior to the installation + # of this chart. crds: - # This value is set to false and cannot be changed; use `operatorcrds.install` instead. create: false admissionWebhooks: certManager: From de24a653c56a9906279c0b2134d3a0173cb16642 Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 15:02:14 -0700 Subject: [PATCH 07/19] Add back UPGRADING.md entry that was accidentally removed --- UPGRADING.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/UPGRADING.md b/UPGRADING.md index 78cdabcddd..2acef91dd1 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -73,6 +73,38 @@ ERROR: INSTALLATION FAILED: failed post-install: warning: Hook post-install splu * Internal error occurred: failed calling webhook "minstrumentation.kb.io": failed to call webhook: Post "https://splunk-otel-collector-operator-webhook.default.svc:443/mutate-opentelemetry-io-v1alpha1-instrumentation?timeout=10s": dial tcp X.X.X.X:443: connect: connection refused ``` +## 0.105.5 to 0.108.0 + +We've simplified the Helm chart configuration for `operator` auto-instrumentation. +The values previously under `.Values.operator.instrumentation.spec.*` have been moved to `.Values.instrumentation.*`. + +- **No Action Needed**: If you have no customizations under `.Values.operator.instrumentation.spec.*`, no migration is required. +- **Action Required**: Continuing to use the old values path will result in a Helm install or upgrade error, blocking the process. + +Migration Steps: + +1. **Find** any references to `.Values.operator.instrumentation.spec.*` in your Helm values with custom values. +2. **Migrate** them from `.Values.operator.instrumentation.spec.*` to `.Values.instrumentation.*`. + +Example Migration: + +Before (Deprecated Path): + +```yaml +operator: + instrumentation: + spec: + endpoint: XXX + ... +``` + +After (Updated Path): +```yaml +instrumentation: + endpoint: XXX + ... +``` + ## 0.105.3 to 0.105.4 The `Java instrumentation` for Operator auto-instrumentation has been upgraded from v1.32.2 to v2.7.0. From 2af180c63e2ae4de8e36636b85add27288166a9a Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:20:23 -0700 Subject: [PATCH 08/19] Update docs/auto-instrumentation-install.md Co-authored-by: Jina Jain --- docs/auto-instrumentation-install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index 1d4bb77d96..d76214bef4 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -416,7 +416,7 @@ When deploying the operator, the required Custom Resource Definitions (CRDs) mus #### Recommended Approach: Automated CRD Deployment -Set the Helm chart value `operatorcrds.install=true` to allow the chart to handle CRD deployment automatically. +Set the Helm chart value `operatorcrds.install=true` to allow the chart to handle CRD installation automatically. _This option deploys the CRDs using a local subchart, available at [opentelemetry-operator-crds](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds)._ #### Alternative Approach: Manual CRD Deployment From d969f4e2e9dd8bc974ea1ce37cd53268dc711508 Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:22:27 -0700 Subject: [PATCH 09/19] Update docs/auto-instrumentation-install.md Co-authored-by: Jina Jain --- docs/auto-instrumentation-install.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index d76214bef4..a9958a392c 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -428,7 +428,13 @@ curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_opampbridges.yaml | kubectl apply -f - curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_instrumentations.yaml | kubectl apply -f - ``` +You can also use below helm template command to get the CRD yamls from the helm chart. This method can be helpful in keeping CRDs in-sync with the version bundled with our helm chart. +```bash +helm template splunk-otel-collector-chart/splunk-otel-collector --include-crds \ +--set="splunkObservability.realm=us0,splunkObservability.accessToken=xxxxxx,clusterName=my-cluster,operatorcrds.install=true" \ +| yq e '. | select(.kind == "CustomResourceDefinition")' \ +| kubectl apply -f - #### CRD Updates With Helm v3.0 and later, CRDs created by this chart are not updated automatically. To update CRDs, you must apply the updated CRD definitions manually. From 0abc5ebb0b42ac7c7622ad41a756de9f4ab1aab1 Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:22:54 -0700 Subject: [PATCH 10/19] Update docs/auto-instrumentation-install.md Co-authored-by: Jina Jain --- docs/auto-instrumentation-install.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index a9958a392c..272742b172 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -449,7 +449,13 @@ kubectl delete crd opentelemetrycollectors.opentelemetry.io kubectl delete crd opampbridges.opentelemetry.io kubectl delete crd instrumentations.opentelemetry.io ``` +You can use below combination of helm and kubectl command to delete CRDs. +```bash +helm template splunk-otel-collector-chart/splunk-otel-collector --include-crds \ +--set="splunkObservability.realm=us0,splunkObservability.accessToken=xxxxxx,clusterName=my-cluster,operatorcrds.install=true" \ +| yq e '. | select(.kind == "CustomResourceDefinition")' \ +| kubectl delete --dry-run=client -f - ### Documentation Resources - https://developers.redhat.com/devnation/tech-talks/using-opentelemetry-on-kubernetes From 2fccb3ae0c5dd42dc5140529ecb3ee97a9765952 Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:23:05 -0700 Subject: [PATCH 11/19] Update UPGRADING.md Co-authored-by: Jina Jain --- UPGRADING.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/UPGRADING.md b/UPGRADING.md index 2acef91dd1..d34157b350 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -18,9 +18,10 @@ CRD deployment has evolved over chart versions: New users are advised to deploy CRDs via the `crds/` directory. For a fresh installation, use the following Helm values: ```yaml -operatorcrds.install: true -operator.enabled: true -``` +operatorcrds: + install: true +operator: + enabled: true To install the chart: From 6a9cfd96fbe549b46d8c4b45da04b58c1361212b Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:23:12 -0700 Subject: [PATCH 12/19] Update UPGRADING.md Co-authored-by: Jina Jain --- UPGRADING.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/UPGRADING.md b/UPGRADING.md index d34157b350..0162c4982e 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -64,9 +64,11 @@ helm install splunk-otel-collector --set operatorcrds.install=tru If you're using chart versions 0.110.0 to 1.113.0 and prefer to continue deploying CRDs via Helm templates (not recommended), you can do so with the following values: ```yaml -operator.enabled: true -operator.crds.create: true -``` +operator: + enabled: true +operator: + crds: + create: true **Warning**: This method may cause race conditions during installation or upgrades, leading to errors like: ```plaintext From 20fb0f5fa2ae2a76459950e0e75a434c694cd2a9 Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:23:23 -0700 Subject: [PATCH 13/19] Update UPGRADING.md Co-authored-by: Jina Jain --- UPGRADING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/UPGRADING.md b/UPGRADING.md index 0162c4982e..af94c752ca 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -26,7 +26,7 @@ operator: To install the chart: ```bash -helm install splunk-otel-collector --set operatorcrds.install=true,operator.enabled=true +helm install splunk-otel-collector-chart/splunk-otel-collector --set operatorcrds.install=true,operator.enabled=true ``` ### **Current Users (Recommended Migration to `crds/` Directory)** From 60883625d889239e662358d3b7a87ae57ae4337b Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 15:28:35 -0700 Subject: [PATCH 14/19] format fixes --- UPGRADING.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/UPGRADING.md b/UPGRADING.md index af94c752ca..248da0c3f6 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -22,6 +22,7 @@ operatorcrds: install: true operator: enabled: true +``` To install the chart: @@ -36,11 +37,13 @@ If you're using chart versions 0.110.0 to 1.113.0, CRDs are likely deployed via #### Step 1: Delete the Existing Chart Remove the chart to prepare for a fresh installation: + ```bash helm delete ``` #### Step 2: Verify or Remove Existing CRDs + Check if the following CRDs are present and delete them if necessary: ```bash @@ -54,6 +57,7 @@ kubectl delete crd instrumentations.opentelemetry.io ``` #### Step 3: Reinstall with Recommended Values + Reinstall the chart with the updated configuration: ```bash helm install splunk-otel-collector --set operatorcrds.install=true,operator.enabled=true @@ -69,6 +73,7 @@ operator: operator: crds: create: true +``` **Warning**: This method may cause race conditions during installation or upgrades, leading to errors like: ```plaintext From 16fdca5e277eefbf857bda14f2762945e725ba2a Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 15:32:05 -0700 Subject: [PATCH 15/19] format fix --- docs/auto-instrumentation-install.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index 272742b172..926b2f9c40 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -428,6 +428,7 @@ curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_opampbridges.yaml | kubectl apply -f - curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_instrumentations.yaml | kubectl apply -f - ``` + You can also use below helm template command to get the CRD yamls from the helm chart. This method can be helpful in keeping CRDs in-sync with the version bundled with our helm chart. ```bash @@ -435,6 +436,8 @@ helm template splunk-otel-collector-chart/splunk-otel-collector --include-crds \ --set="splunkObservability.realm=us0,splunkObservability.accessToken=xxxxxx,clusterName=my-cluster,operatorcrds.install=true" \ | yq e '. | select(.kind == "CustomResourceDefinition")' \ | kubectl apply -f - +``` + #### CRD Updates With Helm v3.0 and later, CRDs created by this chart are not updated automatically. To update CRDs, you must apply the updated CRD definitions manually. @@ -456,6 +459,8 @@ helm template splunk-otel-collector-chart/splunk-otel-collector --include-crds \ --set="splunkObservability.realm=us0,splunkObservability.accessToken=xxxxxx,clusterName=my-cluster,operatorcrds.install=true" \ | yq e '. | select(.kind == "CustomResourceDefinition")' \ | kubectl delete --dry-run=client -f - +``` + ### Documentation Resources - https://developers.redhat.com/devnation/tech-talks/using-opentelemetry-on-kubernetes From 67c13b1bd35703917480addf760c13749b7ad901 Mon Sep 17 00:00:00 2001 From: jvoravong <47871238+jvoravong@users.noreply.github.com> Date: Fri, 17 Jan 2025 15:38:01 -0700 Subject: [PATCH 16/19] Update .chloggen/move-operator-crd-install-method.yaml Co-authored-by: Jina Jain --- .chloggen/move-operator-crd-install-method.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.chloggen/move-operator-crd-install-method.yaml b/.chloggen/move-operator-crd-install-method.yaml index 5d9e1e2344..26d18e666e 100644 --- a/.chloggen/move-operator-crd-install-method.yaml +++ b/.chloggen/move-operator-crd-install-method.yaml @@ -13,4 +13,4 @@ subtext: | - Users enabling the operator (`.Values.operator.enabled=true`) must now set `operatorcrds.install=true` in Helm values or [manually manage CRD installation](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/auto-instrumentation-install.md#crd-management). - Previously, CRDs were installed using templates (`operator.crds.create=true`), which could cause race conditions and installation failures. - CRD installation is now handled via Helm's native `crds/` directory for better stability, using a [localized subchart](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds). - - If you use `operator.create=ture` you may have to follow some migration steps, please see the [Upgrade guidelines](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/UPGRADING.md#0113-to-0116) + - If you use `operator.enabled=true` you may have to follow some migration steps, please see the [Upgrade guidelines](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/UPGRADING.md#0113-to-0116). From 861ec4b23c48dd984685c730362744f8b90f38e5 Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 15:40:18 -0700 Subject: [PATCH 17/19] Add extra note --- docs/auto-instrumentation-install.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index 926b2f9c40..c45a124171 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -418,6 +418,7 @@ When deploying the operator, the required Custom Resource Definitions (CRDs) mus Set the Helm chart value `operatorcrds.install=true` to allow the chart to handle CRD installation automatically. _This option deploys the CRDs using a local subchart, available at [opentelemetry-operator-crds](https://github.com/signalfx/splunk-otel-collector-chart/tree/main/helm-charts/splunk-otel-collector/charts/opentelemetry-operator-crds)._ +_Please note, helm will not update or delete these CRDs after initial install as noted in their [documentation](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations)._ #### Alternative Approach: Manual CRD Deployment From a6c7cc6b13b0e7428336c88447444af7eed98c98 Mon Sep 17 00:00:00 2001 From: jvoravong Date: Fri, 17 Jan 2025 15:44:05 -0700 Subject: [PATCH 18/19] update links --- docs/auto-instrumentation-install.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/auto-instrumentation-install.md b/docs/auto-instrumentation-install.md index c45a124171..5248eda8e9 100644 --- a/docs/auto-instrumentation-install.md +++ b/docs/auto-instrumentation-install.md @@ -425,9 +425,9 @@ _Please note, helm will not update or delete these CRDs after initial install as If you prefer to manage CRD deployment manually, apply the CRDs using the commands below before installing the Helm chart: ```bash -curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_opentelemetrycollectors.yaml | kubectl apply -f - -curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_opampbridges.yaml | kubectl apply -f - -curl -sL https://raw.githubusercontent.com/open-telemetry/opentelemetry-operator/main/config/crd/bases/opentelemetry.io_instrumentations.yaml | kubectl apply -f - +curl -sL https://raw.githubusercontent.com/splunk-otel-collector/charts/opentelemetry-operator-crds/crds/opentelemetry.io_opentelemetrycollectors.yaml | kubectl apply -f - +curl -sL https://raw.githubusercontent.com/splunk-otel-collector/charts/opentelemetry-operator-crds/crds/opentelemetry.io_opampbridges.yaml | kubectl apply -f - +curl -sL https://raw.githubusercontent.com/splunk-otel-collector/charts/opentelemetry-operator-crds/crds/opentelemetry.io_instrumentations.yaml | kubectl apply -f - ``` You can also use below helm template command to get the CRD yamls from the helm chart. This method can be helpful in keeping CRDs in-sync with the version bundled with our helm chart. From 97ed40016e0f20e67db56512bcd0a55f8592308f Mon Sep 17 00:00:00 2001 From: Jina Jain Date: Fri, 17 Jan 2025 14:50:14 -0800 Subject: [PATCH 19/19] Update examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md --- .../otel-demo-nodejs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md b/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md index c835e69d61..dfd9a08fa4 100644 --- a/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md +++ b/examples/enable-operator-and-auto-instrumentation/otel-demo-nodejs.md @@ -20,7 +20,7 @@ curl https://raw.githubusercontent.com/signalfx/splunk-otel-collector-chart/main #### 2.1 Deploy the Helm Chart with the Operator enabled To install the chart with operator in an existing cluster, make sure you have cert-manager installed and available. -Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set operatorcrds.instal=true,operator.enabled=true,certmanager.enabled=true`. +Both the cert-manager and operator are subcharts of this chart and can be enabled with `--set operatorcrds.install=true,operator.enabled=true,certmanager.enabled=true`. These helm install commands will deploy the chart to the current namespace for this example. ```bash