diff --git a/content/docs/next-release-v2/cassandra.md b/content/docs/next-release-v2/cassandra.md index e52b498a..85bb753e 100644 --- a/content/docs/next-release-v2/cassandra.md +++ b/content/docs/next-release-v2/cassandra.md @@ -3,37 +3,33 @@ title: Cassandra hasparent: true --- -* Supported versions: 4+ +* Supported Cassandra versions: 4.x, 5.x -{{< danger >}} -TODO update examples to use config properties, not CLI flags. -{{< /danger >}} - -Deploying Cassandra itself is out of scope for our documentation. One good -source of documentation is the [Apache Cassandra Docs](https://cassandra.apache.org/doc/latest/). +This page describes how to configure Jaeger to use an existing Cassandra cluster as a storage backend for traces. For instructions on how to deploy a Cassandra cluster, please refer to [Apache Cassandra Documentation](https://cassandra.apache.org/doc/latest/). Cassandra also has the following officially supported resources available from the community: - [Docker container](https://hub.docker.com/_/cassandra) for getting a single node up quickly - [Helm chart](https://artifacthub.io/packages/helm/bitnami/cassandra) from Bitnami - [Kubernetes Operator](https://github.com/k8ssandra/cass-operator) from DataStax -#### Configuration +## Configuration -Configuration example for [Jaeger writing to Cassandra](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-cassandra.yaml). +A sample configuration for Jaeger with Cassandra backend is available in the Jaeger repository: [config-cassandra.yaml](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-cassandra.yaml). In the future the configuration documentation will be auto-generated from the schema. Meanwhile, please refer to [config.go](https://github.com/jaegertracing/jaeger/blob/main/pkg/cassandra/config/config.go#L21) as the authoritative source. -#### Schema script +## Initializing Schema -A script is provided to initialize Cassandra keyspace and schema -using Cassandra's interactive shell [`cqlsh`][cqlsh]: +Before Jaeger can use a Cassandra cluster as storage backend, a keyspace and database schema +must be created. There is a script in the [jaeger](https://github.com/jaegertracing/jaeger) repository that generates the initialization instruction that can be executed using Cassandra's interactive shell [`cqlsh`][cqlsh]: ```sh MODE=test sh ./plugin/storage/cassandra/schema/create.sh | cqlsh ``` -Or using the published Docker image (make sure to provide the right IP address): +The same script is packaged as a container image (make sure to provide the right IP address): ```sh docker run \ -e CQLSH_HOST={server IP address} \ + -e MODE=prod \ jaegertracing/jaeger-cassandra-schema:{{< currentVersion >}} ``` @@ -43,24 +39,15 @@ where `{datacenter}` is the name used in the Cassandra configuration / network t The script also allows overriding TTL, keyspace name, replication factor, etc. Run the script without arguments to see the full list of recognized parameters. -**Note**: See [README](https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/cassandra/schema/README.md) for more details on Cassandra schema management. +* **Note 1**: See [README](https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/cassandra/schema/README.md) for more details on Cassandra schema management. +* **Note 2**: In the near future the logic for schema initialization will be moved to the Jaeger binary directly (subscribe to [Issue #5797](https://github.com/jaegertracing/jaeger/issues/5797) for updates). -#### TLS support +## TLS support -Jaeger supports TLS client to node connections as long as you've configured -your Cassandra cluster correctly. After verifying with e.g. `cqlsh`, you can -configure the collector and query like this: +Jaeger supports TLS client-to-node connections as long as you've configured +your Cassandra cluster correctly. You can specify paths to the TLS certificates (`.pem` files) un the `tls:` section under `connection:`. See [configtls.ClientConfig](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md#client-configuration) in the OpenTelemetry Collector repository for available properties. -```sh -docker run \ - -e CASSANDRA_SERVERS=<...> \ - -e CASSANDRA_TLS=true \ - -e CASSANDRA_TLS_SERVER_NAME="CN-in-certificate" \ - -e CASSANDRA_TLS_KEY= \ - -e CASSANDRA_TLS_CERT= \ - -e CASSANDRA_TLS_CA= \ - jaegertracing/jaeger-collector:{{< currentVersion >}} -``` +**Tip**: verify the correctness of your TLS certificates using `cqlsh` first. The schema tool also supports TLS. You need to make a custom cqlshrc file like so: @@ -86,6 +73,6 @@ usercert = ~/.cassandra/client-cert # validate = false ``` -#### Compatible Backends +## Compatible Backends * ScyllaDB [can be used](https://github.com/jaegertracing/jaeger/blob/main/plugin/storage/scylladb/README.md) as a drop-in replacement for Cassandra since it uses the same data model and query language. diff --git a/content/docs/next-release-v2/configuration.md b/content/docs/next-release-v2/configuration.md index 6ec56fd7..9814ce80 100644 --- a/content/docs/next-release-v2/configuration.md +++ b/content/docs/next-release-v2/configuration.md @@ -23,7 +23,7 @@ Here is an example of how to configure the extension: ```yaml jaeger_storage: backends: - some_storage: + some_trace_storage: memory: max_traces: 100000 metric_backends: @@ -48,7 +48,7 @@ In this example: ```yaml jaeger_query: storage: - traces: some_storage + traces: some_trace_storage metrics: some_metrics_storage base-path: / ui: @@ -86,7 +86,7 @@ remote_sampling: ```yaml jaeger_storage_exporter: - trace_storage: some_store + trace_storage: some_trace_storage queue: num_consumers: 10 queue_size: 100 diff --git a/content/docs/next-release-v2/deployment.md b/content/docs/next-release-v2/deployment.md index 2ec7def5..b5743e0b 100644 --- a/content/docs/next-release-v2/deployment.md +++ b/content/docs/next-release-v2/deployment.md @@ -31,50 +31,11 @@ See [APIs](../apis/) for the list of all API ports. Jaeger can be customized via configuration YAML file (see [Configuration](../configuration/)). - -**jaeger** is stateless and thus many instances of **jaeger** can be run in parallel. **jaeger** instances require almost no configuration, except for storage location, such as: - -Cassandra: -``` - jaeger_storage: - backends: - some_storage: - cassandra: - schema: - keyspace: "jaeger_v1_dc1" - connection: - auth: - basic: - username: "cassandra" - password: "cassandra" - tls: - insecure: true -``` - -OpenSearch: -``` - jaeger_storage: - backends: - some_storage: - opensearch: - index_prefix: "jaeger-main" -``` - -ElasticSearch: -``` - jaeger_storage: - backends: - some_storage: - elasticsearch: - index_prefix: "jaeger-main" - another_storage: - elasticsearch: - index_prefix: "jaeger-archive" -``` +Jaeger **collector** is stateless and thus many instances of **collector** can be run in parallel. **collector** instances require almost no configuration, except for storage location, such as [Cassandra](../cassandra/#configuration) or [Elasticsearch](../elasticsearch/#configuration). ## Query Configuration -The `jaeger_query` extension has a few deployment-related configuration options. +The `jaeger_query` extension has a few deployment-related configuration options. In the future the configuration documentation will be auto-generated from the schema. Meanwhile, please refer to [config.go](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/internal/extension/jaegerquery/config.go#L16) as the authoritative source. ### Clock Skew Adjustment @@ -82,26 +43,28 @@ Jaeger backend combines trace data from applications that are usually running on Sometimes these adjustments themselves make the trace hard to understand. For example, when repositioning the server span within the bounds of its parent span, Jaeger does not know the exact relationship between the request and response latencies, so it assumes they are equal and places the child span in the middle of the parent span (see [issue #961](https://github.com/jaegertracing/jaeger/issues/961#issuecomment-453925244)). -**jaeger** query extension supports configuration in the config file +The `jaeger_query` extension supports a configuration property that controls how much clock skew adjustment should be allowed. ``` -query: - max-clock-skew-adjustment: 30s +extensions: + jaeger_query: + max-clock-skew-adjustment: 30s ``` - that controls how much clock skew adjustment should be allowed. Setting this parameter to zero (`0s`) disables clock skew adjustment completely. This setting applies to all traces retrieved from the given query service. There is an open [ticket #197](https://github.com/jaegertracing/jaeger-ui/issues/197) to support toggling the adjustment on and off directly in the UI. + Setting this parameter to zero (`0s`) disables clock skew adjustment completely. This setting applies to all traces retrieved from the given query service. There is an open [ticket #197](https://github.com/jaegertracing/jaeger-ui/issues/197) to support toggling the adjustment on and off directly in the UI. ### UI Base Path -The base path for all **jaeger** query extension HTTP routes can be set to a non-root value, e.g. `/jaeger` would cause all UI URLs to start with `/jaeger`. This can be useful when running **jaeger** behind a reverse proxy. Here is example code to set the base path. +The base path for all `jaeger_query` extension HTTP routes can be set to a non-root value, e.g. `/jaeger` would cause all UI URLs to start with `/jaeger`. This can be useful when running Jaeger behind a reverse proxy. Here is example code to set the base path. ``` -query: - base-path: / - static-files: /go/bin/jaeger-ui-build/build - ui-config: /etc/jaeger/ui-config.json - grpc: - http: +extensions: + jaeger_query: + base-path: / + ui: + config_file: /etc/jaeger/ui-config.json + grpc: + http: ``` ### UI Customization and Embedding diff --git a/content/docs/next-release-v2/elasticsearch.md b/content/docs/next-release-v2/elasticsearch.md index e7cceb6b..5834ee38 100644 --- a/content/docs/next-release-v2/elasticsearch.md +++ b/content/docs/next-release-v2/elasticsearch.md @@ -1,33 +1,24 @@ --- -title: ElasticSearch +title: Elasticsearch hasparent: true --- ## Introduction -* Supported ES versions: 7.x, 8.x (since Jaeger v1.52.0) +* Supported ES versions: 7.x, 8.x -{{< danger >}} -TODO update examples to use config properties, not CLI flags. -{{< /danger >}} +Elasticsearch version is automatically retrieved from root/ping endpoint. Based on this version Jaeger uses compatible index mappings and Elasticsearch REST API. The version can be explicitly provided via `version:` config property. +Elasticsearch does not require initialization other than [installing and running Elasticsearch](https://www.elastic.co/downloads/elasticsearch). Once it is running, pass the correct configuration values to Jaeger. -Elasticsearch version is automatically retrieved from root/ping endpoint. -Based on this version Jaeger uses compatible index mappings and Elasticsearch REST API. -The version can be explicitly provided via `--es.version=` flag. - -Elasticsearch does not require initialization other than -[installing and running Elasticsearch](https://www.elastic.co/downloads/elasticsearch). -Once it is running, pass the correct configuration values to **jaeger**. - -ElasticSearch also has the following officially supported resources available from the community and Elastic: +Elasticsearch also has the following officially supported resources available from the community and Elastic: - [Docker container](https://hub.docker.com/_/elasticsearch) from Elastic for getting a single node up quickly - [Helm chart](https://artifacthub.io/packages/helm/elastic/elasticsearch) from Elastic - [Kubernetes Operator](https://github.com/openshift/elasticsearch-operator) from RedHat ## Configuration -Here is [example configuration](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-elasticsearch.yaml) for ElasticSearch. +A sample configuration for Jaeger with Elasticsearch backend is available in the Jaeger repository: [config-elasticsearch.yaml](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-elasticsearch.yaml). In the future the configuration documentation will be auto-generated from the schema. Meanwhile, please refer to [config.go](https://github.com/jaegertracing/jaeger/blob/main/pkg/es/config/config.go#L86) as the authoritative source. ### Shards and Replicas @@ -39,7 +30,7 @@ more information about choosing how many shards should be chosen for optimizatio [Elasticsearch rollover](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html) is an index management strategy that optimizes use of resources allocated to indices. For example, indices that do not contain any data still allocate shards, and conversely, a single index might contain significantly more data than the others. -Jaeger by default stores data in daily indices which might not optimally utilize resources. Rollover feature can be enabled by `--es.use-aliases=true`. +Jaeger by default stores data in daily indices which might not optimally utilize resources. Rollover feature can be enabled by `use_aliases: true` config property. Rollover lets you configure when to roll over to a new index based on one or more of the following criteria: @@ -59,27 +50,35 @@ For automated rollover, please refer to [Elasticsearch ILM support](#ilm-support The following command prepares Elasticsearch for rollover deployment by creating index aliases, indices, and index templates: ```sh -docker run -it --rm --net=host jaegertracing/jaeger-es-rollover:latest init http://localhost:9200 # <1> +docker run -it --rm --net=host \ + jaegertracing/jaeger-es-rollover:latest \ + init http://localhost:9200 # <1> ``` If you need to initialize archive storage, add `-e ARCHIVE=true`. -After the initialization Jaeger can be deployed with `--es.use-aliases=true`. +After the initialization Jaeger can be deployed with `use-aliases: true`. ### Roll over The next step is to periodically execute the rollover API which rolls the write alias to a new index based on supplied conditions. The command also adds a new index to the read alias to make new data available for search. ```shell -docker run -it --rm --net=host -e CONDITIONS='{"max_age": "2d"}' jaegertracing/jaeger-es-rollover:latest rollover http://localhost:9200 # <1> +docker run -it --rm --net=host \ + -e CONDITIONS='{"max_age": "2d"}' \ + jaegertracing/jaeger-es-rollover:latest \ + rollover http://localhost:9200 # <1> ``` <1> The command rolls the alias over to a new index if the age of the current write index is older than 2 days. For more conditions see [Elasticsearch docs](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html). -The next step is to remove old indices from read aliases. It means that old data will not be available for search. This imitates the behavior of `--es.max-span-age` flag used in the default index-per-day deployment. This step could be optional and old indices could be simply removed by index cleaner in the next step. +The next step is to remove old indices from read aliases. It means that old data will not be available for search. This imitates the behavior of `max_span_age:` config property used in the default index-per-day deployment. This step could be optional and old indices could be simply removed by index cleaner in the next step. ```sh -docker run -it --rm --net=host -e UNIT=days -e UNIT_COUNT=7 jaegertracing/jaeger-es-rollover:latest lookback http://localhost:9200 # <1> +docker run -it --rm --net=host \ + -e UNIT=days -e UNIT_COUNT=7 \ + jaegertracing/jaeger-es-rollover:latest \ + lookback http://localhost:9200 # <1> ``` <1> Removes indices older than 7 days from read alias. @@ -89,7 +88,10 @@ docker run -it --rm --net=host -e UNIT=days -e UNIT_COUNT=7 jaegertracing/jaeger The historical data can be removed with the `jaeger-es-index-cleaner` that is also used for daily indices. ```shell -docker run -it --rm --net=host -e ROLLOVER=true jaegertracing/jaeger-es-index-cleaner:latest 14 http://localhost:9200 # <1> +docker run -it --rm --net=host \ + -e ROLLOVER=true \ + jaegertracing/jaeger-es-index-cleaner:latest \ + 14 http://localhost:9200 # <1> ``` <1> Remove indices older than 14 days. @@ -97,11 +99,6 @@ docker run -it --rm --net=host -e ROLLOVER=true jaegertracing/jaeger-es-index-cl ## ILM support -{{< warning >}} -Experimental feature added in [release v1.22.0](https://github.com/jaegertracing/jaeger/releases/tag/v1.22.0). - -Supported Elasticsearch versions: 7.x -{{< /warning >}} [Elasticsearch ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) automatically manages indices according to performance, resiliency, and retention requirements. For example: @@ -116,7 +113,8 @@ To enable ILM support: older than 1m and delete indices that are older than 2m. ```shell - curl -X PUT http://localhost:9200/_ilm/policy/jaeger-ilm-policy \ + curl -X PUT \ + http://localhost:9200/_ilm/policy/jaeger-ilm-policy \ -H 'Content-Type: application/json; charset=utf-8' \ --data-binary @- << EOF { @@ -147,7 +145,10 @@ To enable ILM support: * Run elasticsearch initializer with `ES_USE_ILM=true`: ```shell - docker run -it --rm --net=host -e ES_USE_ILM=true jaegertracing/jaeger-es-rollover:latest init http://localhost:9200 # <1> + docker run -it --rm --net=host\ + -e ES_USE_ILM=true \ + jaegertracing/jaeger-es-rollover:latest \ + init http://localhost:9200 # <1> ``` <1> If you need to initialize archive storage, add `-e ARCHIVE=true`. @@ -157,7 +158,7 @@ To enable ILM support: "ILM policy jaeger-ilm-policy doesn't exist in Elasticsearch. Please create it and rerun init" {{< /info >}} - After the initialization, deploy Jaeger with `--es.use-ilm=true` and `--es.use-aliases=true`. + After the initialization, deploy Jaeger with `use_ilm: true` and `use_aliases: true`. ## Upgrading @@ -179,32 +180,35 @@ without waiting until indices created by Elasticsearch 6 are removed. 1. Reindex all span indices to new indices with suffix `-1`: - ```bash - curl -ivX POST -H "Content-Type: application/json" http://localhost:9200/_reindex -d @reindex.json - { - "source": { - "index": "jaeger-span-*" - }, - "dest": { - "index": "jaeger-span" - }, - "script": { - "lang": "painless", - "source": "ctx._index = 'jaeger-span-' + (ctx._index.substring('jaeger-span-'.length(), ctx._index.length())) + '-1'" - } - } - ``` +```bash +curl -ivX POST -H "Content-Type: application/json" \ + http://localhost:9200/_reindex -d @reindex.json +{ + "source": { + "index": "jaeger-span-*" + }, + "dest": { + "index": "jaeger-span" + }, + "script": { + "lang": "painless", + "source": "ctx._index = 'jaeger-span-' + (ctx._index.substring('jaeger-span-'.length(), ctx._index.length())) + '-1'" + } +} +``` 2. Delete indices with old mapping: ```bash - curl -ivX DELETE -H "Content-Type: application/json" http://localhost:9200/jaeger-span-\*,-\*-1 + curl -ivX DELETE -H "Content-Type: application/json" \ + http://localhost:9200/jaeger-span-\*,-\*-1 ``` 3. Create indices without `-1` suffix: ```bash - curl -ivX POST -H "Content-Type: application/json" http://localhost:9200/_reindex -d @reindex.json + curl -ivX POST -H "Content-Type: application/json" \ + http://localhost:9200/_reindex -d @reindex.json { "source": { "index": "jaeger-span-*" @@ -222,7 +226,8 @@ without waiting until indices created by Elasticsearch 6 are removed. 4. Remove suffixed indices: ```bash - curl -ivX DELETE -H "Content-Type: application/json" http://localhost:9200/jaeger-span-\*-1 + curl -ivX DELETE -H "Content-Type: application/json" \ + http://localhost:9200/jaeger-span-\*-1 ``` Run the commands analogically for other Jaeger indices. diff --git a/content/docs/next-release-v2/faq.md b/content/docs/next-release-v2/faq.md index ffc523ae..3c8bbeea 100644 --- a/content/docs/next-release-v2/faq.md +++ b/content/docs/next-release-v2/faq.md @@ -46,11 +46,11 @@ Under the hood, at the data model level, the Jaeger trace IDs are a sequence of > Does having high availability of **jaeger-collector** improve the overall system performance like decreasing the dropped span count and having the less outage for trace collection? Is it recommended? If yes, why? These are the reasons to run multiple instances: - * Your clients send so much data that a single **jaeger collector role** is not able to accept it fast enough. - * You want higher availability, e.g., when you do rolling restarts of **jaeger collector role**s for upgrade, to have some instances still running and able to process inbound data. + * Your clients send so much data that a single **jaeger-collector** is not able to accept it fast enough. + * You want higher availability, e.g., when you do rolling restarts of **jaeger-collector**s for upgrade, to have some instances still running and able to process inbound data. These are NOT the reasons to run multiple instances: - * To avoid data loss. **jaeger** drops data when the backend storage is not able to save it fast enough. Increasing the number of **jaeger collector role** instances, with more memory allocated to their internal queues, could provide a small, temporary relief, but does not remove the bottleneck of the storage backend. + * To avoid data loss. Jaeger drops data when the backend storage is not able to save it fast enough. Increasing the number of **jaeger-collector** instances, with more memory allocated to their internal queues, could provide a small, temporary relief, but does not remove the bottleneck of the storage backend. ## How do I configure authentication for Jaeger UI diff --git a/content/docs/next-release-v2/kafka.md b/content/docs/next-release-v2/kafka.md index 526ee779..5fe69304 100644 --- a/content/docs/next-release-v2/kafka.md +++ b/content/docs/next-release-v2/kafka.md @@ -3,10 +3,10 @@ title: Kafka hasparent: true --- -* Supported Kafka versions: 0.9+ +* Supported Kafka versions: 3.x -Kafka can be used as an intermediary buffer between collector and an actual storage. -**jaeger** can be configured to both act as the `collector` for Kafka data before adding it to a Kafka topic and also as a `ingester` to get data from Kafka and forward it to another storage backend. +Kafka can be used as an intermediary buffer between **collector** and an actual storage. +Jaeger can be configured to act both as the **collector** that exports trace data into a Kafka topic as well as the **ingester** to read data from Kafka and write it to a storage backend. Writing to Kafka is particularly useful for building post-processing data pipelines. @@ -15,14 +15,17 @@ Kafka also has the following officially supported resources available from the c - [Helm chart](https://artifacthub.io/packages/helm/bitnami/kafka) by Bitnami - [Strimzi Kubernetes Operator](https://strimzi.io/) -#### Configuration -##### Collector -In order to collect tracing data and add it to a Kafka topic this [configuration](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-kafka-collector.yaml) is what should be used. +## Configuration -##### Ingester -In order to write tracing data which is added to a Kafka topic this [configuration](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-kafka-ingester.yaml) is what should be used. +Please refer to these sample configuration files: + * **collector**: [config-kafka-collector.yaml](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-kafka-collector.yaml) + * **ingester**: [config-kafka-ingester.yaml](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-kafka-ingester.yaml) -#### Topic & partitions +Jaeger uses Kafka exporter and receiver from `opentelemetry-collector-contrib` repository. Please refer to their respective README's for configuration details. + * [Kafka exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/kafkaexporter/README.md) + * [Kafka receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kafkareceiver/README.md) + +## Topic & partitions Unless your Kafka cluster is configured to automatically create topics, you will need to create it ahead of time. You can refer to [the Kafka quickstart documentation](https://kafka.apache.org/documentation/#quickstart_createtopic) to learn how. You can find more information about topics and partitions in general in the [official documentation](https://kafka.apache.org/documentation/#intro_topics). [This article](https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/) provide more details about how to choose the number of partitions. diff --git a/content/docs/next-release-v2/opensearch.md b/content/docs/next-release-v2/opensearch.md index 7af25bdd..9ba5a3f5 100644 --- a/content/docs/next-release-v2/opensearch.md +++ b/content/docs/next-release-v2/opensearch.md @@ -11,7 +11,7 @@ OpenSearch maintains API compatibility between versions and remains compatible p OpenSearch does not require initialization other than [installing and running OpenSearch](https://opensearch.org/downloads.html). -Once it is running, pass the correct configuration values to **jaeger**. +Once it is running, pass the correct configuration values to Jaeger. OpenSearch also has the following officially supported resources available from the community: - [Docker container](https://hub.docker.com/r/opensearchproject/opensearch) for getting a single node up quickly @@ -20,68 +20,4 @@ OpenSearch also has the following officially supported resources available from ## Configuration -Here is [example configuration](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-opensearch.yaml) for OpenSearch. - - -### Shards and Replicas for OpenSearch indices - -Shards and replicas are some configuration values to take special attention to, because this is decided upon -index creation. [This article](https://opster.com/guides/opensearch/opensearch-capacity-planning/how-to-choose-the-correct-number-of-shards-per-index-in-opensearch) goes into -more information about choosing how many shards should be chosen for optimization. - -## Index Rollover - -[OpenSearch rollover](https://opensearch.org/docs/latest/api-reference/index-apis/rollover/) is an index management strategy that optimizes use of resources allocated to indices. -For example, indices that do not contain any data still allocate shards, and conversely, a single index might contain significantly more data than the others. -Jaeger by default stores data in daily indices which might not optimally utilize resources. Rollover feature can be enabled by `--es.use-aliases=true`. - -Rollover lets you configure when to roll over to a new index based on one or more of the following criteria: - -* `max_age` - the maximum age of the index. It uses [time units](https://www.elastic.co/guide/en/elasticsearch/reference/master/common-options.html#time-units): `d`, `h`, `m`. -* `max_docs` - the maximum documents in the index. -* `max_size` - the maximum estimated size of primary shards (since Elasticsearch 6.x). It uses [byte size units](https://www.elastic.co/guide/en/elasticsearch/reference/master/common-options.html#byte-units) `tb`, `gb`, `mb`. - -Rollover index management strategy is more complex than using the default daily indices and it requires an initialization job to prepare the storage and two cron jobs to manage indices. - -To learn more about rollover index management in Jaeger refer to this -[article](https://medium.com/jaegertracing/using-elasticsearch-rollover-to-manage-indices-8b3d0c77915d). - -### Initialize - -The following command prepares OpenSearch for rollover deployment by creating index aliases, indices, and index templates: - -```sh -docker run -it --rm --net=host jaegertracing/jaeger-es-rollover:latest init http://localhost:9200 # <1> -``` - -If you need to initialize archive storage, add `-e ARCHIVE=true`. - -After the initialization Jaeger can be deployed with `--es.use-aliases=true`. - -### Roll over - -The next step is to periodically execute the rollover API which rolls the write alias to a new index based on supplied conditions. The command also adds a new index to the read alias to make new data available for search. - -```shell -docker run -it --rm --net=host -e CONDITIONS='{"max_age": "2d"}' jaegertracing/jaeger-es-rollover:latest rollover http://localhost:9200 # <1> -``` - -<1> The command rolls the alias over to a new index if the age of the current write index is older than 2 days. For more conditions see [OpenSearch documentation](https://opensearch.org/docs/latest/api-reference/index-apis/rollover/). - -The next step is to remove old indices from read aliases. It means that old data will not be available for search. This imitates the behavior of `--es.max-span-age` flag used in the default index-per-day deployment. This step could be optional and old indices could be simply removed by index cleaner in the next step. - -```sh -docker run -it --rm --net=host -e UNIT=days -e UNIT_COUNT=7 jaegertracing/jaeger-es-rollover:latest lookback http://localhost:9200 # <1> -``` - -<1> Removes indices older than 7 days from read alias. - -### Remove old data - -The historical data can be removed with the `jaeger-es-index-cleaner` that is also used for daily indices. - -```shell -docker run -it --rm --net=host -e ROLLOVER=true jaegertracing/jaeger-es-index-cleaner:latest 14 http://localhost:9200 # <1> -``` - -<1> Remove indices older than 14 days. \ No newline at end of file +Here is [example configuration](https://github.com/jaegertracing/jaeger/blob/main/cmd/jaeger/config-opensearch.yaml) for OpenSearch. Overall Jaeger uses the same implementation for OpenSearch as for Elasticsearch, so most of the discussion on the [Elasticsearch page](../elasticsearch/#configuration) applies. diff --git a/content/docs/next-release-v2/spm.md b/content/docs/next-release-v2/spm.md index ae81a236..3924b84b 100644 --- a/content/docs/next-release-v2/spm.md +++ b/content/docs/next-release-v2/spm.md @@ -166,13 +166,26 @@ Note: The following configuration is required to enable the SPM feature: -- [Jaeger UI](../frontend-ui/#monitor) -- Jaeger Query (TODO: these instructions need to be updated for config file) - - Set the `METRICS_STORAGE_TYPE` environment variable to `prometheus`. - - Optional: Set `--prometheus.server-url` (or `PROMETHEUS_SERVER_URL` environment variable) - to the URL of the prometheus server. Default: http://localhost:9090. - - Optional: Set `--prometheus.query.support-spanmetrics-connector=true` to explicitly enable the [SpanMetrics Connector][spanmetrics-conn] if you intend to use it. This will become the default behavior in the future. - +- Set the `monitor.menuEnabled=true` property in [Jaeger UI configuration](../frontend-ui/#monitor) +- Define the configuration for a Prometheus-compatible storage under `metric_backends:` in `jaeger_storage` extension, for example: +```yaml + jaeger_storage: + backends: + some_trace_storage: + ... + metric_backends: + some_metrics_storage: + prometheus: + endpoint: http://prometheus:9090 + normalize_calls: true + normalize_duration: true +``` +- Reference this metrics store in the `jaeger_query` extension: +```yaml + jaeger_query: + traces: some_trace_storage + metrics_storage: some_metrics_storage +``` ## API diff --git a/content/docs/next-release-v2/windows.md b/content/docs/next-release-v2/windows.md index 8e8bbbf8..a4ca71a3 100644 --- a/content/docs/next-release-v2/windows.md +++ b/content/docs/next-release-v2/windows.md @@ -5,23 +5,16 @@ hasparent: true In Windows environments, Jaeger processes can be hosted and managed as Windows services controlled via the `sc` utility. To configure such services on Windows, download [nssm.exe](https://nssm.cc/download) for the appropriate architecture, and issue commands similar to how Jaeger is typically run. The example below showcases a basic Elasticsearch setup, configured using both environment variables and process arguments. -## Agent -```bat -nssm install JaegerAgent C:\Jaeger\jaeger-agent.exe --reporter.grpc.host-port=localhost:14250 - -nssm set JaegerAgent AppStdout C:\Jaeger\jaeger-agent.out.log -nssm set JaegerAgent AppStderr C:\Jaeger\jaeger-agent.err.log -nssm set JaegerAgent Description Jaeger Agent service - -nssm start JaegerAgent -``` +{{< danger >}} +This page has not yet been updated to use the new configuration file required by Jaeger v2. +{{< /danger >}} ## Collector ```bat -nssm install JaegerCollector C:\Jaeger\jaeger-collector.exe --es.server-urls=http://localhost:9200 --es.username=jaeger --es.password=PASSWORD +nssm install JaegerCollector C:\Jaeger\jaeger.exe --es.server-urls=http://localhost:9200 --es.username=jaeger --es.password=PASSWORD -nssm set JaegerCollector AppStdout C:\Jaeger\jaeger-collector.out.log -nssm set JaegerCollector AppStderr C:\Jaeger\jaeger-collector.err.log +nssm set JaegerCollector AppStdout C:\Jaeger\jaeger.out.log +nssm set JaegerCollector AppStderr C:\Jaeger\jaeger.err.log nssm set JaegerCollector Description Jaeger Collector service nssm set JaegerCollector AppEnvironmentExtra SPAN_STORAGE_TYPE=elasticsearch diff --git a/content/docs/next-release/spm.md b/content/docs/next-release/spm.md index 697765e7..65ce4ffe 100644 --- a/content/docs/next-release/spm.md +++ b/content/docs/next-release/spm.md @@ -171,8 +171,6 @@ The following configuration is required to enable the SPM feature: - Set the `METRICS_STORAGE_TYPE` environment variable to `prometheus`. - Optional: Set `--prometheus.server-url` (or `PROMETHEUS_SERVER_URL` environment variable) to the URL of the prometheus server. Default: http://localhost:9090. - - Optional: Set `--prometheus.query.support-spanmetrics-connector=true` to explicitly enable the [SpanMetrics Connector][spanmetrics-conn] if you intend to use it. This will become the default behavior in the future. - ## API diff --git a/scripts/cspell/project-words.txt b/scripts/cspell/project-words.txt index 0ff1a62d..669b8b17 100644 --- a/scripts/cspell/project-words.txt +++ b/scripts/cspell/project-words.txt @@ -7,6 +7,7 @@ certfile changeme clickhouse cncf +configtls containerd cqlsh cqlshrc