Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: add etcd_connect_options for metasrv #3853

Closed
wants to merge 1 commit into from
Closed

refactor: add etcd_connect_options for metasrv #3853

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions config/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,9 @@
| `export_metrics.remote_write` | -- | -- | -- |
| `export_metrics.remote_write.url` | String | `""` | The url the metrics send to. The url example can be: `http://127.0.0.1:4000/v1/prometheus/write?db=information_schema`. |
| `export_metrics.remote_write.headers` | InlineTable | -- | HTTP headers of Prometheus remote-write carry. |
| `etcd_connect_options` | -- | -- | The options for connecting etcd. |
| `etcd_connect_options.timeout` | String | `None` | Apply a timeout to each gRPC request. |
| `etcd_connect_options.connect_timeout` | String | `None` | Apply a timeout to connecting to the endpoint. |


### Datanode
Expand Down
9 changes: 9 additions & 0 deletions config/metasrv.example.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,3 +141,12 @@ url = ""

## HTTP headers of Prometheus remote-write carry.
headers = { }

## The options for connecting etcd.
[etcd_connect_options]
## Apply a timeout to each gRPC request.
## +toml2docs:none-default
timeout = "5s"
## Apply a timeout to connecting to the endpoint.
## +toml2docs:none-default
connect_timeout = "5s"
10 changes: 7 additions & 3 deletions src/meta-srv/src/bootstrap.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,11 @@ async fn create_etcd_client(opts: &MetasrvOptions) -> Result<Client> {
.map(|x| x.trim())
.filter(|x| !x.is_empty())
.collect::<Vec<_>>();
Client::connect(&etcd_endpoints, None)
.await
.context(error::ConnectEtcdSnafu)

Client::connect(
&etcd_endpoints,
Some(opts.etcd_connect_options.clone().into()),
)
.await
.context(error::ConnectEtcdSnafu)
}
30 changes: 30 additions & 0 deletions src/meta-srv/src/metasrv.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,10 @@ pub struct MetasrvOptions {
/// limit the number of operations in a txn because an infinitely large txn could
/// potentially block other operations.
pub max_txn_ops: usize,

/// The etcd connect options.
/// Most of the options are from `etcd_client::ConnectOptions`.
pub etcd_connect_options: EtcdConnectOptions,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This option itself can be Option? Or we define a etcd subsection for metasrv. Extending this option looks like we leak/couple the metasrv with etcd implementation.

Copy link
Collaborator Author

@zyy17 zyy17 May 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems reasonable. It also reminds me that there is more than one option for metasrv backend storage in the future(file system, memory, etcd, ...). I think we need the BackendStorageProvider and etcd is one of the providers.

For example:

[storage]
type = "etcd"
addr = "127.0.0.1:2379"
timeout = "5s"

What do you think? @fengjiachun

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems reasonable. It also reminds me that there is more than one option for metasrv backend storage in the future(file system, memory, etcd, ...). I think we need the BackendStorageProvider and etcd is one of the providers.

For example:

[storage]
type = "etcd"
addr = "127.0.0.1:2379"
timeout = "5s"

What do you think? @fengjiachun

+1

}

impl MetasrvOptions {
Expand Down Expand Up @@ -146,6 +150,7 @@ impl Default for MetasrvOptions {
export_metrics: ExportMetricsOption::default(),
store_key_prefix: String::new(),
max_txn_ops: 128,
etcd_connect_options: EtcdConnectOptions::default(),
}
}
}
Expand All @@ -156,6 +161,31 @@ impl MetasrvOptions {
}
}

#[derive(Default, Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct EtcdConnectOptions {
/// Apply a timeout to each gRPC request.
#[serde(with = "humantime_serde")]
pub timeout: Option<Duration>,

/// Apply a timeout to connecting to the endpoint.
#[serde(with = "humantime_serde")]
pub connect_timeout: Option<Duration>,
// TODO(zyy17): Add TLS options.
}

impl From<EtcdConnectOptions> for etcd_client::ConnectOptions {
fn from(opts: EtcdConnectOptions) -> Self {
let mut etcd_opts = etcd_client::ConnectOptions::new();
if let Some(timeout) = opts.timeout {
etcd_opts = etcd_opts.with_timeout(timeout);
}
if let Some(connect_timeout) = opts.connect_timeout {
etcd_opts = etcd_opts.with_connect_timeout(connect_timeout);
}
etcd_opts
}
}

pub struct MetasrvInfo {
pub server_addr: String,
}
Expand Down
Loading