Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: Minor Postgres optimizations #2166

Merged
merged 6 commits into from
Oct 30, 2023
Merged

chore: Minor Postgres optimizations #2166

merged 6 commits into from
Oct 30, 2023

Conversation

Ivansete-status
Copy link
Collaborator

Description

This PR is intended to add some enhancements regarding the use of parallel connections.

Changes

  • Increase the max limit of connections from 5 to 50 (hard-coded.)
  • Split the connection pool in two so that we ensure the select/read connection can benefit from the caching mechanisms provided by Postgres.
  • Increase the Postgres health check time from 30'' to 60'' (waku/waku_archive/driver/postgres_driver/postgres_healthcheck.nim.)

Issue

#1842

@github-actions
Copy link

This PR may contain changes to database schema of one of the drivers.

If you are introducing any changes to the schema, make sure the upgrade from the latest release to this change passes without any errors/issues.

Please make sure the label release-notes is added to make sure upgrade instructions properly highlight this change.

@Ivansete-status
postgres_healthcheck: valide once per minute instead of 30 sec
d177177
@Ivansete-status
postgres_driver.nim: change MaxNumCons from 5 to 50
0ee4293
@Ivansete-status
postgres_driver.nim: split connPool into writeConPool and readConPool
cfb2a5d 
This aims to avoid clashes in insert and select queries
because the inserts and selects can happen concurrently
in relay and store events, respectively.
@Ivansete-status Ivansete-status force-pushed the optimize-postgres-simple branch from 8a56d00 to cfb2a5d Compare October 27, 2023 14:49
@github-actions
Copy link

github-actions bot commented Oct 27, 2023 •
edited
Loading

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:2166

Built from ddfbd46

@Ivansete-status Ivansete-status marked this pull request as ready for review October 27, 2023 15:24
@Ivansete-status Ivansete-status self-assigned this Oct 27, 2023
SionoiS
SionoiS approved these changes Oct 27, 2023
View reviewed changes
Copy link
Contributor

@SionoiS SionoiS left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

If I understand correctly this would increase max connection count from 5 to 100 (50x2) right?

@Ivansete-status
Copy link
Collaborator Author

If I understand correctly this would increase max connection count from 5 to 100 (50x2) right?

Yes, the database has 100 as max connections by default.

gabrielmer
gabrielmer reviewed Oct 30, 2023
View reviewed changes
Copy link
Contributor

@gabrielmer gabrielmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much! Looks great!
Added a few questions :):)

waku/waku_archive/driver/postgres_driver/postgres_driver.nim Outdated Show resolved Hide resolved
waku/waku_archive/driver/postgres_driver/postgres_driver.nim Show resolved Hide resolved
vpavlin
vpavlin approved these changes Oct 30, 2023
View reviewed changes
Copy link
Member

@vpavlin vpavlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

gabrielmer
gabrielmer approved these changes Oct 30, 2023
View reviewed changes
Copy link
Contributor

@gabrielmer gabrielmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks so much! :):)

@Ivansete-status Ivansete-status merged commit 282c2e8 into master Oct 30, 2023
9 of 10 checks passed
@Ivansete-status Ivansete-status deleted the optimize-postgres-simple branch October 30, 2023 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gabrielmer gabrielmer gabrielmer approved these changes

@SionoiS SionoiS SionoiS approved these changes

@vpavlin vpavlin vpavlin approved these changes

@alrevuelta alrevuelta Awaiting requested review from alrevuelta

@NagyZoltanPeter NagyZoltanPeter Awaiting requested review from NagyZoltanPeter

Assignees

@Ivansete-status Ivansete-status

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Ivansete-status @vpavlin @SionoiS @gabrielmer