Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(inverted_index.search): add index applier #2868

Merged
merged 35 commits into from
Dec 5, 2023

Conversation

zhongzc
Copy link
Contributor

@zhongzc zhongzc commented Dec 4, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Introduce IndexApplier to apply the predefined predicates to the data read by the given index reader, returning a list of relevant indices.

Applier instances are reusable and work with various InvertedIndexReader instances, avoiding repeated compilation of fixed predicates such as regex patterns.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

Prerequisite merging GreptimeTeam/greptime-proto#126

#2705

Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
@zhongzc zhongzc self-assigned this Dec 4, 2023
Copy link

codecov bot commented Dec 4, 2023

Codecov Report

Merging #2868 (c05c99e) into develop (6344b1e) will decrease coverage by 0.46%.
Report is 3 commits behind head on develop.
The diff coverage is 85.37%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2868      +/-   ##
===========================================
- Coverage    84.78%   84.33%   -0.46%     
===========================================
  Files          745      749       +4     
  Lines       117025   117881     +856     
===========================================
+ Hits         99223    99417     +194     
- Misses       17802    18464     +662     

@zhongzc zhongzc force-pushed the zhongzc/index-applier branch from 74c2c67 to 91a1338 Compare December 5, 2023 05:17
Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! A few suggestions attached.
Another suggestion: maybe we can add some benchmarks for inverted index performance to get a baseline and prepare for future improvements.

@zhongzc
Copy link
Contributor Author

zhongzc commented Dec 5, 2023

Another suggestion: maybe we can add some benchmarks for inverted index performance to get a baseline and prepare for future improvements.

Agreed. I will construct some bench cases with local files as the baseline in the upcoming PRs.

Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Zhenchi <[email protected]>
Copy link
Contributor

@evenyag evenyag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

@zhongzc zhongzc enabled auto-merge December 5, 2023 08:16
@zhongzc zhongzc added this pull request to the merge queue Dec 5, 2023
Merged via the queue into GreptimeTeam:develop with commit 0b421b5 Dec 5, 2023
13 checks passed
@zhongzc zhongzc deleted the zhongzc/index-applier branch December 5, 2023 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants