Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add results streaming to gafferpy #10

Open
t92549 opened this issue Dec 2, 2022 · 0 comments
Open

Add results streaming to gafferpy #10

t92549 opened this issue Dec 2, 2022 · 0 comments
Labels
advanced enhancement New feature or request good first issue Good for newcomers

Comments

@t92549
Copy link
Contributor

t92549 commented Dec 2, 2022

Describe the new feature you'd like
gafferpy should be able to stream results back from the rest api, probably in bulk chunks where the user can set the size

Why do you want this feature?
If very large results are returned, this would allow gafferpy users to process the results as they come, effectively utilising the lazy iterable from Accumulo. It would mean that large results that would otherwise not fit into memory can be processed in a stream.

Additional context
The /graph/operations/execute/chunked endpoint should be used to stream results back from the rest api

@t92549 t92549 added enhancement New feature or request advanced good first issue Good for newcomers labels Dec 2, 2022
@t92549 t92549 transferred this issue from gchq/gaffer-tools Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
advanced enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant