Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for projection and filtering to python SDK remote store catalog queries #8629

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

zehiko
Copy link
Contributor

@zehiko zehiko commented Jan 9, 2025

What

Title.

Note: unrelated to this PR it seems that recording_type (which is an arrow dict type) breaks this example script. Instead of investigating why I think it might be better to change this type to string as we already discussed.

Testing done

# project recording id column (note that row_id and timestamp columns are always projected)
$ pixi run -e examples -- python examples/python/remote/metadata.py print --columns rerun_recording_id --recording-ids dd3d7c47-3f04-4d12-9d9b-69b223ee9f56
┌────────────────────────────┬─────────────────────────────────┬─────────────────────────────────┐
│ rerun_registration_time    ┆ row_id                          ┆ rerun_recording_id              │
│ ---                        ┆ ---                             ┆ ---                             │
│ datetime[ns]               ┆ struct[2]                       ┆ str                             │
╞════════════════════════════╪═════════════════════════════════╪═════════════════════════════════╡
│ 2025-01-09 11:24:19.678990 ┆ {1736421859678872875,349829686… ┆ dd3d7c47-3f04-4d12-9d9b-69b223… │
└────────────────────────────┴─────────────────────────────────┴─────────────────────────────────┘

# project storage_url
$ pixi run -e examples -- python examples/python/remote/metadata.py print --columns rerun_storage_url --recording-ids dd3d7c47-3f04-4d12-9d9b-69b223ee9f56

┌─────────────────────────────────┬────────────────────────────┬─────────────────────────────────┐
│ row_id                          ┆ rerun_registration_time    ┆ rerun_storage_url               │
│ ---                             ┆ ---                        ┆ ---                             │
│ struct[2]                       ┆ datetime[ns]               ┆ str                             │
╞═════════════════════════════════╪════════════════════════════╪═════════════════════════════════╡
│ {1736421859678872875,349829686… ┆ 2025-01-09 11:24:19.678990 ┆ file:///tmp/structure_from_mot… │
└─────────────────────────────────┴────────────────────────────┴─────────────────────────────────┘
# select recording id that doesn't exist
$ pixi run -e examples -- python examples/python/remote/metadata.py print --columns row_id rerun_description --recording-ids 12345

┌┐
╞╡
└┘

Copy link

github-actions bot commented Jan 9, 2025

Web viewer built successfully. If applicable, you should also test it:

  • I have tested the web viewer
Result Commit Link Manifest
e4b4c05 https://rerun.io/viewer/pr/8629 +nightly +main

Note: This comment is updated whenever you push a commit.

@zehiko zehiko self-assigned this Jan 9, 2025
@zehiko zehiko added exclude from changelog PRs with this won't show up in CHANGELOG.md remote-store remote store gRPC API labels Jan 9, 2025
Base automatically changed from zehiko/fix-id to main January 10, 2025 07:24
@zehiko zehiko requested a review from jleibs January 10, 2025 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exclude from changelog PRs with this won't show up in CHANGELOG.md remote-store remote store gRPC API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant