Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CBS data missing from external_features #110

Open
balazsdukai opened this issue Dec 1, 2024 · 0 comments
Open

CBS data missing from external_features #110

balazsdukai opened this issue Dec 1, 2024 · 0 comments

Comments

@balazsdukai
Copy link
Member

dagster._core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "external_features":

  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/execute_plan.py", line 245, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/execute_step.py", line 499, in core_dagster_event_sequence_for_step
    for user_event in _step_output_error_checked_user_event_sequence(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/execute_step.py", line 183, in _step_output_error_checked_user_event_sequence
    for user_event in user_event_sequence:
                      ^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/execute_step.py", line 87, in _process_asset_results_to_events
    for user_event in user_event_sequence:
                      ^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/compute.py", line 193, in execute_core_compute
    for step_output in _yield_compute_results(step_context, inputs, compute_fn, compute_context):
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/compute.py", line 162, in _yield_compute_results
    for event in iterate_with_context(
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_utils/__init__.py", line 480, in iterate_with_context
    with context_fn():
         ^^^^^^^^^^^^
  File "/root/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/utils.py", line 84, in op_execution_error_boundary
    raise error_cls(

The above exception was caused by the following exception:
psycopg.errors.UndefinedTable: relation "floors_estimation.cbs_key_figures_districts_neighbourhoods_2023" does not exist
LINE 138: FROM floors_estimation.cbs_key_figures_districts_neighbourho...
               ^

  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_utils/__init__.py", line 482, in iterate_with_context
    next_output = next(iterator)
                  ^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/compute_generator.py", line 140, in _coerce_op_compute_fn_to_iterator
    result = invoke_compute_fn(
             ^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/dagster/_core/execution/plan/compute_generator.py", line 128, in invoke_compute_fn
    return fn(context, **args_to_pass) if context_arg_provided else fn(**args_to_pass)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/packages/floors_estimation/src/bag3d/floors_estimation/assets/floors_estimation.py", line 161, in external_features
    metadata = postgrestable_from_query(context, query, external_features_table)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/packages/common/src/bag3d/common/utils/database.py", line 79, in postgrestable_from_query
    conn.send_query(query)
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/pgutils/connection.py", line 297, in send_query
    return conn.execute(_q)
           ^^^^^^^^^^^^^^^^
  File "/opt/3dbag-pipeline/venv/lib/python3.12/site-packages/psycopg/connection.py", line 251, in execute
    raise ex.with_traceback(None)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant