Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

287 feature extract multiple columns from one prejoined object #297

Conversation

tkiehn
Copy link
Collaborator

@tkiehn tkiehn commented Jan 7, 2025

Add new syntax for prejoins, allowing extraction of multiple columns from the same prejoin-source:

  prejoined_columns:
      - extract_columns: 
           - id
           - number
        aliases:
           - businessid
           - businessnumber
        ref_model: 'business_raw'
        this_column_name: 'ContractId'
        ref_column_name: 'ContractId'

In this example we join the source model with business_raw and extract id and number and assign an alias for each column.
Assigning an alias is optional, but if done, the number of extract_columns has to match the amount of aliases.
For each prejoin-target a new list, containing at least a dictionary for extract_columns and values for ref_model(or source_name & source_table) and the column names, needs to be defined.

aliases are optional, but if given, must be the same length as extract_columns.
this_column_name & ref_column_name allow multiple columns
ref_model are changeable to src_name & src_table

The old syntax is still valid and can be used if desired.

Fixes #287

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation or included information that needs updates

ToDO: Update Docs to include the new syntax

…prejoin_column_names

change extract_input_columns to handle the new prejoin list syntax. change process_prejoined_columns to parse the old dict syntax to the new list syntax, merging prejoins with the same conditions. add extract_prejoin_column_names which can handle the new list syntax
Naming join relations using "AS" is not allowed in oracle
@tkiehn tkiehn requested a review from tkirschke January 7, 2025 11:24
@tkiehn
Copy link
Collaborator Author

tkiehn commented Jan 8, 2025

Merged into test (0557fbf)

@tkiehn tkiehn marked this pull request as draft January 8, 2025 13:07
@tkiehn tkiehn mentioned this pull request Jan 9, 2025
5 tasks
@tkiehn
Copy link
Collaborator Author

tkiehn commented Jan 9, 2025

Closed in favor of #300

@tkiehn tkiehn closed this Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Extract multiple columns from one prejoined object
1 participant