Release 0.5.0 #205
Open
Release 0.5.0 #205
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added access to gpus when running docker. This allows the components related to the causal intervention case to be run. In the future, we can add this flag as optional depending on the user configuration or the `odtp.yml` requirements.
`instance_name` was modified to `container_name`; however, these changes did not propagate to components cli. Also, secrets compatibility has been added.
Here, I added the option to use prebuilt images from dockerhub, or github images registry. The CLI can be tested with: Prepare components command: ``` bash odtp component prepare \ --folder /c/Users/Carlos/pro/odtp_dts/odtp_v1/test \ --image_name odtp-component-example \ --repository https://github.com/odtp-org/odtp-component-example \ --image_link caviri/odtp-component-example:v0.1.6 ``` Adding components. ``` bash odtp new odtp-component-entry \ --name odtp-component-example \ --component-version v0.1.1 \ --repository https://github.com/odtp-org/odtp-sql-dataloader \ --image-link caviri/odtp-component-example ``` Additionally, the digital twins' executions can be tested easily by using a `docker-compose.yml` that runs the different steps https://github.com/odtp-org/dt-mobility-causal-intervention/blob/main/docker-compose.yml
Co-authored-by: sabinem <[email protected]>
* feat: adpat cli and mongodb to parsing of odtp.yml for components - workflow collection is added - cli for new workflow is added - cli for new execution is adapted - a field deprecated is added on most collections * feat: add db function to update documents as deprecated * fix: excution runs on docker installations without gpus flag * feat(cli): allow to create workflow and execution in one go --------- Co-authored-by: sabinem <[email protected]>
* feat(run): Added access to gpus by default * feat(cli-component): Added secrets and wrong parameter name fixed * feat(dockerhub): Compatibility with dockerhub images in CLI * fix(workflow.py): imageLink is an optional value on execution (#210) Co-authored-by: sabinem <[email protected]> * Feat/adapt mongodb to odtp yml (#211) * feat: adpat cli and mongodb to parsing of odtp.yml for components - workflow collection is added - cli for new workflow is added - cli for new execution is adapted - a field deprecated is added on most collections * feat: add db function to update documents as deprecated * fix: excution runs on docker installations without gpus flag * feat(cli): allow to create workflow and execution in one go --------- Co-authored-by: sabinem <[email protected]> * feat: component version add form get version info from odtp.yml file on github --------- Co-authored-by: Carlos Vivar Rios <[email protected]> Co-authored-by: Carlos Vivar Rios <[email protected]> Co-authored-by: sabinem <[email protected]>
…214) feat: add workflow page to the GUI Co-authored-by: sabinem <[email protected]>
* chore: remove debug print statments * style: improve tab header --------- Co-authored-by: sabinem <[email protected]>
Co-authored-by: sabinem <[email protected]>
chore:cleanup code style Co-authored-by: sabinem <[email protected]>
feat: filter out deprecated components on workflow add Co-authored-by: sabinem <[email protected]>
* feat:align workflow table: headers to content * feat: align headers and columns in component table --------- Co-authored-by: sabinem <[email protected]>
* feat:align workflow table: headers to content * feat: align headers and columns in component table --------- Co-authored-by: sabinem <[email protected]>
fix: gui homepage: error reporting and blanks Co-authored-by: sabinem <[email protected]>
* style: remove blanks at end of code lines * chore: cleanup user page create user directory from user name on ODTP_PATH validate github user name and uniqueness of user name in mongo db --------- Co-authored-by: sabinem <[email protected]>
chore: digital twin page improve storage handling, table, flow Co-authored-by: sabinem <[email protected]>
embed forms in try expect blocks Co-authored-by: sabinem <[email protected]>
chore: remove debug print statements Co-authored-by: sabinem <[email protected]>
feat: add links to digital twin table so that it is possible to select Co-authored-by: sabinem <[email protected]>
refactor: improve flow and style in digital twin page * store important properties in current_digital_twin * align columns in table Co-authored-by: sabinem <[email protected]>
feat: allow to link to workflows add argument to workflow page, so that it can be called with a workflow id Co-authored-by: sabinem <[email protected]>
fix: layout problems in workflow table and user header Co-authored-by: sabinem <[email protected]>
Co-authored-by: sabinem <[email protected]>
* chore: remove blanks and unneeded imports * chore: remove blanks at end of line * feat: add exception for failed docker runs --------- Co-authored-by: sabinem <[email protected]>
feat: allow for partial execution runs * run_step flag is added to each step and set to True on the creation of an execution * a update_step function is added to the mongodb to update run_step flag and other properties of steps Co-authored-by: sabinem <[email protected]>
refactor execution run page in dashboard * allow partial execution run by setting flags of whether a step should run * workflow and run code is adapted to consider that `run_step` flag when executing run commands * open: secrets handeling in gui: not yet integrated * cleanup unused code Co-authored-by: sabinem <[email protected]>
* fix: project directory: make sure it exists before deleting it * feat: add cryptography for encrypting secrets * feat: steps should know whether or not they need secrets * fix: model for validating secrets in odtp.yml * feat: make workarea a class on user page * feat: add secrets setup secrets will be stored as encrypted files per user. The files will be added for each docker run where the the step run a component version that has a secret specified in its odtp.yml file --------- Co-authored-by: sabinem <[email protected]>
* fix: offer link from table to detail page when workflow is not deprecated * fix: bugs on workflow detail page workflow info is now showing also initially after the workflow selection --------- Co-authored-by: sabinem <[email protected]>
* feat: allow query in db get collection * fix: digital twin name must only be unique for user * fix: execution name must only be unique within digital twin --------- Co-authored-by: sabinem <[email protected]>
* feat: add secrets to step in gui from an encrypted file * feat: decrypt secrets on run workflow from encrypted file if the step has a secret path set, the secret file is decrypted before running the step secrets are removed from the cli prepare execution command --------- Co-authored-by: sabinem <[email protected]>
render all available properties, that seem to be of interest Co-authored-by: sabinem <[email protected]>
fix: delete temporary code file added during development Co-authored-by: sabinem <[email protected]>
feat: implement links for component versions and workflows So far the links use the id of the workflows or component versions in order to be sure the id is unique Co-authored-by: sabinem <[email protected]>
* feat: prepare run and workflow for capability of rerunning steps steps clear their directory and update their error before they actually start running. On failure the exit error is added to the step error * feat: add reusable functions to display a component versions sections these functions will be reused by component versions, workflows and executions * feat: on the user's page show when a secrets file has been added * refactor: on workflow page use reusable version display fundtions * refactor: on workflow page use reusable functions to display version * feat: add links from digital twins to executions * feat: finish execution page fix bugs and add improve page flow * fix: make directory clean up safe * fix: error message when execution run fails * fix: layout of execution parameters --------- Co-authored-by: sabinem <[email protected]>
* feat: on user page allow upload of different secret files * feat: adapt secrets handeling * secret files can be added by the user * on the execution the files can be chosen and attached to a step so that different steps run with different secret files --------- Co-authored-by: sabinem <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the PR for the release
0.5.0