Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the "setup" notebook functional #917

Open
6 of 12 tasks
promisinganuj opened this issue Dec 4, 2024 · 1 comment · Fixed by #1008
Open
6 of 12 tasks

Make the "setup" notebook functional #917

promisinganuj opened this issue Dec 4, 2024 · 1 comment · Fixed by #1008
Assignees
Labels
e2e: fabric Related with E2E Fabric Sample P1 High Priority

Comments

@promisinganuj
Copy link
Contributor

promisinganuj commented Dec 4, 2024

The initial version of the "setup" notebook is already deployed using Terraform Fabric provider. This item is to make this notebook functional by performing the following:

Depends on merging of #862

DoD

  • Address the issue where the shortcut can't be accessed without it being accessed once (some sort of authentication issues).
    import os, time
    
    config = configparser.ConfigParser(interpolation=configparser.ExtendedInterpolation())
    config.read(f"{local_data_mount_path}/{config_file_path}")
    
  • Migrate some of the comments from the notebook and move it to the markdown in GitHub.
  • Decide and update the usage of default lakehouse vs direct account via mounts.
  • Review the code and see if we can move some of the content as a library. #1007
  • Update the main README.md.
    • Add "How to use the sample" section.
    • Explain the manual process of uploading config/reference data to ADLS gen2.
    • Explain the manual process of uploading custom python library/environment.yml to Fabric environment.
    • Explain the steps to run the notebook.
    • Explain the output of the run. i.e., tables created, reference data loaded.

Out of scope

  • Automation of uploading config/reference data to the ADLS gen2 storage.
  • Running of the notebook via Fabric pipeline (though it's already integrated).
@promisinganuj promisinganuj added P1 High Priority e2e: fabric Related with E2E Fabric Sample labels Dec 4, 2024
@naga-nandyala
Copy link
Contributor

workaround is to list the contents of local_data_mount_path.

import os
print(os.listdir(local_data_mount_path))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
e2e: fabric Related with E2E Fabric Sample P1 High Priority
Projects
None yet
2 participants