Getting started 2 : Added a colab notebook, updated for local data. (#…

…572) * updated code to work with local data. Added a colab notebook - simplified python and ray notebooks to work with 'Input-Test_Data' that is part of repo - removed 'Output-Test-Data' directory. This is generated - Added a notebook that is Google Colab friendly. This one requires no local setup. And easy to run - Updated project README.md accordingly * updated file links * fixing pip install timeouts on google colab
IBM · Sep 6, 2024 · 51c8676 · 51c8676
1 parent d0a80ba
commit 51c8676
Show file tree

Hide file tree

Showing 8 changed files with 3,869 additions and 313 deletions.
diff --git a/README.md b/README.md
@@ -131,10 +131,14 @@ If there are no errors, you are good to go!
 
 Let's try a simple transform to extract content from PDF files. We have following notebooks that demonstrate how to run a data preparation transformation that extracts content from PDF files using the data-prep-kit.
 
-- Option 1: Pure python notebook : [examples/notebooks/Run_your_first_transform_python.ipynb](examples/notebooks/Run_your_first_transform_python.ipynb) - easiest to get started
-- Option 2: This one uses Ray framework for parallel execution while still allowing local processing : [examples/notebooks/Run_your_first_transform_ray.ipynb](examples/notebooks/Run_your_first_transform_ray.ipynb)
+**Notebook versions**
+
+You can try one or all 😄
+
+- Option 1: Google Colab friendly notebook (no setup necessary, easiest to get started): [examples/notebooks/Run_your_first_transform_colab.ipynb](examples/notebooks/Run_your_first_transform_colab.ipynb)  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/IBM/data-prep-kit/blob/dev/examples/notebooks/Run_your_first_transform_colab.ipynb)
+- Option 2: Pure python notebook (runs locally) : [examples/notebooks/Run_your_first_transform_python.ipynb](examples/notebooks/Run_your_first_transform_python.ipynb) - easiest to get started
+- Option 3: Ray version (runs locally): This one uses Ray framework for parallel execution while still allowing local processing - [examples/notebooks/Run_your_first_transform_ray.ipynb](examples/notebooks/Run_your_first_transform_ray.ipynb)
 
-You can try either one, or both 😄
 
 To run the notebooks, launch jupyter from the same virtual environment you created using the command below. 
 

diff --git a/examples/notebooks/.gitignore b/examples/notebooks/.gitignore
@@ -1,2 +1 @@
-Input-Test-Data
-Output-Test-Data
+Output-Test-Data*
diff --git a/examples/notebooks/Input-Test-Data/attention_is_all_you_need.pdf b/examples/notebooks/Input-Test-Data/attention_is_all_you_need.pdf
diff --git a/examples/notebooks/Output-Test-Data/GraniteCodePaper.parquet b/examples/notebooks/Output-Test-Data/GraniteCodePaper.parquet
diff --git a/examples/notebooks/Output-Test-Data/metadata.json b/examples/notebooks/Output-Test-Data/metadata.json