-
Notifications
You must be signed in to change notification settings - Fork 154
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Maroun Touma <[email protected]>
- Loading branch information
Showing
75 changed files
with
131 additions
and
131 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 | ||
parameterized | ||
pandas |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_code2parquet_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "code2parquet Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -10,8 +10,8 @@ authors = [ | |
{ name = "Boris Lublinsky", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-code2parquet-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
"dpk-code2parquet-transform-python==0.2.2.dev2", | ||
"parameterized", | ||
"pandas", | ||
] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 | ||
parameterized | ||
pandas | ||
aiolimiter==1.1.0 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,16 +1,16 @@ | ||
[project] | ||
name = "dpk_code_profiler_transform_ray" | ||
version = "0.2.2.dev0" | ||
requires-python = ">=3.10" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Code Profiler Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
readme = {file = "README.md", content-type = "text/markdown"} | ||
authors = [ | ||
{ name = "Pankaj Thorat", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-code-profiler-transform-python==0.2.2.dev0", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-code-profiler-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 | ||
bs4==0.0.2 | ||
transformers==4.38.2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_code_quality_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Code Quality Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,8 +9,8 @@ authors = [ | |
{ name = "Shivdeep Singh", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-code-quality-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-code-quality-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 | ||
scancode-toolkit==32.1.0 ; platform_system != 'Darwin' | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_header_cleanser_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "License and copyright removal Transform for Ray" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,8 +9,8 @@ authors = [ | |
{ name = "Yash kalathiya", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-header-cleanser-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-header-cleanser-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
"scancode-toolkit==32.1.0", | ||
] | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_license_select_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "License Select Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -10,8 +10,8 @@ authors = [ | |
{ name = "Mark Lewis", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-license-select-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-license-select-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_malware_transform_python" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Malware Python Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,7 +9,7 @@ authors = [ | |
{ name = "Takuya Goto", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"data-prep-toolkit==0.2.2.dev1", | ||
"data-prep-toolkit==0.2.2.dev2", | ||
"clamd==1.0.2", | ||
] | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_malware_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Malware Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,8 +9,8 @@ authors = [ | |
{ name = "Takuya Goto", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-malware-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-malware-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_proglang_select_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Programming Language Selection Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,8 +9,8 @@ authors = [ | |
{ name = "Shivdeep Singh", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-proglang-select-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-proglang-select-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_repo_level_order_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "repo_level_order Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -11,7 +11,7 @@ authors = [ | |
{ name = "Shanmukha Guttula", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
"networkx==3.3", | ||
"colorlog==6.8.2", | ||
"func-timeout==4.3.5", | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 | ||
docling-core==1.7.2 | ||
llama-index-core>=0.11.0,<0.12.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_doc_chunk_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "chunk documents Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -11,8 +11,8 @@ authors = [ | |
{ name = "Christoph Auer", email = "[email protected]" }, | ||
] | ||
dependencies = [ | ||
"dpk-doc-chunk-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-doc-chunk-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
|
||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_doc_quality_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Document Quality Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,8 +9,8 @@ authors = [ | |
{ name = "Daiki Tsuzuku", email = "[email protected]" } | ||
] | ||
dependencies = [ | ||
"dpk-doc_quality-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-doc_quality-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
data-prep-toolkit==0.2.2.dev1 | ||
data-prep-toolkit==0.2.2.dev2 | ||
trafilatura==1.12.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
dpk-html2parquet-transform-python==0.2.2.dev1 | ||
data-prep-toolkit[ray]==0.2.2.dev1 | ||
dpk-html2parquet-transform-python==0.2.2.dev2 | ||
data-prep-toolkit[ray]==0.2.2.dev2 | ||
trafilatura==1.12.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
[project] | ||
name = "dpk_lang_id_transform_ray" | ||
version = "0.2.2.dev1" | ||
version = "0.2.2.dev2" | ||
requires-python = ">=3.10,<3.13" | ||
description = "Language Identification Ray Transform" | ||
license = {text = "Apache-2.0"} | ||
|
@@ -9,8 +9,8 @@ authors = [ | |
{ name = "Daiki Tsuzuku", email = "[email protected]" } | ||
] | ||
dependencies = [ | ||
"dpk-lang_id-transform-python==0.2.2.dev1", | ||
"data-prep-toolkit[ray]==0.2.2.dev1", | ||
"dpk-lang_id-transform-python==0.2.2.dev2", | ||
"data-prep-toolkit[ray]==0.2.2.dev2", | ||
] | ||
|
||
[build-system] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.