From b194a503db48d44df1a1dd481e8836a370bb660e Mon Sep 17 00:00:00 2001 From: nsheff Date: Thu, 23 May 2024 15:24:04 -0400 Subject: [PATCH 1/9] streamline pephub docs --- docs/pephub/README.md | 47 ++++++------------- .../developer/{organization.md => setup.md} | 0 docs/pephub/user/accounts.md | 15 +----- docs/pephub/user/getting-started.md | 7 ++- docs/pephub/user/pops.md | 6 +++ docs/pephub/user/uploading.md | 6 +-- docs/pephub/user/views.md | 4 ++ mkdocs.yml | 11 +++-- 8 files changed, 43 insertions(+), 53 deletions(-) rename docs/pephub/developer/{organization.md => setup.md} (100%) create mode 100644 docs/pephub/user/views.md diff --git a/docs/pephub/README.md b/docs/pephub/README.md index 9f4ead5b..397c74d2 100644 --- a/docs/pephub/README.md +++ b/docs/pephub/README.md @@ -7,38 +7,21 @@ -PEPhub is a database, web interface, and API for sharing, retrieving, and validating sample metadata. PEPhub takes advantage of the Portable Encapsulated Projects (PEP) biological metadata standard to store, edit, and access your PEPs in one place. PEPhub consists of two components: +PEPhub is an open-source database, web interface, and API for sharing, retrieving, and validating sample metadata. PEPhub consists of: -- **Deployed public user interface**: https://pephub.databio.org/ +- **Public user interface**: https://pephub.databio.org/ - **API**: https://pephub-api.databio.org/api/v1/docs ----- - - -## How to use this documentation -We have two types of documentation for PEPhub: **user** and **developer** documentation. The user documentation is intended for users who want to learn how to use PEPhub. The developer documentation is intended for developers who want to contribute to the PEPhub codebase or build on top of the PEPhub API and deploy their own instance. - -### Users -- **[Getting started](user/getting-started.md)**: Learn how to create an account, log in, and upload your first PEP. -- **[Creating an account and logging in](user/accounts.md)**: Learn how to create an account and log in to PEPhub. -- **[Uploading your first PEP](user/uploading.md)**: Learn how to upload your first PEP to PEPhub. - -- **[Use a PEP in an existing pipeline](user/pipelines.md)**: Learn how to incorporate PEPhub into an existing pipeline. - - -### Developers -- **[Setup](developer/organization.md)**: Learn how to set up the PEPhub development environment. -- **[Development](developer/api.md)**: Learn about developing the PEPhub API and user interface. -- **[Deployment](developer/deployment.md)**: Learn how to deploy PEPhub. -- **[Authentication](developer/authentication.md)**: Learn about the authentication methods used in PEPhub. -- **[Device authentication](developer/authentication-device.md)**: Learn about the device authentication method used in PEPhub. -- **[Semantic search](developer/semantic-search.md)**: Learn about the semantic search functionality in PEPhub. -- **[Server settings](developer/server-settings.md)**: Learn about the server settings in PEPhub. -- **[PEPembed](developer/pepembed/README.md)**: Learn about `pepembed`, the PEP embedding tool for PEPhub. -- **[pepdbagent](developer/pepdbagent/README.md)**: Learn about `pepdbagent`, the database driver for PEPhub. -- **[PEPhubClient](developer/pephubclient/README.md)**: Learn about `PEPhubClient`, the command-line tool and Python API for PEPhub. -- **[geopephub](developer/geopephub/README.md)**: Learn about PEPhubs GEO namespace for the gene expression omnibus. \ No newline at end of file + +Documentation for PEPhub is split in two, choose which you need: + +
+ +- :fontawesome-regular-user: [**User guide**](user/getting-started.md) + Teaches you how to use PEPhub to manage, share, and validate your sample metadata. + +- :fontawesome-solid-code: [**Developer guide**](developer/setup.md) + Teaches you how to contribute to PEPhub, build tools on the PEPhub API, or deploy your own instance. + +
+ diff --git a/docs/pephub/developer/organization.md b/docs/pephub/developer/setup.md similarity index 100% rename from docs/pephub/developer/organization.md rename to docs/pephub/developer/setup.md diff --git a/docs/pephub/user/accounts.md b/docs/pephub/user/accounts.md index 370a7363..0ae2e3d3 100644 --- a/docs/pephub/user/accounts.md +++ b/docs/pephub/user/accounts.md @@ -1,18 +1,7 @@ # Logging into PEPhub -*Learn about how PEPhub manages user accounts and authentication.* -## Logging into PEPhub -To login to PEPhub you will need a GitHub account. This also allows us to leverage GitHub's OAuth system for secure authentication and namespacing. If you don't have a GitHub account, you can create one [here](https://github.com/signup). - -Once you have a GitHub account, you can log in to PEPhub: - -1. Go to the [PEPhub home page](https://pephub.databio.org). -2. Click the "Login" button in the top right corner. -3. You will be redirected to GitHub to authorize the PEPhub application. +PEPhub accounts are linked to GitHub. This allows us to leverage GitHub's OAuth system for secure authentication and namespacing. Once you have a [GitHub account](https://github.com/signup), you can log in to PEPhub. Just click the "Login" button in the top right corner of the [PEPhub home page](https://pephub.databio.org). ![PEPhub login button](../img/login.png) -## You are now logged in! -Now you can start uploading your first PEP to PEPhub. - -[Uploading your first PEP](/pephub/user/uploading){ .md-button .md-button--primary } \ No newline at end of file +You will be redirected to GitHub to authorize the PEPhub application. You are now logged in and can upload your first PEP! diff --git a/docs/pephub/user/getting-started.md b/docs/pephub/user/getting-started.md index ff1fa03d..820fa423 100644 --- a/docs/pephub/user/getting-started.md +++ b/docs/pephub/user/getting-started.md @@ -1,6 +1,7 @@ # Getting started with PEPhub -*Use PEPhub to collaborate on sample-intensive biological projects.* + ## What is PEPhub? + Just like GitHub allows you to share and edit projects that you are tracking with `git`, PEPhub allows you to share and edit **biological metadata** that is formatted as a **PEP** (a Portable Encapsulated Project). PEPhub allows you to: - **Upload** your metadata to a central database @@ -8,14 +9,16 @@ Just like GitHub allows you to share and edit projects that you are tracking wit - **Share** your metadata with collaborators ## What is a PEP? + Portable Encapsulated Projects (PEPs) are standard format for biological sample metadata. A PEP is simply a **yaml** + **csv** file (or just csv) -- the CSV file is a sample table, while the YAML file provides project-level metadata and sample modifiers. PEPs are a common input for running workflows using tools such as [Snakemake](https://snakemake.readthedocs.io/en/stable/), [Common Workflow Language](https://www.commonwl.org/), [Looper](http://pep.databio.org/looper), and other workflow systems. For more details, read the [PEP specification](http://pep.databio.org/spec/simple-example). ## How PEPhub and PEPs work together + Storing files in a central location is a key feature of PEPhub. Local, individualized storage of PEPs can quickly become intractable for large teams. To that end, PEPhub allows you to store these in a database that is accessible to anyone through a web interface. This allows for easy sharing and collaboration on projects. In short, PEPs are the standard format for biological metadata, and PEPhub is the platform that allows you to store, edit, and share these PEPs. Through the API, one can easily access and retrieve PEPs for use in workflows and analyses. ## Ready to get started? + To get started storing metadata on PEPhub, you will need to log in. Click the button below to learn more about getting an account and logging in. -[Logging into PEPhub](/pephub/user/accounts){ .md-button .md-button--primary } \ No newline at end of file diff --git a/docs/pephub/user/pops.md b/docs/pephub/user/pops.md index e69de29b..c564b532 100644 --- a/docs/pephub/user/pops.md +++ b/docs/pephub/user/pops.md @@ -0,0 +1,6 @@ +# POPs + +POPs are PEPs of PEPs. + +Documentation forthcoming. + diff --git a/docs/pephub/user/uploading.md b/docs/pephub/user/uploading.md index 0dfa6115..fb02e9e2 100644 --- a/docs/pephub/user/uploading.md +++ b/docs/pephub/user/uploading.md @@ -1,17 +1,17 @@ # Uploading PEPs to PEPhub -*Learn about how you can start adding PEPs to your namespace on PEPhub.* There are two mains ways to add a PEP to your PEPhub namespace: you can either [upload a PEP directly](#uploading-a-pep), or you can [create a new PEP from scratch](#creating-a-new-pep-from-scratch) using the web interface. This guide will walk you through both methods. ## Uploading a PEP + Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username}`) and click the "Add" button in the top right. Click the "Upload PEP" tab. You will be prompted to select a PEP file from your local machine. Fill in the details about your PEP and then either drag files to the drop zone or click the drop zone to select files from your computer. Click "Submit" to add the PEP to your namespace. ## Creating a new PEP from scratch + Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username}`) and click the "Add" button in the top right. Click the "Blank PEP" tab. Again, fill in the details about your PEP and then you can start filling in the sample table. Click "Submit" to add the PEP to your namespace. ![Submission form for a new PEP](../img/add-pep-form.png) ## Ready to edit your PEP -Once you have uploaded or created a PEP, you can now start using it in pipelines! -[Use your PEP in a pipeline](/pephub/user/pipelines){ .md-button .md-button--primary } \ No newline at end of file +Once you have uploaded or created a PEP, you can now start using it in pipelines! diff --git a/docs/pephub/user/views.md b/docs/pephub/user/views.md new file mode 100644 index 00000000..81f32245 --- /dev/null +++ b/docs/pephub/user/views.md @@ -0,0 +1,4 @@ +# How to use PEPhub views + +*Documentation pending* + diff --git a/mkdocs.yml b/mkdocs.yml index 894a0125..4db4b308 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -40,8 +40,12 @@ theme: markdown_extensions: - attr_list + - md_in_html - pymdownx.highlight: use_pygments: true + - pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg - pymdownx.superfences: custom_fences: - name: mermaid @@ -176,17 +180,18 @@ nav: - PEPhub: pephub/README.md - User guide: - Getting started: pephub/user/getting-started.md - - Creating an account and logging in: pephub/user/accounts.md + - Logging in: pephub/user/accounts.md - Uploading your first PEP: pephub/user/uploading.md # - Editing a PEP: pephub/user/editing.md # - Sharing your PEP: pephub/user/sharing.md + - How to use views: pephub/user/views.md - Use a PEP in an existing pipeline: pephub/user/pipelines.md # - Validating PEPs: pephub/user/validation.md # - Version control: pephub/user/version-control.md # - Semantic search: pephub/user/semantic-search.md - # - Building POPs: pephub/user/pops.md + - PEP of PEPs (POP): pephub/user/pops.md - Developer guide: - - Setup: pephub/developer/organization.md + - Setup: pephub/developer/setup.md - Deployment: pephub/developer/deployment.md - Development: pephub/developer/development.md - Authentication: pephub/developer/authentication.md From b5312a9ba9adfd46fed1b43ea79d5b2902fc8093 Mon Sep 17 00:00:00 2001 From: nsheff Date: Thu, 23 May 2024 16:06:37 -0400 Subject: [PATCH 2/9] major work and restructure on pephub docs --- docs/pephub/README.md | 19 +- docs/pephub/changelog.md | 428 ++++++++++++++++++ .../developer/pepdbagent/db_tutorial.md | 2 +- docs/pephub/user/accounts.md | 7 - docs/pephub/user/editing.md | 0 docs/pephub/user/geo.md | 8 + docs/pephub/user/getting-started.md | 42 +- docs/pephub/user/overview.md | 45 -- docs/pephub/user/sharing.md | 0 docs/pephub/user/uploading.md | 8 + mkdocs.yml | 13 +- 11 files changed, 507 insertions(+), 65 deletions(-) create mode 100644 docs/pephub/changelog.md delete mode 100644 docs/pephub/user/accounts.md delete mode 100644 docs/pephub/user/editing.md create mode 100644 docs/pephub/user/geo.md delete mode 100644 docs/pephub/user/overview.md delete mode 100644 docs/pephub/user/sharing.md diff --git a/docs/pephub/README.md b/docs/pephub/README.md index 397c74d2..d86b62cd 100644 --- a/docs/pephub/README.md +++ b/docs/pephub/README.md @@ -13,7 +13,24 @@ PEPhub is an open-source database, web interface, and API for sharing, retrievin - **API**: https://pephub-api.databio.org/api/v1/docs -Documentation for PEPhub is split in two, choose which you need: +## Features at-a-glance + + +- **Validation**. PEPhub validates sample metadata with [eido](../eido/README.md). Users can specify a schema to which the PEP should adhere. All schemas are available on the official website: [https://schema.databio.org/](https://schema.databio.org/). Schemas are particularly useful before running pipelines, as validation provides essential information about PEP compatibility with specific pipelines and highlights any errors in the PEP structure. + +- **Semantic search**. PEPhub has semantic search functionality based on cutting-edge semantic machine learning. Information from each PEP is encoded using a sentence transformer and stored in a fast vector database. The PEPhub search interface then provides an extremely fast and powerful semantic search of sample metadata. + +- **Authorization**. PEPhub has a robust user authorization system to allow users to submit and edit their own PEPs. Users authenticate via GitHub, and then may upload, modify, and delete PEPs, and star projects. You can also set projects as private to restrict access. PEPhub also provides group-level permissions using GitHub organization membership, providing organizational namespaces that correspond to GitHub organizations to make it possible to collaborate on PEPs. + +- **Group PEPs with using a PEP of PEPs (POP)**. A PEP of PEPs, or simply a POP, is a specific type of PEP in which each row is itself a PEP. Essentially, a POP is a structure to group PEPs, allowing users to organize projects. This allows PEPs related to a specific topic to be consolidated, streamlining organization and accessibility. + +- **Re-processing of GEO metadata**. The public PEPhub instance [geo namespace](https://pephub.databio.org/geo) holds metadata from nearly 99% of the [Gene Expression Omnibus](https://www.ncbi.nlm.nih.gov/geo/). PEPhub is updated weekly using [GEOfetch](../geofetch/README.md) to produce standardized PEP sample tables, providing a convienient API interface to GEO metadata. + +- **PEPHubClient (phc)**. PEPhubClient is a command-line tool and Python API, which allows users to authenticate with PEPhub, download and upload public or private projects. For more information, see the [PEPHubClient documentation](developer/pephubclient/README.md). + +## Next steps + +Choose your adventure:
diff --git a/docs/pephub/changelog.md b/docs/pephub/changelog.md new file mode 100644 index 00000000..0db8121e --- /dev/null +++ b/docs/pephub/changelog.md @@ -0,0 +1,428 @@ +# Changelog + +This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format. + +## [0.11.6] - 02-26-2024 + +- Fix some bugs introduced as a result of the last release: + - tag's were being removed from the URL params when selecting a project view + - stabilized query params along the way on the namespace page + +## [0.11.7] - 02-22-2024 + +- Added interface for selecting and viewing project views +- optimized loading of very large sample tables + +## [0.11.6] - 02-08-2024 + +### Fixed + +- Docs and docs links +- Bug in handsontable +- Response errors in samples and views + +### Added + +- Namespace endpoint + +## [0.11.5] - 02-02-2024 + +### Fixed + +- POP updated +- Bug in updating project config file +- Subsample endpoint +- Lots more UI bugs that include some security vulnerabilities and stability issues + +## [0.11.4] - 01-22-2024 + +### Fixed + +- Downloading zip files + +## [0.11.2] - 01-17-2024 + +### Added + +- Add section to `/about` discussing browser support + +## [0.11.1] - 01-17-2024 + +### Added + +- `browserslist` support + +### Changed + +- `useBiggestNamespaces` no longer cached. + +## [0.11.0] - 01-16-2024 + +### Added + +- Group of PEPs, create a new type of PEP called a "POP" (a PEP of PEPs) +- Ability to star/favorite a PEP +- Updated search functionality to be more robust + +### Changed + +- Switch to `fastembed` for query embeddings to lower container size +- Minor UI updates + +## [0.10.5] - 12-04-2023 + +### Changed + +- optimized web interface fetching of PEP annotation data. + +### Added + +- project annotation endpoint (#234) + +# [0.10.4] - 10-02-2023 + +### Fixed + +- PEP zip downloading + +# [0.10.3] - 08-31-2023 + +### Changed + +- Add support for twitter cards, change some things. + +# [0.10.2] - 08-31-2023 + +### Changed + +- Changed image link for open graph image + +# [0.10.1] - 08-30-2023 + +### Changed + +- Add opengraph image link + +# [0.10.0] - 08-24-2023 + +### Added + +- Date filter to project annotation endpoint + +## [0.9.9] - 08-22-2023 + +### Changed + +- schema tag URL to route to schema splash page + +## [0.9.8] - 07-24-2023 + +### Fixed + +- cant add a PEP to a namespace if you don't have any to begin with + +## [0.9.7] - 07-24-2023 + +### Fixed + +- sample table would exhibit odd, erratic behavior if column names were left blank +- alnding page styling was not otpimal + +## [0.9.6] - 07-20-2023 + +### Fixed + +- Upload raw project errors + +### Changed + +- More stylish landing page that exemplifies pephub features +- Better error handling on queries + +## [0.9.5] - 07-19-2023 + +### Fixed + +- Changing sample_name error + +### Added + +- Landing sample table +- UI tweaks +- About page (In progress) +- Sample, subsample, config update simultaneously when saved + +### Changed + +- Landing page + +## [0.9.4] - 07-18-2023 + +### Fixed + +- Typo in tooltip for search bar + +### Added + +- Tooltip on landing page namespace list + +### Changed + +- Styling of landing namespaces to more clearly indicate they are links + +## [0.9.3] - 07-17-2023 + +### Changed + +- Styling updates +- Landing tooltips +- Minor UI updates + +## [0.9.2] - 07-12-2023 + +### Fixed + +- validating was not firing when updating sample table, subsample table, or config + +### Added + +- github organizations UI visibility +- schema tag has link to schema + +## [0.9.1] - 07-11-2023 + +### Fixed + +- forking was broken +- order in config file was incorrect + +### Added + +- config endpoint + +## [0.9.0] - 07-05-2023 + +### Fixed + +- description updating was broken +- strip markdown in description of projects in project list +- sample table stability updates + +### Added + +- better authentication awareness, app now checks for login status on every render, removes session if no longer valid +- added basic subsample table editing +- better validation error messages for universal validator + +## [0.8.4] - 06-21-2023 + +### Fixed + +- lots of sample table editing bugs +- sample table editing now works as expected +- logging in sent you to the homepage no matter what + +### Changed + +- authentication uses `localStorage` instead of browser cookies +- forking a PEP brings in the description of the PEP +- landing page changes + +## [0.8.3] - 06-11-2023 + +### Fixed + +- schema validation error causing crash in production. + +## [0.8.2] - 06-11-2023 + +### Fixed + +- Dockerfile build error due to missing `gcc` dependency. + +## [0.8.1] - 06-11-2023 + +### Fixed + +- Dockerfile build error due to `psycopg2` issue. + +## [0.8.0] - 06-09-2023 + +### Fixed + +- Sample table editing had bugs +- Width of sample table was incorrect +- Not found projects would load forever +- Case-sensitivity was causing errors +- Search links were not working + +### Added + +- Interface updated to support new features +- Markdwon support for PEPs +- Assign schemas to PEPs +- Ability to sort PEPs on namespace page +- Database setup is now streamlined + +### Removed + +- `cli` is now deprecated + +## [0.7.4] - 2022-04-15 + +### Fixed + +- Couldnt remove columns from sample table +- Drag n Drop was broken -> now fixed +- Fix sample table editing due to trailing commas +- File upload now fixed + +### Changed + +- Redesign project page for a better user experience +- Key binding for global search is now better + +### Added + +- CSS utility classes for better styling + +## [0.7.4] - 2022-04-15 + +### Fixed + +- Dockerfile not pulling in `/public` from `web/` + +## [0.7.3] - 2022-04-15 + +### Changed + +- Fixed dependencies to _actually_ install the `cpu` only version of `torch` +- Added new macOS dependencies to support native development/deployment + +## [0.7.2] - 2022-04-14 + +### Changed + +- Fix bug that was deleting images from the frontend + +## [0.7.1] - 2022-04-14 + +### Changed + +- Optimized Dockerfile for faster, slimmer deployments + +## [0.7.0] - 2022-04-14 + +### Fixed + +- Private projects were viewable to those not logged in +- Some projects were causing 500 server errors + +### Added + +- Device authentication flow +- Ability to "fork" a PEP +- Web interface lets you download a zip file of a PEP +- New submission flow for JSON representation of a PEP + +### Changed + +- Reimplemented web interface in React.js +- New deploymnet strategy that uses `uvicorn` instead of `pip install .` + +## [0.6.0] - 2022-03-06 + +### Fixed + +- Buffering configuration file +- Saving failures in production +- Renewed login bug + +### Added + +- Advanced searching features like score, offset, limit +- Ability to add a "blank" PEP +- Web-based metadata builder UI basics + +### Changed + +- Use `Authorization Bearer` headers to authenticate requests with the API +- Include nunjucks through CDN +- Switch `env` files to take advantage of `pass` + +## [0.5.0] - 2022-01-17 + +### Added + +- New landing page +- New namespace page +- Ability to edit PEPs with the new project edit page +- Vector search for PEPs + +### Changed + +- Moved api endpoints to `/api/v1` + +### Removed + +- Removed `/view` endpoints + +## [0.4.0] - 2022-11-09 + +### Added + +- Sort PEP's by authentication state (private PEPs) +- New and improbed web validator (thanks Alip!) + +### Changed + +- Revamped `pepdbagent` with better stability and type safety + +## [0.3.0] - 2022-09-07 + +### Added + +- User authentication to submit PEPs +- More thorough out `/view` endpoints +- More PEPs 🎉 +- Users can now specify a `?tag=` query parameter to fetch a PEP by its tag. + +### Changed + +- PEPhub is now backed by a postgres database +- Utilizes `pepgbagent` to interface with database + +## [0.2.0] - 2022-06-16 + +### Added + +- Better `/view` endpoints (switch from cards to tables) +- More namespace endpoints +- Sample view endpoints + +### Fixed + +- Poor output of `/pep-list` +- filters wrapped in `json` +- `pephub` version missing + +## [0.1.0] - 2022-05-16 + +### Added + +- Endpoints for converting stored PEPs using filters. +- dotfiles (`.pep.yaml`) are now standardized +- removed redundant `_project` representation from endpoints + +### Fixed: + +- project endpoints are no longer case-sensitive +- don't open demo links in new window +- better splash page styling +- no more pinned requirements file + +## [0.0.1] - 2021-11-03 + +### Added + +- Initial release diff --git a/docs/pephub/developer/pepdbagent/db_tutorial.md b/docs/pephub/developer/pepdbagent/db_tutorial.md index 8bf6907e..f63e6884 100644 --- a/docs/pephub/developer/pepdbagent/db_tutorial.md +++ b/docs/pephub/developer/pepdbagent/db_tutorial.md @@ -2,7 +2,7 @@ ### Container installation: -0) Go to [pep_db](pep_db) directory and then run the following lines.
+0) Go to `pep_db` directory and then run the following lines.
1) Build the docker: ``` docker build -t pep-db ./ diff --git a/docs/pephub/user/accounts.md b/docs/pephub/user/accounts.md deleted file mode 100644 index 0ae2e3d3..00000000 --- a/docs/pephub/user/accounts.md +++ /dev/null @@ -1,7 +0,0 @@ -# Logging into PEPhub - -PEPhub accounts are linked to GitHub. This allows us to leverage GitHub's OAuth system for secure authentication and namespacing. Once you have a [GitHub account](https://github.com/signup), you can log in to PEPhub. Just click the "Login" button in the top right corner of the [PEPhub home page](https://pephub.databio.org). - -![PEPhub login button](../img/login.png) - -You will be redirected to GitHub to authorize the PEPhub application. You are now logged in and can upload your first PEP! diff --git a/docs/pephub/user/editing.md b/docs/pephub/user/editing.md deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/pephub/user/geo.md b/docs/pephub/user/geo.md new file mode 100644 index 00000000..b1e0e090 --- /dev/null +++ b/docs/pephub/user/geo.md @@ -0,0 +1,8 @@ + +# Accessing GEO data through PEPhub + +Moreover, users can download all project as `tar file` from the GEO namespace using the link available on the geo namespace page. PEPhub doesn't store actual files in the database. Because of this, if you want to download files, there are two options: + + - Use links to the files that are stored in the project sample table. + - Use geofetch on a local machine to download these files. +Example: `geofetch -i GSE95654 --processed`, where `--processed` indicates that you want to download processed data, not SRA. More information about PEP can be found on the official website [GEOfetch](https://geofetch.databio.org/en/latest/). diff --git a/docs/pephub/user/getting-started.md b/docs/pephub/user/getting-started.md index 820fa423..b6f4a35c 100644 --- a/docs/pephub/user/getting-started.md +++ b/docs/pephub/user/getting-started.md @@ -14,11 +14,47 @@ Portable Encapsulated Projects (PEPs) are standard format for biological sample ## How PEPhub and PEPs work together -Storing files in a central location is a key feature of PEPhub. Local, individualized storage of PEPs can quickly become intractable for large teams. To that end, PEPhub allows you to store these in a database that is accessible to anyone through a web interface. This allows for easy sharing and collaboration on projects. +PEPhub gives you a central location to store and collaborate on your PEPs. This makes it easier to work together for large or small teams. Instead of relying on local files that you send back-and-forth, PEPhub provides a centralized interface and API that simplifies sharing and collaboration. In short, PEPs are the standard format for biological metadata, and PEPhub is the platform that allows you to store, edit, and share these PEPs. Through the API, one can easily access and retrieve PEPs for use in workflows and analyses. -## Ready to get started? +## Logging into PEPhub + +PEPhub accounts are linked to GitHub. This allows us to leverage GitHub's OAuth system for secure authentication and namespacing. Once you have a [GitHub account](https://github.com/signup), you can log in to PEPhub. Just click the "Login" button in the top right corner of the [PEPhub home page](https://pephub.databio.org). + +![PEPhub login button](../img/login.png) + +You will be redirected to GitHub to authorize the PEPhub application. You are now logged in and can upload your first PEP! There are two mains ways to add a PEP to your PEPhub namespace: you can either [upload a PEP directly](#uploading-a-pep), or you can [create a new PEP from scratch](#creating-a-new-pep-from-scratch) using the web interface. This guide will walk you through both methods. + +## Uploading a PEP + +Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username}`) and click the "Add" button in the top right. Click the "Upload PEP" tab. You will be prompted to select a PEP file from your local machine. Fill in the details about your PEP and then either drag files to the drop zone or click the drop zone to select files from your computer. Click "Submit" to add the PEP to your namespace. + +## Creating a new PEP from scratch + +Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username}`) and click the "Add" button in the top right. Click the "Blank PEP" tab. Again, fill in the details about your PEP and then you can start filling in the sample table. Click "Submit" to add the PEP to your namespace. + +![Submission form for a new PEP](../img/add-pep-form.png) + +## Ready to edit your PEP + +Once you have uploaded or created a PEP, you can now start using it in pipelines! + +## Editing a PEP + +Editing a PEP is easy; just make changes in the table and click `Save` when you are finished. + +## Sharing your PEP + +By default, new PEPs are set to public access. This means anyone who has the link can read your table. You can share it by simply sharing the URL. You can restrict this by marking it as `private` in the Project edit menu. + +If you want to give another user *write* access, then you'll need to do a bit more work. Since PEPhub user permissions are inherited directly from GitHub, you'll use GitHub to manage these. You should transfer the PEP into an organization where both users are public members. This will give both users write access to the PEP. + + + + + + + -To get started storing metadata on PEPhub, you will need to log in. Click the button below to learn more about getting an account and logging in. diff --git a/docs/pephub/user/overview.md b/docs/pephub/user/overview.md deleted file mode 100644 index 20c42214..00000000 --- a/docs/pephub/user/overview.md +++ /dev/null @@ -1,45 +0,0 @@ -## Overview -## What is PEP? - - -### Validation - -PEPhub valides sample metadata with [eido](/eido). Users can specify a schema to which the PEP should adhere. All schemas are available on the official website: [https://schema.databio.org/](https://schema.databio.org/). Schemas are particularly useful before running pipelines, as validation provides essential information about PEP compatibility with specific pipelines and highlights any errors in the PEP structure. - -### Search -PEPhub has semantic search functionality. The vector database is populated with important information extracted from the PEP config file. Through the search capability, users can efficiently locate projects, POPs, and Namespaces related to the string they provide. -More information can be found in [PEPhub semantic search docs](semantic-search.md). - - -### Authorization - -A key feature of PEPhub is that users can submit and edit their own PEPs. To facilitate this, we have implemented a robust user authorization system. Users authenticate via GitHub, which provides access to a range of features, including the ability to upload, modify, and delete PEPs, and star projects. Authorized users can also designate projects as private, ensuring that only they have visibility, with restricted access for others. -Moreover, users have the capability to modify projects within organizational namespaces, which correspond to the GitHub organizations they belong to. This feature facilitates collaborative efforts within organizational groups. - -### PEP of PEPs (POP) - -The PEP of PEPs, often referred to as a Project of Projects or simply POP, is a specific type of PEP in which each sample is itself a PEP. Essentially, a POP can be thought of as a grouping of PEPs, allowing users to organize projects for various purposes. This approach offers several advantages: all PEPs related to a specific topic can be consolidated in one central location, streamlining organization and accessibility. - -### GEO data - -PEPhub has a namespace called GEO. This namespace contains projects from [Gene Expression Omnibus](https://www.ncbi.nlm.nih.gov/geo/) that are downloaded and processed using [GEOfetch](/geofetch). GEOfetch produces a standardized PEP sample table with GSM (GEO sample) and a YAML config that has metadata from GSE (GEO project). Nearly 99% of projects are downloaded to PEPhub. Users can utilize the namespace search to find projects by accession ID and description. -GEO namespace link: [https://pephub.databio.org/geo](https://pephub.databio.org/geo). -Moreover, users can download all project as `tar file` from the GEO namespace using the link available on the geo namespace page. -PEPhub doesn't store actual files in the database. Because of this, if you want to download files, there are two options: - - - Use links to the files that are stored in the project sample table. - - Use geofetch on a local machine to download these files. -Example: `geofetch -i GSE95654 --processed`, where `--processed` indicates that you want to download processed data, not SRA. More information about PEP can be found on the official website [GEOfetch](https://geofetch.databio.org/en/latest/). - - -## PEPHubClient (phc) - -PEPhub provides an additional command-line tool and Python API, connecting Python to the PEPhub. The dedicated Python package for this integration is called PEPHubClient, and it is compatible with Linux, Windows, and Mac systems. - -Key features of PEPHubClient include: - -- **Authorization**: Users can log in to PEPhub using PEPHubClient, enabling the loading of private projects and the ability to upload projects to PEPhub. -- **Load and Download**: PEPHubClient facilitates the loading and downloading of PEPs directly from PEPhub. -- **Push**: Users can push PEPs from their local environment to PEPhub. - -More information is available in the pephubclient docs: [PEPHubClient](https://pep.databio.org/pephub/pephubclient). diff --git a/docs/pephub/user/sharing.md b/docs/pephub/user/sharing.md deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/pephub/user/uploading.md b/docs/pephub/user/uploading.md index fb02e9e2..b2a3b380 100644 --- a/docs/pephub/user/uploading.md +++ b/docs/pephub/user/uploading.md @@ -15,3 +15,11 @@ Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username} ## Ready to edit your PEP Once you have uploaded or created a PEP, you can now start using it in pipelines! + +## Editing a PEP + +... + +## Sharing your PEP + +... diff --git a/mkdocs.yml b/mkdocs.yml index 4db4b308..236e0d6d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -180,16 +180,13 @@ nav: - PEPhub: pephub/README.md - User guide: - Getting started: pephub/user/getting-started.md - - Logging in: pephub/user/accounts.md - - Uploading your first PEP: pephub/user/uploading.md - # - Editing a PEP: pephub/user/editing.md - # - Sharing your PEP: pephub/user/sharing.md - - How to use views: pephub/user/views.md - Use a PEP in an existing pipeline: pephub/user/pipelines.md - # - Validating PEPs: pephub/user/validation.md - # - Version control: pephub/user/version-control.md - # - Semantic search: pephub/user/semantic-search.md + - Validating PEPs: pephub/user/validation.md + - Version control: pephub/user/version-control.md + - Semantic search: pephub/user/semantic-search.md + - How to use views: pephub/user/views.md - PEP of PEPs (POP): pephub/user/pops.md + - Accessing GEO metadata: pephub/user/geo.md - Developer guide: - Setup: pephub/developer/setup.md - Deployment: pephub/developer/deployment.md From a978c9ac534a01ccbb97f7de860c098363f3c4a9 Mon Sep 17 00:00:00 2001 From: nsheff Date: Thu, 23 May 2024 16:10:56 -0400 Subject: [PATCH 3/9] remove file --- docs/pephub/user/uploading.md | 25 ------------------------- 1 file changed, 25 deletions(-) delete mode 100644 docs/pephub/user/uploading.md diff --git a/docs/pephub/user/uploading.md b/docs/pephub/user/uploading.md deleted file mode 100644 index b2a3b380..00000000 --- a/docs/pephub/user/uploading.md +++ /dev/null @@ -1,25 +0,0 @@ -# Uploading PEPs to PEPhub - -There are two mains ways to add a PEP to your PEPhub namespace: you can either [upload a PEP directly](#uploading-a-pep), or you can [create a new PEP from scratch](#creating-a-new-pep-from-scratch) using the web interface. This guide will walk you through both methods. - -## Uploading a PEP - -Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username}`) and click the "Add" button in the top right. Click the "Upload PEP" tab. You will be prompted to select a PEP file from your local machine. Fill in the details about your PEP and then either drag files to the drop zone or click the drop zone to select files from your computer. Click "Submit" to add the PEP to your namespace. - -## Creating a new PEP from scratch - -Navigate to your PEPhub namespace (`https://pephub.databio.org/{github username}`) and click the "Add" button in the top right. Click the "Blank PEP" tab. Again, fill in the details about your PEP and then you can start filling in the sample table. Click "Submit" to add the PEP to your namespace. - -![Submission form for a new PEP](../img/add-pep-form.png) - -## Ready to edit your PEP - -Once you have uploaded or created a PEP, you can now start using it in pipelines! - -## Editing a PEP - -... - -## Sharing your PEP - -... From 3609c747a0b17bcbfe9e19731d3631001cb43b4c Mon Sep 17 00:00:00 2001 From: nsheff Date: Thu, 23 May 2024 16:12:11 -0400 Subject: [PATCH 4/9] typos --- docs/pephub/changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pephub/changelog.md b/docs/pephub/changelog.md index 0db8121e..9a8981da 100644 --- a/docs/pephub/changelog.md +++ b/docs/pephub/changelog.md @@ -328,7 +328,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ### Changed - Reimplemented web interface in React.js -- New deploymnet strategy that uses `uvicorn` instead of `pip install .` +- New deployment strategy that uses `uvicorn` instead of `pip install .` ## [0.6.0] - 2022-03-06 From 187a378d33be73012d079a778cc3fb882cadd287 Mon Sep 17 00:00:00 2001 From: nsheff Date: Thu, 23 May 2024 16:13:30 -0400 Subject: [PATCH 5/9] typos --- docs/pephub/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pephub/README.md b/docs/pephub/README.md index d86b62cd..1616843c 100644 --- a/docs/pephub/README.md +++ b/docs/pephub/README.md @@ -24,7 +24,7 @@ PEPhub is an open-source database, web interface, and API for sharing, retrievin - **Group PEPs with using a PEP of PEPs (POP)**. A PEP of PEPs, or simply a POP, is a specific type of PEP in which each row is itself a PEP. Essentially, a POP is a structure to group PEPs, allowing users to organize projects. This allows PEPs related to a specific topic to be consolidated, streamlining organization and accessibility. -- **Re-processing of GEO metadata**. The public PEPhub instance [geo namespace](https://pephub.databio.org/geo) holds metadata from nearly 99% of the [Gene Expression Omnibus](https://www.ncbi.nlm.nih.gov/geo/). PEPhub is updated weekly using [GEOfetch](../geofetch/README.md) to produce standardized PEP sample tables, providing a convienient API interface to GEO metadata. +- **Re-processing of GEO metadata**. The public PEPhub instance [geo namespace](https://pephub.databio.org/geo) holds metadata from nearly 99% of the [Gene Expression Omnibus](https://www.ncbi.nlm.nih.gov/geo/). PEPhub is updated weekly using [GEOfetch](../geofetch/README.md) to produce standardized PEP sample tables, providing a convenient API interface to GEO metadata. - **PEPHubClient (phc)**. PEPhubClient is a command-line tool and Python API, which allows users to authenticate with PEPhub, download and upload public or private projects. For more information, see the [PEPHubClient documentation](developer/pephubclient/README.md). From 8060ab5088d2f2975dea16ceeb4857511fd85867 Mon Sep 17 00:00:00 2001 From: Khoroshevskyi Date: Thu, 23 May 2024 17:21:33 -0400 Subject: [PATCH 6/9] updated PEPHub POPs and pephubclient --- autodoc.py | 2 +- .../pephubclient/phc_samples_usage.md | 116 ++ .../developer/pephubclient/phc_usage.md | 196 +++ .../developer/pephubclient/phc_views_usage.md | 158 +++ docs/pephub/img/pop.svg | 1126 +++++++++++++++++ docs/pephub/user/pops.md | 16 + mkdocs.yml | 5 +- 7 files changed, 1617 insertions(+), 2 deletions(-) create mode 100644 docs/pephub/developer/pephubclient/phc_samples_usage.md create mode 100644 docs/pephub/developer/pephubclient/phc_usage.md create mode 100644 docs/pephub/developer/pephubclient/phc_views_usage.md create mode 100644 docs/pephub/img/pop.svg diff --git a/autodoc.py b/autodoc.py index e5c7aa14..5231f588 100644 --- a/autodoc.py +++ b/autodoc.py @@ -99,4 +99,4 @@ with open(out, "w") as stream: stream.write(md_result) else: - print("Skipping jupyter notebooks") \ No newline at end of file + print("Skipping jupyter notebooks") diff --git a/docs/pephub/developer/pephubclient/phc_samples_usage.md b/docs/pephub/developer/pephubclient/phc_samples_usage.md new file mode 100644 index 00000000..6a0f0e08 --- /dev/null +++ b/docs/pephub/developer/pephubclient/phc_samples_usage.md @@ -0,0 +1,116 @@ + + + +# module `pephubclient.modules.sample` + + + + +**Global Variables** +--------------- +- **PEPHUB_SAMPLE_URL** + + +--- + + +## class `PEPHubSample` +Class for managing samples in PEPhub and provides methods for getting, creating, updating and removing samples. This class is not related to peppy.Sample class. + + +### method `__init__` + +```python +__init__(jwt_data: str = None) +``` + + +- :param jwt_data: jwt token for authorization + + + + +--- + + +### method `create` + +```python +create( + namespace: str, + name: str, + tag: str, + sample_name: str, + sample_dict: dict, + overwrite: bool = False +) → None +``` + +Create sample in project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param sample_dict: sample dict +- :param sample_name: sample name +- :param overwrite: overwrite sample if it exists :return: None + +--- + + +### method `get` + +```python +get(namespace: str, name: str, tag: str, sample_name: str = None) → dict +``` + +Get sample from project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param sample_name: sample name :return: Sample object + +--- + + +### method `remove` + +```python +remove(namespace: str, name: str, tag: str, sample_name: str) +``` + +Remove sample from project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param sample_name: sample name :return: None + +--- + + +### method `update` + +```python +update(namespace: str, name: str, tag: str, sample_name: str, sample_dict: dict) +``` + +Update sample in project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param sample_name: sample name +- :param sample_dict: sample dict, that contain elements to update, or :return: None + + + + +--- + +_This file was automatically generated via [lazydocs](https://github.com/ml-tooling/lazydocs)._ diff --git a/docs/pephub/developer/pephubclient/phc_usage.md b/docs/pephub/developer/pephubclient/phc_usage.md new file mode 100644 index 00000000..ee2482dd --- /dev/null +++ b/docs/pephub/developer/pephubclient/phc_usage.md @@ -0,0 +1,196 @@ + + + +# module `pephubclient.pephubclient` + +--- + + +## class `PEPHubClient` + + +--- + +#### property sample + +view represents the PHCSample class which contains all samples API + + +--- + +#### property view + +view represents the PHCView class which contains all views API + + + +--- + +### method `find_project` + +```python +find_project( + namespace: str, + query_string: str = '', + limit: int = 100, + offset: int = 0, + filter_by: Literal['submission_date', 'last_update_date'] = None, + start_date: str = None, + end_date: str = None +) → SearchReturnModel +``` + +Find project in specific namespace and return list of PEP annotation + +- +- - :param namespace: Namespace where to search for projects +- +- - :param query_string: Search query +- +- - :param limit: Return limit +- +- - :param offset: Return offset +- +- - :param filter_by: Use filter date. Option: [submission_date, last_update_date] +- +- - :param start_date: filter beginning date +- +- - :param end_date: filter end date (if none today's date is used) :return: + +--- + + +### method `load_project` + +```python +load_project( + project_registry_path: str, + query_param: Optional[dict] = None +) → Project +``` + +Load peppy project from PEPhub in peppy.Project object + + +- :param project_registry_path: registry path of the project + +- :param query_param: query parameters used in get request :return Project: peppy project. + +--- + + +### method `load_raw_pep` + +```python +load_raw_pep(registry_path: str, query_param: Optional[dict] = None) → dict +``` + +Request PEPhub and return the requested project as peppy.Project object. + + +- :param registry_path: Project namespace, eg. "geo/GSE124224:tag" +- :param query_param: Optional variables to be passed to PEPhub :return: Raw project in dict. + +--- + + +### method `login` + +```python +login() → NoReturn +``` + +Log in to PEPhub + +--- + + +### method `logout` + +```python +logout() → NoReturn +``` + +Log out from PEPhub + +--- + + +### method `pull` + +```python +pull( + project_registry_path: str, + force: Optional[bool] = False, + zip: Optional[bool] = False, + output: Optional[str] = None +) → None +``` + +Download project locally + + +- :param str project_registry_path: Project registry path in PEPhub (e.g. databio/base:default) +- :param bool force: if project exists, overwrite it. +- :param bool zip: if True, save project as zip file +- :param str output: path where project will be saved :return: None + +--- + + +### method `push` + +```python +push( + cfg: str, + namespace: str, + name: Optional[str] = None, + tag: Optional[str] = None, + is_private: Optional[bool] = False, + force: Optional[bool] = False +) → None +``` + +Push (upload/update) project to Pephub using config/csv path + + +- :param str cfg: Project config file (YAML) or sample table (CSV/TSV) with one row per sample to constitute project +- :param str namespace: namespace +- :param str name: project name +- :param str tag: project tag +- :param bool is_private: Specifies whether project should be private [Default= False] +- :param bool force: Force push to the database. Use it to update, or upload project. [Default= False] :return: None + +--- + + +### method `upload` + +```python +upload( + project: Project, + namespace: str, + name: str = None, + tag: str = None, + is_private: bool = False, + force: bool = True +) → None +``` + +Upload peppy project to the PEPhub. + + +- :param peppy.Project project: Project object that has to be uploaded to the DB +- :param namespace: namespace +- :param name: project name +- :param tag: project tag +- :param force: Force push to the database. Use it to update, or upload project. +- :param is_private: Make project private +- :param force: overwrite project if it exists :return: None + + + + +--- + +_This file was automatically generated via [lazydocs](https://github.com/ml-tooling/lazydocs)._ diff --git a/docs/pephub/developer/pephubclient/phc_views_usage.md b/docs/pephub/developer/pephubclient/phc_views_usage.md new file mode 100644 index 00000000..cfd97b41 --- /dev/null +++ b/docs/pephub/developer/pephubclient/phc_views_usage.md @@ -0,0 +1,158 @@ + + + +# module `pephubclient.modules.view` + + + + +**Global Variables** +--------------- +- **PEPHUB_VIEW_URL** +- **PEPHUB_VIEW_SAMPLE_URL** + + +--- + + +## class `PEPHubView` +Class for managing views in PEPhub and provides methods for getting, creating, updating and removing views. + +This class aims to warp the Views API for easier maintenance and better user experience. + + +### method `__init__` + +```python +__init__(jwt_data: str = None) +``` + + +- :param jwt_data: jwt token for authorization + + + + +--- + + +### method `add_sample` + +```python +add_sample( + namespace: str, + name: str, + tag: str, + view_name: str, + sample_name: str +) +``` + +Add sample to view in project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param view_name: name of the view +- :param sample_name: name of the sample + +--- + + +### method `create` + +```python +create( + namespace: str, + name: str, + tag: str, + view_name: str, + description: str = None, + sample_list: list = None, + no_fail: bool = False +) +``` + +Create view in project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param description: description of the view +- :param view_name: name of the view +- :param sample_list: list of sample names +- :param no_fail: whether to raise an error if view was not added to the project + +--- + + +### method `delete` + +```python +delete(namespace: str, name: str, tag: str, view_name: str) → None +``` + +Delete view from project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param view_name: name of the view :return: None + +--- + + +### method `get` + +```python +get( + namespace: str, + name: str, + tag: str, + view_name: str, + raw: bool = False +) → Union[Project, dict] +``` + +Get view from project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param view_name: name of the view +- :param raw: if True, return raw response :return: peppy.Project object or dictionary of the project (view) + +--- + + +### method `remove_sample` + +```python +remove_sample( + namespace: str, + name: str, + tag: str, + view_name: str, + sample_name: str +) +``` + +Remove sample from view in project in PEPhub. + + +- :param namespace: namespace of project +- :param name: name of project +- :param tag: tag of project +- :param view_name: name of the view +- :param sample_name: name of the sample :return: None + + + + +--- + +_This file was automatically generated via [lazydocs](https://github.com/ml-tooling/lazydocs)._ diff --git a/docs/pephub/img/pop.svg b/docs/pephub/img/pop.svg new file mode 100644 index 00000000..a5650d0c --- /dev/null +++ b/docs/pephub/img/pop.svg @@ -0,0 +1,1126 @@ + +image/svg+xmlPEPPEPPEPPEPPOPPEPPEPPEPPOPPEPhub diff --git a/docs/pephub/user/pops.md b/docs/pephub/user/pops.md index e69de29b..0ed43ca6 100644 --- a/docs/pephub/user/pops.md +++ b/docs/pephub/user/pops.md @@ -0,0 +1,16 @@ +# PEPs vs POPs + +**Portable Encapsulated Projects (PEPs)** vs **PEP of PEPs (POPs).** + +### POPs + +POP is simply a group of PEPs represented as a PEP. Users can think of it as a group, collection, or folder of projects. +It groups PEPs together to represent similar projects, analyses, or related information, making it easier to manage and navigate related proposals. + +Each POP can contain other PEPs or POPs (since a POP is a PEP), allowing for a recursive structure. +This recursive nature enables complex hierarchies and relationships to be efficiently organized within the PEP framework. + +### Database structure +In the database POP in represented as a PEP with a special schema, that containing the list of PEPs that are part of the POP. + +![POPs](../img/pop.svg) diff --git a/mkdocs.yml b/mkdocs.yml index 89824e9c..9290a905 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -181,6 +181,7 @@ nav: # - Editing a PEP: pephub/user/editing.md # - Sharing your PEP: pephub/user/sharing.md - Use a PEP in an existing pipeline: pephub/user/pipelines.md + - What is POP?: pephub/user/pops.md # - Validating PEPs: pephub/user/validation.md # - Version control: pephub/user/version-control.md # - Semantic search: pephub/user/semantic-search.md @@ -195,7 +196,9 @@ nav: - PEPHubClient: - PEPhubClient: pephub/developer/pephubclient/README.md - Changelog: pephub/developer/pephubclient/changelog.md - - Python API: pephub/developer/pephubclient/python-api.md + - Python API: pephub/developer/pephubclient/phc_usage.md + - Python API samples: pephub/developer/pephubclient/phc_samples_usage.md + - Python API views: pephub/developer/pephubclient/phc_views_usage.md - CLI usage: pephub/developer/pephubclient/cli.md - Server settings: pephub/developer/server-settings.md - PEPembed: From ad30a9d880694fe56cb0504dd9040fe66b1231ad Mon Sep 17 00:00:00 2001 From: Khoroshevskyi Date: Thu, 23 May 2024 17:22:34 -0400 Subject: [PATCH 7/9] updated pephubclient changelog --- .../developer/pephubclient/changelog.md | 29 +++++++++++++++---- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/docs/pephub/developer/pephubclient/changelog.md b/docs/pephub/developer/pephubclient/changelog.md index 0a0194b4..6d63f1da 100644 --- a/docs/pephub/developer/pephubclient/changelog.md +++ b/docs/pephub/developer/pephubclient/changelog.md @@ -2,13 +2,32 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format. -## [0.3.0] - 2024-XX-XX +## [0.4.2] - 2024-04-16 +### Updated +- View creation, by adding description and no_fail flag + + +## [0.4.1] - 2024-03-07 +### Fixed +- Expired token error handling ([#17](https://github.com/pepkit/pephubclient/issues/17)) + +## [0.4.0] - 2024-02-12 ### Added -- Added param parent dir where peps should be saved -- Added zip option to save_pep function +- a parameter that points to where peps should be saved ([#32](https://github.com/pepkit/pephubclient/issues/32)) +- pep zipping option to `save_pep` function ([#34](https://github.com/pepkit/pephubclient/issues/34)) +- API for samples ([#29](https://github.com/pepkit/pephubclient/issues/29)) +- API for projects ([#28](https://github.com/pepkit/pephubclient/issues/28)) -### Changed -- Transferred save_pep function to helpers +### Updated +- Transferred `save_pep` function to helpers + +## [0.3.0] - 2024-01-17 +### Added +- customization of the base PEPhub URL ([#22](https://github.com/pepkit/pephubclient/issues/22)) + +### Updated +- Updated PEPhub API URL +- Increased the required pydantic version to >2.5.0 ## [0.2.2] - 2024-01-17 ### Added From 2406d0e87de07847f15e619ff42da1a0252000c9 Mon Sep 17 00:00:00 2001 From: Khoroshevskyi Date: Fri, 24 May 2024 10:15:57 -0400 Subject: [PATCH 8/9] updated phc api --- .../developer/pephubclient/phc_usage.md | 21 +++++++------------ 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/docs/pephub/developer/pephubclient/phc_usage.md b/docs/pephub/developer/pephubclient/phc_usage.md index ee2482dd..06a9f00a 100644 --- a/docs/pephub/developer/pephubclient/phc_usage.md +++ b/docs/pephub/developer/pephubclient/phc_usage.md @@ -42,20 +42,13 @@ find_project( Find project in specific namespace and return list of PEP annotation -- -- - :param namespace: Namespace where to search for projects -- -- - :param query_string: Search query -- -- - :param limit: Return limit -- -- - :param offset: Return offset -- -- - :param filter_by: Use filter date. Option: [submission_date, last_update_date] -- -- - :param start_date: filter beginning date -- -- - :param end_date: filter end date (if none today's date is used) :return: +- :param namespace: Namespace where to search for projects +- :param query_string: Search query +- :param limit: Return limit +- :param offset: Return offset +- :param filter_by: Use filter date. Option: [submission_date, last_update_date] +- :param start_date: filter beginning date +- :param end_date: filter end date (if none today's date is used) :return: --- From f9dd93122b877d1034b45c7e7a2a453cee601ffc Mon Sep 17 00:00:00 2001 From: Khoroshevskyi Date: Fri, 24 May 2024 11:06:37 -0400 Subject: [PATCH 9/9] added quickstart for phc --- .../developer/pephubclient/python-api.md | 25 ------- .../pephub/developer/pephubclient/tutorial.md | 65 +++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 66 insertions(+), 25 deletions(-) delete mode 100644 docs/pephub/developer/pephubclient/python-api.md create mode 100644 docs/pephub/developer/pephubclient/tutorial.md diff --git a/docs/pephub/developer/pephubclient/python-api.md b/docs/pephub/developer/pephubclient/python-api.md deleted file mode 100644 index e1617858..00000000 --- a/docs/pephub/developer/pephubclient/python-api.md +++ /dev/null @@ -1,25 +0,0 @@ -# Using the PEPHubClient Python API - -```python -from pephubclient import PEPHubClient - -# initiate pephubclient object - -phc = PEPHubClient() - -# load pep as peppy.Project object - -example_pep = phc.load_project("databio/example:default") -print(example_pep) - -# Printed: -## Project -## 6 samples: 4-1_11102016, 3-1_11102016, 2-2_11102016, 2-1_11102016, 8-3_11152016, 8-1_11152016 -## Sections: pep_version, sample_table, name, description - -# To upload a project: -phc.upload(example_pep, namespace="databio", name="example", force=True) - -``` - - diff --git a/docs/pephub/developer/pephubclient/tutorial.md b/docs/pephub/developer/pephubclient/tutorial.md new file mode 100644 index 00000000..dbd21df7 --- /dev/null +++ b/docs/pephub/developer/pephubclient/tutorial.md @@ -0,0 +1,65 @@ +# PEPHub Client + +User guide for the PEPHub client. + +PEPhubclient is available as Python API and CLI. This document will guide you through the usage of the PEPHub client. + +## Installation + +To install PEPHubClient from PyPI, use the following command: + +```bash +pip install pephubclient +``` + +To install `pephubclient` from the [GitHub repository](https://github.com/pepkit/pephubclient), use the following command: + +```bash +pip install git+https://github.com/pepkit/pephubclient.git +``` + +## Authentication + +To login, use the `login` command: + +``` +phc login +``` + + +## Using the PEPHubClient CLI + +PEPHubClient provides a CLI for interacting with PEPhub. +You can find the list of available commands here: +[PEPHubClient CLI](./python-api.md) + + +## Using the PEPHubClient Python API + +```python +from pephubclient import PEPHubClient + +# initiate pephubclient object +phc = PEPHubClient() + +# load pep as peppy.Project object +example_pep = phc.load_project("databio/example:default") +print(example_pep) + +# Printed: +## Project +## 6 samples: 4-1_11102016, 3-1_11102016, 2-2_11102016, 2-1_11102016, 8-3_11152016, 8-1_11152016 +## Sections: pep_version, sample_table, name, description + +# To upload a project: +phc.upload(example_pep, namespace="databio", name="example", force=True) +``` + +Full documentation for the Python API can be found here: + +- [PEPHubClient Python API](./phc_usage.md) +- [PEPHubClient Python API - Samples](./phc_samples_usage.md) +- [PEPHubClient Python API - Views](./phc_views_usage.md) + +---- +If you have any questions or need help, please contact us at [databio.org](https://databio.org) \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 9290a905..50b3e60f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -195,6 +195,7 @@ nav: - Semantic search: pephub/developer/semantic-search.md - PEPHubClient: - PEPhubClient: pephub/developer/pephubclient/README.md + - Quickstart: pephub/developer/pephubclient/tutorial.md - Changelog: pephub/developer/pephubclient/changelog.md - Python API: pephub/developer/pephubclient/phc_usage.md - Python API samples: pephub/developer/pephubclient/phc_samples_usage.md