Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Update README and .gitignore #33

Merged
merged 5 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ dbt_packages/
logs/
*/.env
*/dbt_packages/
.idea
27 changes: 14 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
CHT Pipeline
========================
# CHT Pipeline

CHT Pipeline is a tool used to define data models for transforming the raw data we get from Couch DB into models that can then be queried to build dashboards
CHT Pipeline is a tool used to define data models for transforming the raw data we get from Couch DB into models that can then be queried to build dashboards.

# Dev installation instructions
## Dev installation instructions

1. Follow the instructions in [cht-sync](https://github.com/medic/cht-sync) to set up the required environment variables
1. Run `docker-compose up` in the cht-sync directory to run the pipeline. Please ensure the `DATAEMON_INITAL_PACKAGE` env variable is set to your preferred branch of the cht-pipeline repo.

# Test
## Run dbt models unit tests locally
### Prerequisites

- `Docker`
- `PostgreSQL client`
- `dbt-postgres`
- `Python`

1. Navigate to tests folder
2. Run setup script:
### Setup environment variables
Follow the instructions in [cht-sync](https://github.com/medic/cht-sync) to set up the required environment variables.

### Run the pipeline
Run `docker-compose up` in the `cht-sync` directory to run the pipeline. Please ensure the `DATAEMON_INITAL_PACKAGE` env variable is set to your preferred branch of the `cht-pipeline` repo.

## Run dbt models unit tests locally

1. Navigate to `tests` folder.
2. Run the `setup` script:

```sh
# Set environment variables, create postgres database, schema and user:
./setup.sh
```

3. Run dbt test
3. Run the `dbt` tests:

```sh
# set environment variables, install dbt dependencies, seed data, run dbt, run test
Expand Down