diff --git a/README.rst b/README.rst index 4f9b2f7..8abbcd8 100644 --- a/README.rst +++ b/README.rst @@ -4,8 +4,55 @@ Platform Plugin Aspects Purpose ******* -This repository holds various Aspects related plugins for edx-platform. +This repository holds various Aspects plugins for the Open edX platform. +Sinks +***** + +Sinks are components that listen for events and store them in ClickHouse. The +events are emitted by the Open edX platform via `Open edX events`_ or Django signals. + +Available Sinks +=============== + +- `CourseOverviewSink` - Listens for the `COURSE_PUBLISHED` event and stores the + course structure data through `XBlockSink` in ClickHouse. +- `ExternalIdSink` - Listens for the `post_save` Django signal on the `ExternalId` + model and stores the external id data in ClickHouse. +- `UserProfile` - Listens for the `post_save` Django signal on the `UserProfile` + model and stores the user profile data in ClickHouse. +- `UserRetirementSink` - Listen for the `USER_RETIRE_LMS_MISC` Django signal and + remove the user PII information from ClickHouse. + +Commands +======== + +In addition to being an event listener, this package provides the following commands: + +- `dump_data_to_clickhouse` - This command allows bulk export of the data from the Sinks. + Allows bootstrapping a new data platform or backfilling lost or missing data. + + ``python manage.py cms dump_courses_to_clickhouse`` + +- `load_test_tracking_events` - This command allows loading test tracking events into + ClickHouse. This is useful for testing the ClickHouse connection to measure the performance of the + different data pipelines such as Vector, Event Bus (Redis and Kafka), and Celery. + + Do not use this command in production as it will generate a large amount of data + and will slow down the system. + + ``python manage.py cms load_test_tracking_events`` + +- `monitor_load_test_tracking` - Monitors the load test tracking script and saves + output for later analysis. + + ``python manage.py cms monitor_load_test_tracking`` + +Instructor Dashboard Integration +=============================== + +Dashboards from `Aspects`_ are integrated into the Instructor Dashboard via `Superset Embedded SDK`_. +See `Configuration`_ for more details. Getting Started with Development ******************************** @@ -15,12 +62,88 @@ Please see the Open edX documentation for `guidance on Python development `_) +Documentation for this component can be found at `Aspects`_. More Help ========= @@ -96,3 +215,10 @@ Reporting Security Issues ************************* Please do not report security issues in public. Please email security@openedx.org. + +.. _Open edX events: https://github.com/openedx/openedx-events +.. _Edx Platform: https://github.com/openedx/edx-platform +.. _ClickHouse: https://clickhouse.com +.. _Aspects: https://docs.openedx.org/projects/openedx-aspects/en/latest/index.html +.. _Superset Embedded SDK: https://www.npmjs.com/package/@superset-ui/embedded-sdk +.. _Open edX Filter: https://docs.openedx.org/projects/openedx-filters/en/latest/ \ No newline at end of file