Skip to content

Commit

Permalink
feature: add support to clarify.py for time series explainability…
Browse files Browse the repository at this point in the history
… jobs (aws#4503)

* feature: Added components to support time series explainability with Clarify. These components are TimeSeriesDataConfig, TimeSeriesModelConfig, and AsymmetricSHAPConfig, alomg with unit tests for them.
fix: Modified DataConfig, ModelConfig, and _AnalysisConfigGenerator to support the new components and time series explainability.
documentation: added docstrings for the new components and their tests.

* fix: removed field use_future_covariates and related unit tests from TimeSeriesModelConfig

* change: rename ``TimeSeriesDataConfig.analysis_config`` to ``time_series_data_config``

* change: validate DataConfig content_type and accept_type for TS exp.

change: renamed TimeSeriesDataConfig.predictor_config to
time_series_model_config

change: modified default value of forecast_horizon to 1

* change: reworked validation in AsymmetricSHAPConfig

change: removed default value for num_samples

change: changed default value for explanation_type

change: added more explicit type hint for explanation_type

documentation: added more information for parameters

change: reworked unit tests for AsymmetricSHAPConfig

* change: time series case no longer uses _merge_explainability_configs

change: _merge_explainability_configs reordered to put validation first

change: unit tests reworked

* fix: minor style changes to meet formatting reqs

* change: modified how time_series_case flag is set

change: removed now-redundant check in time_series_case

* fix: set time_series_case to False to prevent exception

* change: params for ``TimeSeriesDataConfig`` now must all be same type

change: updated ``TimeSeriesDataConfig`` unit tests to reflect above
change

* fix: schema entries for related_ts and item_metadata to keep list items same type

* change: remove forecast_horizon from TimeSeriesModelConfig

* fix: modified type hints in ``TimeSeriesDataConfig`` to match schema

* change: add errors when ts data or model config are given but no asym_shap config

change: add unit tests for _AnalysisConfigGenerator.explainability

fix: slightly modify AsymmetricSHAPConfig mock builder function

documentation: minor docstring update in tests

* change: remove flag ``time_series_case``, modify unit tests accordingly

change: add a check to _AnalysisConfiGenerator.bias_and_explainability
to prevent time series components being provided, added unit tests for
this

* change: rename `AsymmetricSHAPConfig` to `AsymmetricShapleyValueConfig`

* documentation: add description to ``AsymmetricShapleyValueConfig``

documentation: reword `target_time_series` parameter description

documentation: remove TODOs

* change: split ``explanation_type`` into ``explanation_ direction`` and ``granularity``

update tests and documentation accordingly

* fix: rename ``explanation_granularity`` to ``granularity``

* change: rename ``explanation_direction`` to ``direction``

* change: rename ``item_metadata`` to ``static_covariates``

change: add ``dataset_format`` as a parameter for time series cases

change: allow features jmespaths to be none for time series cases

change: add validation to prevent non-json dataset formats for time
series cases

test: update unit tests to reflect above changes

* fix: update clarify files to meet formatting reqs

* change: require headers for time series explainability

* feat: add (early version of) baseline config to asym shap val config

* refactor: change baseline config param names to keep with convention and be more cx friendly

* refactor: baseline config from list to dictionary where key is item_id value

* fix: set dataset_uri from s3_data_input_path

* fix: undo previous bug fix

what i believed was a bug was actually intended behaviour

* feat: add ``ITEM_RECORDS`` as a supported dataset format

change: remove ``headers`` as a requirement for time series

doc: add example dataset formats to ``TimeSeriesJSONDatasetFormat``

* doc: make docs for TimeSeriesJSONDatasetFormat sphinx-compliant

change: add ``item_records`` as a supported format to the schema

doc: add documentation for baseline

doc: remove references to deprecated ``forecast_horizon``

* feat: validation for asymmetric shapley value config baseline

doc: fix baseline doc to be sphinx-compliant

* feat: add validation for static covariates in tsx baseline

* fix: add call to validate baseline static covariates method

* fix: check if baseline dict is s3 uri in scv validation function

* fix: replace all added asserts with ValueError

---------

Co-authored-by: Mufaddal Rohawala <[email protected]>
  • Loading branch information
2 people authored and jiapinw committed Jun 25, 2024
1 parent 30195e4 commit 092a9fc
Show file tree
Hide file tree
Showing 2 changed files with 1,602 additions and 13 deletions.
Loading

0 comments on commit 092a9fc

Please sign in to comment.