SDNist v2.4

What's Changed

SDNist Repo:

Replaced old sample reports with the ones generated with sdnist version 2.3. Sample reports are available in repo at path: sdnist/report/sample-reports

SDNist Library:

• Fixed issue with deleting rows when there are unknown values.
• Fixed sdnist crash issue when deid. data has less than 3 features.
• Fixed issue with pca-pair plot when data has less than 5 features.

Assets from previous release

• toy_deidentified_data.zip

SDNist v2.3

What's Changed

Data:

• Updated INDP feature details in data_description.json file.

SDNist Library:

• Added explanatory text to inconsistencies and UEM (Unique Exact Matches) metric, fixed typographic errors, improved formatting in privacy section, renamed 'k-marginal breakdown' to 'worst-performing PUMA breakdown' and adjusted json structure accordingly.
• Added 1% and 5% sampling error on the k-marginal.
• Improved readability of propensity image.
• Fixed feature space size in UEM.
• Fixed deidentified data percentage in UEM metric.
• Fixed PCA plot scaling issues.

Assets from previous release

• toy_deidentified_data.zip

SDNist v2.2

What's Changed

Data:

• Updated description of the feature PINCP_DECILE in the data_dictionary.json

SDNist Library:

• Added new privacy metric unique-exact-matches to compute number of unique target data records that exactly matched records in a deidentified data.
• Reduced report bundle size.
• Added deidentified data labels to the generated json report.
• Update to show only worst three univariates in the k-marginal breakdown section.
• Fixes to the regression metric.
• Few other cosmetic changes to the report UI.

Assets from previous release

• toy_deidentified_data.zip

SDNist v2.1.1

What's Changed

• Updated toy data
• Added PCA metric highlight queries.
• Improved stability of K-marginal scoring, and added k-marginal sub-sampling baseline for each PUMA.
• Added support for including metadata labels in the evaluation report.
• Enhanced data validation: does not evaluate deidentified data features that contain out of bound values.
• Added to report the list of feature that are not included in the evaluation due to out of bound values.
• Removed incorrect child_NOC inconsistency check.
• Changed feature ordering of pair-wise correlation charts to match data dictionary ordering.
• Added colors to the console output for better readability.
• Few other cosmetic changes to the report UI.

SDNist v2.0

This is the stable release of the SDNist Synthetic Report Generator and the SDNist version intended for use in the 2023 Collaborative Research cycle.

What's Changed

• Added new metrics
• Included multiple sample results
• Added some more READMEs (report generator tool and sample report)
• Added better documentation in the READMEs
• Streamlined the repo by removing materials specific to the Temporal Map Challenge.

v1.4.1-b.1: Beta release

This release uses same data resource files as release v1.4.0-b.1
Code and Readme

changes:

• Fix synthetic data binning.
• Fix PCA plot axes.

v1.4.0-b.2: Beta release

This release uses same resource files as release v1.4.0-b.1
Code and Readme

changes:

• Fixed synthetic data validation.
• Fixed apparent match metric when no matches occur.
• Fixed sampling error computation to make sub-samples with same size as the target data.
• Update propensity metric to use tree depth 6.
• Update pearson and kendall correlation to have minimum upper range of 0.15 on colorbars.
• Update k-marginal to perform 100 permutations of 3-marginal selection.

v1.3.0: Stable release with all data resouces

This release is the v1.3.0 stable release and contains all data assets.

Data files in parquet/json and csv are included. Use pip install from the unzipped package to install sdnist. Download SDNist-data-1.3.0 and run sdnist from the unzipped directory to prevent auto attempts to download the SDNist github release 1.3.0.

SDNist-data-1.3.0.zip contains all data tables in parquet/json and csv and also geojson files for census data.

v1.3.0 Chanelog: https://github.com/usnistgov/SDNist/blob/main/CHANGELOG.md
Full Changelog: v0.1.2...v1.3.0

Stable release with all data resouces

This release is the first stable release and contains all data assets.

Data files in parquet/json and csv are included. Use pip install from the unzipped package to install sdnist. Download SDNist-data-1.2.01 and run sdnist from the unzipped directory to prevent attempts to download the data from NIST servers.

SDNist-data-1.2.0.zip contains all data tables in parquet/json and csv.

Full Changelog: v0.1.2...v1.2.0