Skip to content

Commit

Permalink
Update README overview
Browse files Browse the repository at this point in the history
  • Loading branch information
adjavon committed Jul 25, 2024
1 parent 04081c0 commit 84f0a44
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 4 deletions.
20 changes: 16 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
# Exercise 9: Explainable AI and Knowledge Extraction

## Overview
The goal of this exercise is to learn how to probe what a pre-trained classifier has learned about the data it was trained on.

In this exercise we will:
1. Use a gradient-based attribution method to try to find out what parts of an image contribute to its classification
2. Train a CycleGAN to create counterfactual images
3. Run a discriminative attribution from counterfactuals
We will be working with a simple example which is a fun derivation on the MNIST dataset that you will have seen in previous exercises in this course.
Unlike regular MNIST, our dataset is classified not by number, but by color!

![CMNIST](assets/cmnist.png)

In this exercise, we will return to conventional, gradient-based attribution methods to see what they can tell us about what the classifier knows.
We will see that, even for such a simple problem, there is some information that these methods do not give us.

We will then train a generative adversarial network, or GAN, to try to create counterfactual images.
These images are modifications of the originals, which are able to fool the classifier into thinking they come from a different class!.
We will evaluate this GAN using our classifier; Is it really able to change an image's class in a meaningful way?

Finally, we will combine the two methods — attribution and counterfactual — to get a full explanation of what exactly it is that the classifier is doing. We will likely learn whether it can teach us anything, and whether we should trust it!

If time permits, we will try to apply this all over again as a bonus exercise to a much more complex and more biologically relevant problem.

![synister](assets/synister.png)
## Setup

Before anything else, in the super-repository called `DL-MBL-2024`:
Expand Down
Binary file added assets/cmnist.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 84f0a44

Please sign in to comment.