TMatrixSampler Error Comparison

[alexlafleur]

[greenTara]

There is no text here, just plots. Can this issue be closed?

[alexlafleur]

These plots show the differences in errors from our error calculation (left) and the error calculation that was based on the count matrix from TransitionMatrixSampler(C=C1bayes).
I think it can be closed.

[greenTara]

There was something being tested in this check, and it doesn't look like we have brought this test to a satisfactory conclusion, unless that is documented elsewhere.

[greenTara]

As I recall, the question was about how the error estimate that comes from the bayes approach - without taking advantage of our knowledge of the ground truth transition matrix - compares to the error estimate that does take advantage of that knowledge. In looking at these graphs, I recall that we found a bug in the test, because the behavior was not as expected. But I don't recall where the rest of the development was documented.

[alexlafleur]

There is this script called "test_tmatrix_sampler", but we did not document anything for that as far as I remember.

[greenTara]

I checked the script test_tmatrix_sampler. It is basically checking that the bayesian uncertainty estimation coming from the count matrix is roughly the same as the actual error that we calculate based on the known model. I think the plots are out of date in the issue, but in the end the test was succeeding - there is an assertion in there checking at the errors are not too different.(ratio between 1/4 and 4).