Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New label-balancing weight scheme; option to use specific samples for calibration #170

Merged
merged 32 commits into from
Jan 14, 2025
Merged

New label-balancing weight scheme; option to use specific samples for calibration #170

merged 32 commits into from
Jan 14, 2025

Conversation

davidbenjamin
Copy link
Owner

No description provided.

@davidbenjamin
acc_vs_logit is by source
76a02fb
@davidbenjamin
ditto for acc_vs_logit_all_counts
48ecce6
@davidbenjamin
ditto for acc_vs_cnt
d490972
@davidbenjamin
ditto for roc_data
1761206
@davidbenjamin
ditto for roc_data_by_cnt
5264dd0
@davidbenjamin
each source gets its own set of plots
d36ff24
@davidbenjamin
Reduce figure size a bit
7c1f66c
@davidbenjamin
dataset totals and weights are now balanced within each source
a30fb53
@davidbenjamin
incomplete work on soure weights
369bbb4
@davidbenjamin
eliminated a nested for loop
cf10929
@davidbenjamin
more incomplete work -- removed a sqrt in the weights that I thinkw a…
5c89f71 
…s a bug
@davidbenjamin
finished the usperior sweeights refactor stuff on source weights
4719d51
@davidbenjamin
BaseDataset batch sampler can restrict to specific sources
742f089
@davidbenjamin
likewise for BaseDataset::make_data_laoder
0ad13fe
@davidbenjamin
likewise for artifact batch sampler and artifact dataset make_data_lo…
cda9b70 
…ader
@davidbenjamin
maybe improved weight-balancing
29376c8
@davidbenjamin
added a calibration sources argument to artifact model training
0b397c5
@davidbenjamin
now there is a distinct calibration loader. Not using it yet.
9cd647e
@davidbenjamin
torch.full size bug fix
a53d48e
@davidbenjamin
calibration epochs use calibration loader
03a9997
@davidbenjamin
using that calibration loader for the final calibration step, but is …
0cfd8da 
…this sensible with unlabeled data?
@davidbenjamin
learned parameter for proportion on unbalanced data that are artifacts
43aff82
@davidbenjamin
recalculate new artifact and nonartifact label-balancing weights at e…
a663435 
…ach batch
@davidbenjamin
using the new label-balancing weights
258fbbc
@davidbenjamin
whoops
928ee18
@davidbenjamin
deleted final logit adjustment step
d27cfa7
@davidbenjamin
deleted the seT_adjustments method
92a1e6e
@davidbenjamin
final adjustments are always turned on
e6658d6
@davidbenjamin
calibration epochs only adjust the final shift
d5388dc
@davidbenjamin
Whoops
578bf56
@davidbenjamin davidbenjamin merged commit 32d587d into master Jan 14, 2025
@davidbenjamin davidbenjamin deleted the db_calibration_on_specific_source branch January 14, 2025 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@davidbenjamin