Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Parallel decoupled Kalman filter training #61

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

[WIP] Parallel decoupled Kalman filter training #61

wants to merge 10 commits into from

Conversation

singraber
Copy link
Member

Implements decoupled Kalman filter (DEKF) NN training. Different levels of decoupling are possible:

  • No decoupling (GEKF), limiting case of single weight group.
  • Element-wise decoupling (ED-GEKF), see also https://doi.org/10.1021/acs.jctc.5b00211
  • Per-layer decoupling.
  • Per-node decoupling (NDEKF).
  • Per-weight decoupling (full decoupling), limiting case.

@singraber singraber added the enhancement New feature or request label Sep 5, 2020
singraber and others added 9 commits September 6, 2020 00:55
@singraber
Fix examples with new keyword.
dce6dd9
@singraber
Added MPI communicator and decoupling group mask
d0a6af3
@singraber
Started with group mask for layer decoupling, WIP!
31bcd58
@singraber
All decoupling group masks look correct
33719c4
@singraber
Introduced new NN weight memory layout
5c540db 
- Weights and biases grouped by neuron.
- Simplifies node decoupling implementation => block submatrices.
- Old weight ordering available via compile flag
(-DALTERNATIVE_WEIGHT_ORDERING)
- Weight file output remains unchanged.
@singraber
Changed decoupling setup
0235ce5 
- Works now with limits instead of a group mask.
- Groups are assumed to be contiguous.
- Implemented rank 0 only update via new communicator.
@singraber
Decoupling on single processor works
0a95ce8
@singraber
Communicate weight segments correctly
11dc116
Merge branch 'master' of github.com:CompPhysVienna/n2p2 into parallel…
9963d14 
…_dekf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@singraber