Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Interfacing ML code with MiMA #10

Open
wants to merge 63 commits into
base: master
Choose a base branch
from
Open

Feature: Interfacing ML code with MiMA #10

wants to merge 63 commits into from

Conversation

mondus
Copy link

@mondus mondus commented May 15, 2023

Closes #7

@jatkinson1000 - covering #4 and #5

  • Adapt cg_drag.f90 to accommodate drag forcing from various different approaches
  • Create .f90 files with modules & subroutines for relevant drag calculations
  • Adapt PT NN to receive individual tensors of each variable and combine in a prepended layer
    This way each approach will call the NN twice, once with zonal, once with meridional, but tensors will only be written to shared memory once.

@SimonClifford - covering #3

  • Inspect PT model and construct matching TF model
  • Migrate weights
  • Add TF code into MiMA (requires @jatkinson1000 to write .f90 files above)

We then need to conduct benchmarking:

  • Instrument MiMA for ReBench
  • Plan a set of runs
  • Perform the set of runs

jatkinson1000 and others added 30 commits January 4, 2023 15:46
Update to reflect model now takes 3 inputs and concatenates
them, rather than a pre-concatted input.

Test is very basic but shows that TF davenet does the same
thing as PyTorch davenet.
@SimonClifford
Copy link

Closes #13 in the early @SimonClifford commits.

@SimonClifford
Copy link

@SimonClifford commits address TensorFlow coupling.
Closes #12
Closes #3

@SimonClifford
Copy link

@jatkinson1000 commits close #11 and close #12 and therefore close #4 .

@SimonClifford
Copy link

@jatkinson1000 closes #5, although we may revisit.

@jatkinson1000
Copy link
Member

Also closes #1 in earliest @jatkinson1000 commits (we could merge cmake into this branch to remove unused dependencies that caused failure on Sherlock).

CMake docs are included in docs/GettingStarted.md.

Copy link
Author

@mondus mondus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small suggestions.

  1. Move the CSD3 submissions scrips into a scripts\csd3 folder as they are hardware specific.
  2. Remove hardcoded home directories in the submissions scripts. Replace with a variable or argument.
  3. pytorch README.md could do with a bit more detail on what the scripts are. This could be opened as an issue for future work though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Interfacing MiMA with ML codes
3 participants