Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate removing reordered variables to reduce memory usage #41

Open
JamieJQuinn opened this issue Feb 19, 2024 · 1 comment
Open
Assignees
Labels
memory Anything that improves memory usage

Comments

@JamieJQuinn
Copy link
Collaborator

@semi-h Can you summarise here?

@Nanoseb Nanoseb added the memory Anything that improves memory usage label Feb 19, 2024
@semi-h
Copy link
Member

semi-h commented Feb 19, 2024

This is very briefly mentioned in #37. It only requires some changes in the solver class, so completely independent from the backends.

We reserve 3 fields for 3 components of velocity field when we instantiate the solver.

x3d2/src/solver.f90

Lines 80 to 82 in 00d0986

solver%u => solver%backend%allocator%get_block()
solver%v => solver%backend%allocator%get_block()
solver%w => solver%backend%allocator%get_block()

So long as solver lives we have 3 field size equivalent memory with us all the time. It makes sense because after all we need to do a timestep after transeq such that
$u_i = u_i + dt \cdot du_i$
and then after pressure is sorted we need to do a pressure correction such that
$u_i = u_i - p_x \cdot \epsilon ??$

So obviously we can't get rid of them. But inside transeq we can play a trick to limit the total amount of temporary storage transeq requires.

We first use the velocity field components here, and obtain all the derivatives we need in x direction.

call self%backend%transeq_x(du, dv, dw, u, v, w, self%xdirps)

Then we have to reorder them into y so that y derivatives can be obtained

x3d2/src/solver.f90

Lines 162 to 164 in 00d0986

call self%backend%reorder(u_y, u, RDR_X2Y)
call self%backend%reorder(v_y, v, RDR_X2Y)
call self%backend%reorder(w_y, w, RDR_X2Y)

And this is the key part. Because at this stage we have a copy of velocity field in y orientation as well as the original arrays in x orientation, we can actually get rid of the memory of the original arrays.

If we do this then the reordering below will have to go from y to z.

x3d2/src/solver.f90

Lines 194 to 196 in 00d0986

call self%backend%reorder(u_z, u, RDR_X2Z)
call self%backend%reorder(v_z, v, RDR_X2Z)
call self%backend%reorder(w_z, w, RDR_X2Z)

And then at the very end we'll have to reorder z->x so that the original u, v, and w fields are restrored before returning back to run method in solver.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
memory Anything that improves memory usage
Projects
None yet
Development

No branches or pull requests

3 participants