Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transition to arraycontext #139

Draft
wants to merge 62 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
791dfd3
Initial work towards using array contexts
inducer May 13, 2022
2da9092
update tests to pass actx
alexfikl Sep 3, 2022
3123ec7
sumpy.array_context additions
alexfikl Sep 3, 2022
ecc9d7f
port p2p to arraycontext
alexfikl Sep 3, 2022
2285243
port p2e to arraycontext
alexfikl Sep 3, 2022
c87c528
port e2p to arraycontext
alexfikl Sep 3, 2022
2aa656b
port e2e to arraycontext
alexfikl Sep 3, 2022
57971a5
port tools and toys to arraycontext
alexfikl Sep 3, 2022
1abff07
port test_tools to arraycontext
alexfikl Sep 3, 2022
5d95944
port test_misc to arraycontext
alexfikl Sep 3, 2022
9b4fbde
bump requirements
alexfikl Sep 4, 2022
71e6c65
start porting test_kernels to arraycontext
alexfikl Sep 4, 2022
7086997
more porting in test_kernels
alexfikl Sep 5, 2022
4523020
add arraycontext to docs
alexfikl Sep 5, 2022
aa5a50b
add some annotations to make_loopy_program
alexfikl Sep 5, 2022
bfce329
finish porting in test_kernels
alexfikl Sep 5, 2022
80b1045
port qbx to arraycontext
alexfikl Sep 5, 2022
81b92bd
port curve-pot to arraycontext
alexfikl Sep 5, 2022
5cc1919
add pytools to intersphinx
alexfikl Sep 5, 2022
04bb78a
add assumptions at kernel creation
alexfikl Sep 5, 2022
dd9e7d9
port expansion-toys to arraycontext
alexfikl Sep 5, 2022
5bba7e2
move get_kernel calls to separate line for debugging
alexfikl Sep 6, 2022
2332fd5
add fixed_parameters to make_loopy_program
alexfikl Sep 6, 2022
6da8704
port test_qbx to arraycontext
alexfikl Sep 6, 2022
4aca058
port test_matrixgen to arraycontext
alexfikl Sep 6, 2022
711f938
continue porting fmm to arraycontext
alexfikl Sep 6, 2022
276b42a
update drive_fmm from boxtree
alexfikl Sep 8, 2022
3f8b2b8
Merge branch 'main' into towards-array-context
alexfikl Sep 17, 2022
4c1985d
more work towards getting the fmm working
alexfikl Sep 17, 2022
2c93c4e
fix up fmm tests
alexfikl Sep 17, 2022
ddaa4ad
port distributed to arraycontext
alexfikl Sep 17, 2022
be9c909
add missing actx
alexfikl Sep 18, 2022
fd7e61f
fix kernel return values
alexfikl Sep 21, 2022
c3e35a5
actually loop over all results
alexfikl Sep 21, 2022
f6d6e9d
back up some more dictionary kernel accesses
alexfikl Sep 21, 2022
0fa102f
fix matrix generation
alexfikl Sep 21, 2022
a7bb63a
rip out timing collection
alexfikl Sep 25, 2022
15a15f2
Merge branch 'main' into towards-array-context
alexfikl Sep 25, 2022
567b947
remove unused imports (flake8)
alexfikl Sep 25, 2022
ea7656d
fix return value for form_locals
alexfikl Sep 25, 2022
2fd594d
Merge branch 'main' into towards-array-context
alexfikl Sep 26, 2022
bd5f578
remove ctx arg in KernelComputation
alexfikl Sep 26, 2022
4c3d838
back out some unneeded changes
alexfikl Sep 29, 2022
60e194d
point ci to updated pytential
alexfikl Sep 29, 2022
45a023e
fix kwargs name
alexfikl Sep 29, 2022
39d0757
Merge branch 'main' into towards-array-context
alexfikl Oct 17, 2022
5cbce16
Merge branch 'main' into towards-array-context
alexfikl Oct 30, 2022
e28d295
fix merge
alexfikl Oct 30, 2022
f1efc3a
Merge branch 'main' into towards-array-context
alexfikl Nov 6, 2022
447335d
Merge branch 'main' into towards-array-context
alexfikl Nov 28, 2022
b7df3ce
Merge branch 'main' into towards-array-context
alexfikl Jan 11, 2023
8397a93
Merge branch 'main' into towards-array-context
alexfikl Jan 31, 2023
237dad4
Merge branch 'main' into towards-array-context
alexfikl Apr 4, 2023
6f65431
Merge branch 'main' into towards-array-context
alexfikl Apr 26, 2023
53933e7
docs: add pytools to intersphinx
alexfikl Apr 28, 2023
b6b3e2e
Merge branch 'main' into towards-array-context
alexfikl Jun 16, 2023
b7203e2
fix device handling in p2p
alexfikl Jun 16, 2023
43148e1
Merge branch 'main' into towards-array-context
alexfikl Aug 2, 2023
a791aaf
Merge branch 'main' into towards-array-context
alexfikl Aug 5, 2023
51ac5f5
fix bad merge
alexfikl Aug 5, 2023
d88b9f4
Merge branch 'main' into towards-array-context
alexfikl Oct 17, 2023
4fe4f01
fix bad merge
alexfikl Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 8 additions & 83 deletions sumpy/fmm.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,6 @@ class SumpyTreeIndependentDataForWrangler(TreeIndependentDataForWrangler):
necessarily must have an :class:`arraycontext.ArrayContext`, but it
is allowed to be more ephemeral than the code, the code's lifetime
is decoupled by storing it in this object.

Timing results returned by this wrangler contain the values *wall_elapsed*
which measures elapsed wall time. This requires an array container with
profiling enabled.
"""

def __init__(self, array_context: PyOpenCLArrayContext,
Expand Down Expand Up @@ -186,69 +182,6 @@ def opencl_fft_app(self, shape, dtype, inverse):
# }}}


# {{{ timing future

_SECONDS_PER_NANOSECOND = 1e-9


"""
EventLike objects have an attribute native_event that returns
a cl.Event that indicates the end of the event.
"""
EventLike = TypeVar("CLEventLike")


class UnableToCollectTimingData(UserWarning):
pass


class SumpyTimingFuture:

def __init__(self, queue, events: List[EventLike]):
self.queue = queue
self.events = events

@property
def native_events(self) -> List[EventLike]:
import pyopencl as cl
return [evt if isinstance(evt, cl.Event) else evt.native_event
for evt in self.events]

@memoize_method
def result(self):
import pyopencl as cl
from boxtree.timing import TimingResult

if not self.queue.properties & cl.command_queue_properties.PROFILING_ENABLE:
from warnings import warn
warn(
"Profiling was not enabled in the command queue. "
"Timing data will not be collected.",
category=UnableToCollectTimingData,
stacklevel=3)
return TimingResult(wall_elapsed=None)

if self.events:
cl.wait_for_events(self.native_events)

result = 0
for event in self.events:
result += (
(event.profile.end - event.profile.start)
* _SECONDS_PER_NANOSECOND)

return TimingResult(wall_elapsed=result)

def done(self):
import pyopencl as cl
return all(
event.get_info(cl.event_info.COMMAND_EXECUTION_STATUS)
== cl.command_execution_status.COMPLETE
for event in self.native_events)

# }}}


# {{{ expansion wrangler

class SumpyExpansionWrangler(ExpansionWranglerInterface):
Expand Down Expand Up @@ -284,7 +217,6 @@ def __init__(self, tree_indep, traversal, dtype, fmm_level_to_order,
preprocessed_mpole_dtype=None,
*, _disable_translation_classes=False):
super().__init__(tree_indep, traversal)
self.issued_timing_data_warning = False

self.dtype = dtype

Expand Down Expand Up @@ -609,8 +541,7 @@ def form_multipoles(self,

assert mpoles_res is mpoles_view

# FIXME: rip out SumpyTimingFuture
return mpoles, SumpyTimingFuture(actx.queue, [])
return mpoles

def coarsen_multipoles(self,
actx: PyOpenCLArrayContext,
Expand Down Expand Up @@ -664,7 +595,7 @@ def coarsen_multipoles(self,

assert mpoles_res is target_mpoles_view

return mpoles, SumpyTimingFuture(actx.queue, [])
return mpoles

def eval_direct(self,
actx: PyOpenCLArrayContext,
Expand All @@ -690,7 +621,7 @@ def eval_direct(self,
for pot_i, pot_res_i in zip(pot, pot_res):
assert pot_i is pot_res_i

return pot, SumpyTimingFuture(actx.queue, [])
return pot

@memoize_method
def multipole_to_local_precompute(self):
Expand Down Expand Up @@ -789,10 +720,6 @@ def multipole_to_local(self,
mpole_exps_view_func = self.multipole_expansions_view
local_exps_view_func = self.local_expansions_view

preprocess_evts = []
translate_evts = []
postprocess_evts = []

for lev in range(self.tree.nlevels):
wait_for = []

Expand Down Expand Up @@ -894,9 +821,7 @@ def multipole_to_local(self,
**self.kernel_extra_kwargs,
)

timing_events = preprocess_evts + translate_evts + postprocess_evts

return (local_exps, SumpyTimingFuture(actx.queue, timing_events))
return local_exps

def eval_multipoles(self,
actx: PyOpenCLArrayContext,
Expand Down Expand Up @@ -937,7 +862,7 @@ def eval_multipoles(self,
for pot_i, pot_res_i in zip(pot, pot_res):
assert pot_i is pot_res_i

return pot, SumpyTimingFuture(actx.queue, [])
return pot

def form_locals(self,
actx: PyOpenCLArrayContext,
Expand Down Expand Up @@ -978,7 +903,7 @@ def form_locals(self,

assert result is target_local_exps_view

return local_exps, SumpyTimingFuture(actx.queue, [])
return local_exps

def refine_locals(self,
actx: PyOpenCLArrayContext,
Expand Down Expand Up @@ -1021,7 +946,7 @@ def refine_locals(self,

assert local_exps_res is target_local_exps_view

return local_exps, SumpyTimingFuture(actx.queue, [])
return local_exps

def eval_locals(self,
actx: PyOpenCLArrayContext,
Expand Down Expand Up @@ -1059,7 +984,7 @@ def eval_locals(self,
for pot_i, pot_res_i in zip(pot, pot_res):
assert pot_i is pot_res_i

return pot, SumpyTimingFuture(actx.queue, [])
return pot

def finalize_potentials(self, actx: PyOpenCLArrayContext, potentials):
return potentials
Expand Down
5 changes: 1 addition & 4 deletions test/test_distributed.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,7 @@ def wrangler_factory(local_traversal, global_traversal):
distribued_fmm_info = DistributedFMMRunner(
actx, global_tree_dev, traversal_builder, wrangler_factory, comm=comm)

timing_data = {}
distributed_potential = distribued_fmm_info.drive_dfmm(
actx, [sources_weights], timing_data=timing_data)
assert timing_data
distributed_potential = distribued_fmm_info.drive_dfmm(actx, [sources_weights])

if mpi_rank == 0:
assert shmem_potential.shape == (1,)
Expand Down
6 changes: 1 addition & 5 deletions test/test_fmm.py
Original file line number Diff line number Diff line change
Expand Up @@ -485,11 +485,7 @@ def test_sumpy_fmm_timing_data_collection(ctx_factory, use_fft, visualize=False)
fmm_level_to_order=lambda kernel, kernel_args, tree, lev: order)
from boxtree.fmm import drive_fmm

timing_data = {}
pot, = drive_fmm(actx, wrangler, (weights,), timing_data=timing_data)
logger.info("timing_data:\n%s", timing_data)

assert timing_data
pot, = drive_fmm(actx, wrangler, (weights,))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. What's the recommended way to get the timing data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure. I briefly talked about that with @inducer and it should be baked into the array context somehow so it doesn't clutter the wrangler interface.

However, that probably needs a bit of effort to get back into a working state, so not quite sure what a timeline would be..

Copy link
Contributor Author

@alexfikl alexfikl Sep 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to write out some questions about that to get an idea (from you and @inducer). I don't really use it so this has been a brutal delete sort of thing..

  • Do you rely on that interface to get results?
  • Can this be merged without it?
  • Who's going to do the work to put it back? 😢

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you rely on that interface to get results?

Yes, I use it to get results for a paper.

Who's going to do the work to put it back?

I can try if there's an outline of how to reintroduce it. I have not much experience on arraycontexts, so I have no idea what needs to be done.

Copy link
Contributor Author

@alexfikl alexfikl Sep 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try if there's an outline of how to reintroduce it. I have not much experience on arraycontexts, so I have no idea what needs to be done.

Yeah, I don't have a good overview either, so that probably needs some discussion. There's a profiling array context here
https://github.com/illinois-ceesd/mirgecom/blob/f9cf0f3557b7d5c89f58e8362ebf50820261219f/mirgecom/profiling.py
but not sure if a similar interface will work for gathering timings on the FMM.

Yes, I use it to get results for a paper.

Ok, then definitely either revert or hold off on merging.

At the moment, the timing code and the manual event handling are not particularly compatible with the arraycontext stuff (as it is) and removing both is pretty breaking.



def test_sumpy_fmm_exclude_self(actx_factory, visualize=False):
Expand Down