ROCm packages: externals consistency

[scheibelp]

@rfhaque

(This PR targets another PR: #61)

When ROCm components are added as externals, we can't enforce consistency among the versions because externals do not consider dependencies (so even though the Spack hip builtin package enforces version locking for all dependencies, that is lost when it is external).

This creates a mixin rocm-consistency package we can use to enforce a consistent choice of versions among rocm components. This avoids the addition of version-aware Ramble identifiers in the experiments (those are removed here).

[rfhaque]

This works for rocm/5.4.3. However, if I change the compiler to [email protected], the spack concretizer still uses v5.4.3 for some packages. Following is the concretizer output for [email protected]

==> Concretized [email protected]%[email protected]~double~hwloc+mpi+openmp+rocm+sycl amdgpu_target=gfx90a
 -   2fjfk3y  [email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cp2k~cuda~cycle_subcounters~double+gmxapi~heffte~hwloc~intel-data-center-gpu-max~intel_provided_gcc~ipo~mdrun_only+mpi+nblib~nosuffix~opencl+openmp~plumed~relaxed_double_precision+rocm+shared+sycl amdgpu_target=gfx90a build_system=cmake build_type=Release generator=make openmp_max_threads=none arch=linux-rhel8-x86_64
[e]  jmwvqsy      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rhel8-x86_64
[e]  elntlub      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2'  build_system=cmake build_type=Release generator=make arch=linux-rhel8-x86_64
[e]  v42qoop      ^[email protected]%[email protected]~gtl+wrappers build_system=generic arch=linux-rhel8-x86_64
[e]  6nfuiwx      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~guile build_system=generic patches=ca60bd9,fe5b60d arch=linux-rhel8-x86_64
[e]  43ygv4w      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cuda+rocm build_system=cmake build_type=Release generator=make patches=5068750,c2ee21c,ca523f1,ddd86f0 arch=linux-rhel8-x86_64
 -   uisys2n      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cuda~ipo+rocm amdgpu_target=gfx90a build_system=cmake build_type=Release generator=make arch=linux-rhel8-x86_64
 -   rzxtvzu          ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~atomic~chrono~clanglibcpp~container+context~contract~coroutine~date_time~debug~exception+fiber+filesystem~graph~graph_parallel~icu~iostreams~json~locale~log~math~mpi+multithreaded~nowide~numpy~pic~program_options~python~random~regex~serialization+shared~signals~singlethreaded~stacktrace~system~taggedlayout~test~thread~timer~type_erasure~versionedlayout~wave build_system=generic context-impl=fcontext cxxstd=17 patches=3a83d90,5da7ad2,607b077 visibility=hidden arch=linux-rhel8-x86_64
[e]  goajfe5          ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' +bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,4c24573,ebdca64,f2fd060 arch=linux-rhel8-x86_64
[e]  qd5rbts      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~asan+image+shared build_system=cmake build_type=Release generator=make patches=9267179 arch=linux-rhel8-x86_64
[e]  3f7dz3g      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~asan+shared build_system=cmake build_type=Release generator=make patches=114b05a arch=linux-rhel8-x86_64
[e]  e4ljwkx      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cluster+envmods~ilp64+shared build_system=generic mpi_family=none threads=none arch=linux-rhel8-x86_64
[e]  5554olq      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~link_llvm_dylib~llvm_dylib~openmp+rocm-device-libs build_system=cmake build_type=Release generator=ninja patches=a08bbe1 arch=linux-rhel8-x86_64
[e]  tsvmocc      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2'  build_system=cmake build_type=Release generator=make arch=linux-rhel8-x86_64
[e]  6bvget5      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2'  build_system=generic arch=linux-rhel8-x86_64

==> Concretized [email protected]%[email protected]+rocm
 -   uisys2n  [email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cuda~ipo+rocm amdgpu_target=gfx90a build_system=cmake build_type=Release generator=make arch=linux-rhel8-x86_64
 -   rzxtvzu      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~atomic~chrono~clanglibcpp~container+context~contract~coroutine~date_time~debug~exception+fiber+filesystem~graph~graph_parallel~icu~iostreams~json~locale~log~math~mpi+multithreaded~nowide~numpy~pic~program_options~python~random~regex~serialization+shared~signals~singlethreaded~stacktrace~system~taggedlayout~test~thread~timer~type_erasure~versionedlayout~wave build_system=generic context-impl=fcontext cxxstd=17 patches=3a83d90,5da7ad2,607b077 visibility=hidden arch=linux-rhel8-x86_64
[e]  jmwvqsy      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rhel8-x86_64
[e]  6nfuiwx      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~guile build_system=generic patches=ca60bd9,fe5b60d arch=linux-rhel8-x86_64
[e]  43ygv4w      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cuda+rocm build_system=cmake build_type=Release generator=make patches=5068750,c2ee21c,ca523f1,ddd86f0 arch=linux-rhel8-x86_64
[e]  qd5rbts      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~asan+image+shared build_system=cmake build_type=Release generator=make patches=9267179 arch=linux-rhel8-x86_64
[e]  5554olq      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~link_llvm_dylib~llvm_dylib~openmp+rocm-device-libs build_system=cmake build_type=Release generator=ninja patches=a08bbe1 arch=linux-rhel8-x86_64
[e]  goajfe5      ^[email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' +bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,4c24573,ebdca64,f2fd060 arch=linux-rhel8-x86_64

==> Concretized [email protected]
[e]  e4ljwkx  [email protected]%[email protected] cflags='-g -O2' cxxflags='-g -O2' ~cluster+envmods~ilp64+shared build_system=generic mpi_family=none threads=none arch=linux-rhel8-x86_64

[scheibelp]

Thanks: the components were synchronized with each other WRT version, but not with the compiler version. I added another constraint for that in 1f16b8b So when building with %[email protected], we will use ^[email protected].

I'll note that with this change specifically, Benchpark constraints for rocmcc/rocm-externals are now more strict than within the non-external Spack builtin case (there's nothing inherently wrong with that).

[rfhaque]

Would this still work for a non-rocmcc compiler, say cce?

[scheibelp]

Yes: this does not force the use of %rocmcc, it just makes sure that if you use %rocmcc, that the compiler version matches the component version.

[rfhaque]

The spack concretization works for consistently for both rocm 5.4 and 5.5

[pearce8]

@scheibelp thank you, this looks like a good solution.

What are your thoughts on upstreaming this to Spack? Are you planning to upstream rocm-consistency package? Looks like we would need to if we wanted to upstream the Gromacs package with the dependence on rocm-consistency package.

[scheibelp]

What are your thoughts on upstreaming this to Spack? Are you planning to upstream rocm-consistency package?

I'd like to explore a more general mechanism in Spack vs. upstreaming rocm-consistency there (i.e. I consider rocm-consistency a stopgap).

Looks like we would need to if we wanted to upstream the Gromacs package with the dependence on rocm-consistency package.

Most of the changes related to building with ROCm I can upstream independently. FWIW, other packages in benchpark that we build with ROCm support will need to use this mixin as well - in those cases they will look much simpler:

import benchpark.RocmConsistency
import builtin.Foo

class Foo(builtin.Foo, RocmConsistency):
    pass

[pearce8]

Sounds good. Feel free to merge.