Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RELEASE] cugraph v24.02 #4142

Merged
merged 73 commits into from
Feb 13, 2024
Merged

[RELEASE] cugraph v24.02 #4142

merged 73 commits into from
Feb 13, 2024

Conversation

raydouglass
Copy link
Member

❄️ Code freeze for branch-24.02 and v24.02 release

What does this mean?

Only critical/hotfix level issues should be merged into branch-24.02 until release (merging of this PR).

What is the purpose of this PR?

  • Update documentation
  • Allow testing for the new release
  • Enable a means to merge branch-24.02 into main for the release

raydouglass and others added 30 commits November 14, 2023 11:31
@raydouglass
v24.02 Updates [skip ci]
2f44a9c
@GPUtester
Merge pull request #3997 from rapidsai/branch-23.12
264fc79 
Forward-merge branch-23.12 to branch-24.02
@vyasr
Merge branch 'branch-23.12' into branch-24.02-merge-branch-23.12
ce6a801
@AyodeAwe
Merge pull request #4012 from vyasr/branch-24.02-merge-branch-23.12
8e213ec 
Branch 24.02 merge branch 23.12
@AyodeAwe
Merge pull request #3999 from rapidsai/branch-23.12
0dbdc9b 
Forward-merge branch-23.12 to branch-24.02
@GPUtester
Merge pull request #4016 from rapidsai/branch-23.12
64fd4c7 
Forward-merge branch-23.12 to branch-24.02
@GPUtester
Merge pull request #4017 from rapidsai/branch-23.12
942d5ee 
Forward-merge branch-23.12 to branch-24.02
@rlratzel
Merge branch-23.12 into branch-24.02
b2731e2
@rlratzel
Updates version for 24.02
0a29752
@rlratzel
Merge remote-tracking branch 'upstream/branch-23.12' into branch-24.0…
5047ffd 
…2-merge-23.12
@AyodeAwe
Merge pull request #4020 from rlratzel/branch-24.02-merge-23.12
5d4bcce 
Resolves conflicts from forward-merging branch-23.12 into branch-24.02
@GPUtester
Merge pull request #4023 from rapidsai/branch-23.12
96c39cc 
Forward-merge branch-23.12 to branch-24.02
@rlratzel
Removes unsupported setup.py calls, cleans up text (#4024)
6041ab9 
closes #4008 

* Removes unsupported `setup.py` calls, which resulted in issue #4008 . The document now describes how to use `build.sh`, which is more consistent with docs for other RAPIDS libs.
* Removes `build.sh` output from docs in favor of documenting how to use `--help` to eliminate the doc being out-of-date with the script.
* Removes extra wording in places and cleans up various sections.

Authors:
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Don Acosta (https://github.com/acostadon)
  - Brad Rees (https://github.com/BradReesWork)

URL: #4024
@seunghwak
Update count_if_e, transform_reduce_e, and transform_e to support edg…
5eaae7d 
…e masking (#4001)

Update transform_e to support graphs with edge masking. This is necessary for K-truss.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Naim (https://github.com/naimnv)

Approvers:
  - Naim (https://github.com/naimnv)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4001
@ChuckHastings
Mtmg updates for rmm (#4031)
c6aa981 
Discovered RAII capabilities in RMM while reviewing issues related to MTMG testing.  This PR modifies the MTMG implementation to use the RAII capabilities for setting the device id temporarily to another device.

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)

URL: #4031
@seunghwak
Fix % 0 bug in MG_SELECT_RANDOM_VERTICES test (#4034)
90fec71 
`% 0` throws an exception (`mg_graph_view.local_vertex_partition_range_size()` & `num_of_elements_in_given_set` can be 0). This PR fixes the bug.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4034
@GPUtester
Merge pull request #4039 from rapidsai/branch-23.12
20145b4 
Forward-merge branch-23.12 to branch-24.02
@ChuckHastings
Clean up self-loop and multi-edge removal logic (#4032)
32eaa5e 
There are mask utilities that perform some of the functions that were implemented to do this cleanup.

Use the mask utilities instead of replicating functionality.

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)

URL: #4032
@GPUtester
Merge pull request #4049 from rapidsai/branch-23.12
84999be 
Forward-merge branch-23.12 to branch-24.02
@ajschmidt8
Prevent automatic labeler from adding Label Checker labels (#4048)
783e4dc 
This PR prevents the `doc` label from being automatically added to PRs since it can interfere with the [Label Checker](https://docs.rapids.ai/resources/label-checker/) check.

[skip ci]
@ChuckHastings
Add support for Louvain to MTMG (#4033)
f80480d 
Added vertex result instantiation for `vertex_t` which is necessary to handle Louvain results.

Added an MTMG test for Louvain to demonstrate how to use Louvain in MTMG.

Closes rapidsai/graph_dl#330

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Joseph Nke (https://github.com/jnke2016)
  - Naim (https://github.com/naimnv)

URL: #4033
@VibhuJawa
Add a barrier before cugraph Graph creation (#4046)
a5718c6 
This PR introduces a short term fix for #4037 . 


CC: @jnke2016 , @rlratzel

Authors:
  - Vibhu Jawa (https://github.com/VibhuJawa)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4046
@seunghwak
Fix HITS convergence error. (#4043)
65df1a2 
We set epsilon in hits call to 1e-6 or 1e-8 in our tests. HITS internally uses max. norm to normalize HITS values after each iteration before computing HITS value changes in two consecutive iterations. Sum of HITS values tends to grow with the number of vertices. Using a fixed epsilon leads to convergence failure in large graphs.

This PR updates HITS to compare sum of HITS value changes in two consecutive iterations with `epsilon` * graph_view.number_of_vertices() following networkx documentation (https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.link_analysis.hits_alg.hits.html).

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Joseph Nke (https://github.com/jnke2016)
  - Naim (https://github.com/naimnv)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4043
@vyasr
Remove CUGRAPH_BUILD_WHEELS and standardize Python builds (#4041)
c709fc9 
Some minor simplification in advance of the scikit-build-core migration to better align wheel and non-wheel Python builds.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - Brad Rees (https://github.com/BradReesWork)

URL: #4041
@vyasr
Update dependencies.yaml to new pip index (#4045)
1df6217 
This PR changes all references to pypi.nvidia.com to pypi.anaconda.org/rapidsai-wheels-nightly.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: #4045
@seunghwak
Update for CCCL 2.x (#3862)
18ab76b 
This PR adds `cuda::proclaim_return_type` to device lambdas used in `thrust::transform` and `thrust::make_transform_iterator`.

This PR requires libcudacxx 2.1.0, which was provided by rapidsai/rapids-cmake#464.

Closes #3863.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #3862
@tingyu66
Add HeteroGATConv to cugraph-pyg (#3914)
1655003 
Fixes rapidsai/graph_dl#311
Adding @stadlmax POC code to cugraph-pyg

Authors:
  - Tingyu Wang (https://github.com/tingyu66)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Alex Barghi (https://github.com/alexbarghi-nv)

URL: #3914
@eriknw
nx-cugraph: update usage of nodes_or_number for nx compat (#4028)
c637b33 
These changes will be necessary when networkx/networkx#7066 is merged.

Authors:
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4028
@seunghwak
Replace graph_view.hpp::number_of_edges with compute_number_of_edges (#…
7fe7bea 
…4026)

Replace graph_view.hpp::number_of_edges (deprecated, throws an exception if an edge mask is attached to the graph view object) with compute_number_of_edges (this function works with or without edge mask)

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Naim (https://github.com/naimnv)

Approvers:
  - Joseph Nke (https://github.com/jnke2016)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Naim (https://github.com/naimnv)

URL: #4026
@naimnv
Test select_random_vertices for all possible values of flags (#4042)
23e70a6 
Test `select_random_vertices` for all possible values of flags.

Authors:
  - Naim (https://github.com/naimnv)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4042
eriknw and others added 13 commits January 24, 2024 22:54
@eriknw
nx-cugraph: rename plc= to _plc= (#4106)
82552ab 
As discussed here: #4093 (comment)

Authors:
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4106
@seunghwak
Implement has_edge() & compute_multiplicity() (#4096)
3526af4 
`graph_view.has_edge()` query whether the graph has given (src, dst) pairs as edges.

`graph_view.compute_multiplicity()` query the edge multiplicity of given (src, dst) pairs (assumes that multiplicity is 0 if there is no edge between a given pair). This function throws an exception if `graph_view.is_multigraph()` is false (better use `has_edge()` for non-multigraph).

In addition to adding the above two functions, this PR includes few code cleanups.

* `major_idx_from_major_nocheck()` to `edge_partition_device_view_t`
* Move `count_invalid_vertex_pais` from `nbr_intersection.cuh` to `error_check_utils.cuh`
* Update `cugraph::test::to_host`, `to_device`, `device_gatherv`, and `device_allgatherv` to support `bool` type (and to handle `std::vector<bool>` which stores `bool` values in a packed format)

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Naim (https://github.com/naimnv)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4096
@eriknw
nx-cugraph: add complement and reverse (#4103)
9a261ff 
We apparently already had `G.reverse()`, which made that function extra easy :)

Authors:
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4103
@seunghwak
Update per_v_transform_reduce_incoming|outgoing_e to support edge mas…
ff76a38 
…king (#4085)

per_v_transform_reduce_(incoming|outgoing_e) now supports edge masking.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Naim (https://github.com/naimnv)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4085
@tingyu66
Create cugraph-equivariant package (#4036)
3ff2abd 
Bring up `cugraph-equivariant` package and add TensorProduct conv layers.

Authors:
  - Tingyu Wang (https://github.com/tingyu66)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)
  - https://github.com/DejunL
  - Maximilian Stadler (https://github.com/stadlmax)
  - Mario Geiger (https://github.com/mariogeiger)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4036
@rlratzel
Adds option to rapids_cpm_find for raft to disable hnswlib feature, a…
b1495ed 
…dds updates for pytest 8 compat, temporarily skips IO intensive test in CI (#4121)

* Adds option to rapids_cpm_find for raft to disable hnswlib feature, which is not needed by cugraph.  This resolves cmake failures such as the following:
   ```
        CMake Error at /home/coder/cugraph/cpp/build/release/_deps/raft-build/raft-targets.cmake:56 (set_target_properties):
         The link interface of target "raft::raft" contains:
  
           hnswlib::hnswlib
  
  
         but the target was not found. 
   ```
* TEMPORARILY skips bulk sampling IO tests which were hanging on CI runners. These tests pass locally, so we believe this is related to IO issues on CI runners, but will investigate further in 24.04
* Adds updates for pytest >=8
  * Adds option to pytest to append source paths to PYTHONPATH instead of prepend (the default). The prepended PYTHONPATH caused the dask `LocalCUDACluster` to import cugraph modules from the source tree instead of the install location, which resulted in `ImportError`s. This appears to be a behavior change in pytest 8.
  * Adds an `xfail` for an [NX test known to be incompatible with pytest>=8](networkx/networkx#7203)

Authors:
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Divye Gala (https://github.com/divyegala)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Ray Douglass (https://github.com/raydouglass)
  - Erik Welch (https://github.com/eriknw)

URL: #4121
@rlratzel
Adds benchmarks for additional nx-cugraph 24.02 algos (#4112)
ba73b02 
* Adds 23 benchmarks for BFS, connected components, triangles, and other related algos
* Adds a shell script to run the algo benchmarks individually for easier comparison to NX
* Fixes a bug in `nx_cugraph.generic_bfs_edges()` uncovered by these benchmarks

Authors:
  - Rick Ratzel (https://github.com/rlratzel)
  - Erik Welch (https://github.com/eriknw)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Erik Welch (https://github.com/eriknw)

URL: #4112
@eriknw
nx-cugraph: use coverage to ensure all algorithms were run (#4108)
20f7dca 
Heh, there is probably a "better" way to do this, but this way was fast and easy enough to do, and I hope is "good enough".

Authors:
  - Erik Welch (https://github.com/eriknw)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Ray Douglass (https://github.com/raydouglass)

URL: #4108
@jnke2016
Optimize the drop-duplicate functionality (#4095)
3d52f17 
Our current python API leverages dask to implement the `drop-duplicate` functionality but it carries a lot of overhead as it draws a significant amount of host memory and results into a crash when processing large graphs (4+ billion edges).

This PR
1. Leverages the CAPI to internally drop multi edges when creating the PLC graph.
2. Deprecates the parameter `multi` which, when set to False, triggers the dask based `drop-duplicate` functionality
3. Add flag `do_expensive_check` to check for `NULL` values in the edgelist

Authors:
  - Joseph Nke (https://github.com/jnke2016)

Approvers:
  - Vibhu Jawa (https://github.com/VibhuJawa)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4095
@rlratzel
Removes the networkx_algorithm decorator to all SCC functions to di…
acb3add 
…sable dispatching to them (#4120)

The current cugraph `strongly_connected_components` is a legacy implementation with known issues, and in most cases should not be used until the cugraph team can provide an update.  

This PR removes the `networkx_algorithm` decorator from all SCC functions to disable dispatching.  Users can still run the SCC functions here by accessing them directly from `nx_cugraph`:

```python
>>> import nx_cugraph as nxcg
>>> nxcg.strongly_connected_components(...)
```

Tested by running the `nx_cugraph` tests (`pytest nx_cugraph/tests`) and the NetworkX tests (`run_nx_tests.sh`)

_Note: using the "non-breaking" label since this API was only present in nightlies and never released._

Authors:
  - Rick Ratzel (https://github.com/rlratzel)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Erik Welch (https://github.com/eriknw)

URL: #4120
@acostadon
corrected links in C API and added groups for support functions (#4131)
581d356 
Fixes broken links in C API docs in Traversal, Sampling and Community algorithm sections

resolves issue #4116

Authors:
  - Don Acosta (https://github.com/acostadon)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)

URL: #4131
@rlratzel
Updates nx-cugraph README.md with latest algos (#4135)
f64df60 
closes #4079 

* Updates nx-cugraph `README.md` with latest algos
* Adds script to auto-update `README.md` from `nx_cugraph` metadata directly

Authors:
  - Rick Ratzel (https://github.com/rlratzel)
  - Erik Welch (https://github.com/eriknw)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Don Acosta (https://github.com/acostadon)
  - Erik Welch (https://github.com/eriknw)

URL: #4135
@tingyu66
Constraint pytorch-dependent wheel test to only run on amd64 (#4133)
51a72fb 
Fixes rapidsai/graph_dl#446
fixes rapidsai/graph_dl#447

Cannot test these wheels in CI due to lack of ARM binary builds for pytorch.

Authors:
  - Tingyu Wang (https://github.com/tingyu66)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #4133
@raydouglass raydouglass requested review from a team as code owners February 2, 2024 21:39
@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Feb 2, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@raydouglass raydouglass merged commit 1033ff9 into main Feb 13, 2024
140 of 141 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
benchmarks ci CMake conda cuGraph doc Documentation python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.