Distributed regridding v2 - source data on distributed space #1175
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the second PR of ClimaCoupler SDI #188, the first being #1107.
Major goals to accomplish in this PR
Specific changes to be implemented in this PR
source_global_elem_lidx
field toLinearMap
. Use this to perform matrix multiplication for local indices of source data only inremap!
. Since at this point all processes have all the source data, this approach will include redundant multiplication. However, this setup will be useful when we use the super-halo exchange to send only the necessary source data.generate_map
function, one for the serial case and one for the distributed case.generate_map
, use only the distributed source and target spaces (i.e. no serial spaces).Spaces.unique_nodes
to correctly return the number of unique nodes in a distributed space.The most involved component of this PR will be extending
Spaces.unique_nodes
to handle a distributed space as input. This will require developing an algorithm to count the unique nodes and then implementing it in the code. This may be best done as a separate PR to minimize scope of each PR.The other non-trivial component of this PR is sending the source data to all processes. We should be able to use the existing DSS code, but we may need to add some data structures and functions to our code to use it. We should develop this part with the super-halo implementation in mind, so that any relevant infrastructure can easily be extended for that case.
QA