gather_scatter Module Reference

Gather-scatter.

Data Types
type	gs_t
	Gather-scatter kernel. More...

Functions/Subroutines
subroutine	gs_init (gs, dofmap, bcknd, comm_bcknd)
	The runtime autotuning of the comm. backend, implemented in the gs_tune submodule: everything the selection needs (the candidate list, the NEKO_GS_TUNE parsing, switching a live gs over to another backend) is private to it, and only what gs_init drives is declared here.
subroutine, public	gs_comm_alloc (comm, comm_bcknd)
	Allocate a gather-scatter comm. backend of type comm_bcknd.
character(len=12) function, public	gs_comm_name (comm_bcknd)
	Name of the gather-scatter comm. backend comm_bcknd, right-adjusted for the log.
subroutine	gs_free (gs)
	Deallocate a gather-scatter kernel.
subroutine	gs_init_mapping (gs)
	Setup mapping of dofs to gather-scatter operations.
subroutine	gs_schedule (gs)
	Schedule shared gather-scatter operations.
subroutine	gs_sort_i8 (a, ind, n)
	Heap sort for 64-bit integer arrays, returning the permutation ind. Local helper for gs_schedule (the generic math sort has no i8 variant).
subroutine	gs_op_fld (gs, u, op, event)
	Gather-scatter operation on a field u with op op.
subroutine	gs_op_r4 (gs, u, n, op, event)
	Gather-scatter operation on a rank 4 array.
subroutine	gs_op_vector (gs, u, n, op, event)
	Gather-scatter operation on a vector u with op op.
subroutine	gs_op_r3 (gs, u1, u2, u3, n, op, event)
	Gather-scatter operation on a 3-component vector of rank-4 arrays (u1, u2, u3) with op op; see gs_op_vector3.
subroutine	gs_op_vector3 (gs, u1, u2, u3, n, op, event)
	Gather-scatter operation on a 3-component vector (u1, u2, u3) with op op. When the comm backend supports a fused vector halo exchange, the three components are communicated in a single round; otherwise this falls back to three independent scalar gs_op_vector calls (identical result, 3 rounds). The on-node gather/scatter is always done per component (scalar), so only the communication cost is reduced. On device backends the caller's event is recorded once, after the last component's scatter (or superseded by hard synchronisation on host-mirrored comms), so a single event sync covers all components.
subroutine	gs_op_r3_device (gs, u1, u2, u3, n, op, nc, lo, so, m, l, tid, scatter_event)
	Device path for the fused 3-component gs. The device backend's shared gather/scatter always operate on its internal shared buffer, so each component is staged between that buffer and its column of the compact vector buffer gsshared_gs_v: through the host mirror when the comm backend is a host backend (shared_on_host, e.g. OpenCL/Metal or a device build with host MPI), or with device-to-device copies when the comm is device-resident (device MPI/NCCL/NVSHMEM). Apart from the column staging the two modes are identical; the comm backends transparently pick the host array or its device pointer, as in gs_op_vector.

Variables
integer, parameter	gs_tune_ntrials = 100
	Number of timed (and untimed, warm-up) gather-scatter operations per candidate in the runtime autotuning of comm. backends and device synchronisation strategies.
integer, parameter	gs_tune_nwarmup = 2

Function/Subroutine Documentation

gs_comm_alloc()

subroutine, public gather_scatter::gs_comm_alloc	(	class(gs_comm_t), intent(out), allocatable	comm,
		integer, intent(in)	comm_bcknd
	)

Parameters

comm	the allocated backend. Must not hold a live backend on entry: intent(out) releases it without running its free, which would leak whatever resources it holds (symmetric memory, coarrays, uTofu VCQs). Both callers free the previous backend themselves, see gs_free and gs_comm_switch (in the gs_tune submodule).
comm_bcknd	comm. backend to allocate, one of the GS_COMM_* constants

Definition at line 436 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_comm_name()

character(len=12) function, public gather_scatter::gs_comm_name

(

integer, intent(in)

comm_bcknd

)

Parameters

comm_bcknd

comm. backend to name, one of the GS_COMM_* constants

Definition at line 466 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_free()

subroutine gather_scatter::gs_free ( class(gs_t), intent(inout)  gs )

private

Definition at line 495 of file gather_scatter.f90.

gs_init()

subroutine gather_scatter::gs_init ( class(gs_t), intent(inout)  gs, type(dofmap_t), intent(inout), target  dofmap, integer, optional  bcknd, integer, optional  comm_bcknd  )

private

Time ntrials gather-scatter operations on u, returning the average wall time (s) per operation on this rank Benchmark the candidate comm. backends and leave gs holding the fastest one Initialize a gather-scatter kernel

Parameters

dofmap,global	numbering of points and connectivity to base gs on
bcknd,backend	for executing the gs_ops
comm_bcknd,backend	for excuting the communication with

Definition at line 174 of file gather_scatter.f90.

gs_init_mapping()

subroutine gather_scatter::gs_init_mapping ( type(gs_t), intent(inout), target  gs )

private

Note

this might be a bit overkill, but having many collisions makes the init take too long. This is really critical to performance of the init

Definition at line 600 of file gather_scatter.f90.

Here is the call graph for this function:

gs_op_fld()

subroutine gather_scatter::gs_op_fld ( class(gs_t), intent(inout)  gs, type(field_t), intent(inout)  u, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1668 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_op_r3()

subroutine gather_scatter::gs_op_r3 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u1, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u2, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u3, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1773 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_op_r3_device()

subroutine gather_scatter::gs_op_r3_device ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u1, real(kind=rp), dimension(n), intent(inout)  u2, real(kind=rp), dimension(n), intent(inout)  u3, integer, intent(in)  n, integer, intent(in)  op, integer, intent(in)  nc, integer, intent(in)  lo, integer, intent(in)  so, integer, intent(in)  m, integer, intent(in)  l, integer, intent(in)  tid, type(c_ptr), intent(inout)  scatter_event  )

private

Definition at line 1912 of file gather_scatter.f90.

Here is the call graph for this function:

gs_op_r4()

subroutine gather_scatter::gs_op_r4 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1684 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_op_vector()

subroutine gather_scatter::gs_op_vector ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1700 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_vector3()

subroutine gather_scatter::gs_op_vector3 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u1, real(kind=rp), dimension(n), intent(inout)  u2, real(kind=rp), dimension(n), intent(inout)  u3, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1797 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_schedule()

subroutine gather_scatter::gs_schedule ( type(gs_t), intent(inout), target  gs )

private

Discovers, for every shared dof, the set of ranks that also hold it, using a canonical-owner rendezvous routed through the crystal router instead of the previous O(P) shifted neighbour exchange. Each dof global id is hashed to an owner rank, mod(gid, P); the owner gathers all holders and reflects, to each holder, the other holders. Each holder then registers the dof for both send and receive with every peer. The per-peer dof lists are ordered by dof global id, which gives the same ordering on both ranks of a pair (the matching invariant the gather-scatter exchange relies on) without ever transmitting a single sequenced key list. Scales to >1e5 ranks without O(P) buffers or unexpected-message buffer exhaustion.

Definition at line 1475 of file gather_scatter.f90.

Here is the call graph for this function:

gs_sort_i8()

subroutine gather_scatter::gs_sort_i8 ( integer(i8), dimension(n), intent(inout)  a, integer, dimension(n), intent(out)  ind, integer, intent(in)  n  )

private

Definition at line 1615 of file gather_scatter.f90.

Variable Documentation

gs_tune_ntrials

integer, parameter gather_scatter::gs_tune_ntrials = 100

Definition at line 141 of file gather_scatter.f90.

gs_tune_nwarmup

integer, parameter gather_scatter::gs_tune_nwarmup = 2

private

Definition at line 142 of file gather_scatter.f90.