gather_scatter Module Reference

Gather-scatter.

Data Types
type	gs_t

Functions/Subroutines
subroutine	gs_init (gs, dofmap, bcknd, comm_bcknd)
	Initialize a gather-scatter kernel.
subroutine	gs_free (gs)
	Deallocate a gather-scatter kernel.
subroutine	gs_init_mapping (gs)
	Setup mapping of dofs to gather-scatter operations.
subroutine	gs_schedule (gs)
	Schedule shared gather-scatter operations.
subroutine	gs_sort_i8 (a, ind, n)
	Heap sort for 64-bit integer arrays, returning the permutation ind. Local helper for gs_schedule (the generic math sort has no i8 variant).
subroutine	gs_op_fld (gs, u, op, event)
	Gather-scatter operation on a field u with op op.
subroutine	gs_op_r4 (gs, u, n, op, event)
	Gather-scatter operation on a rank 4 array.
subroutine	gs_op_vector (gs, u, n, op, event)
	Gather-scatter operation on a vector u with op op.
subroutine	gs_op_r3 (gs, u1, u2, u3, n, op, event)
	Gather-scatter operation on a 3-component vector of rank-4 arrays (u1, u2, u3) with op op; see gs_op_vector3.
subroutine	gs_op_vector3 (gs, u1, u2, u3, n, op, event)
	Gather-scatter operation on a 3-component vector (u1, u2, u3) with op op. When the comm backend supports a fused vector halo exchange, the three components are communicated in a single round; otherwise this falls back to three independent scalar gs_op_vector calls (identical result, 3 rounds). The on-node gather/scatter is always done per component (scalar), so only the communication cost is reduced. On device backends the caller's event is recorded once, after the last component's scatter (or superseded by hard synchronisation on host-mirrored comms), so a single event sync covers all components.
subroutine	gs_op_r3_device (gs, u1, u2, u3, n, op, nc, lo, so, m, l, tid, scatter_event)
	Device path for the fused 3-component gs. The device backend's shared gather/scatter always operate on its internal shared buffer, so each component is staged between that buffer and its column of the compact vector buffer gsshared_gs_v: through the host mirror when the comm backend is a host backend (shared_on_host, e.g. OpenCL/Metal or a device build with host MPI), or with device-to-device copies when the comm is device-resident (device MPI/NCCL/NVSHMEM). Apart from the column staging the two modes are identical; the comm backends transparently pick the host array or its device pointer, as in gs_op_vector.

Function/Subroutine Documentation

gs_free()

subroutine gather_scatter::gs_free ( class(gs_t), intent(inout)  gs )

private

Definition at line 402 of file gather_scatter.f90.

gs_init()

subroutine gather_scatter::gs_init	(	class(gs_t), intent(inout)	gs,
		type(dofmap_t), intent(inout), target	dofmap,
		integer, optional	bcknd,
		integer, optional	comm_bcknd
	)

Parameters

dofmap,global	numbering of points and connectivity to base gs on
bcknd,backend	for executing the gs_ops
comm_bcknd,backend	for excuting the communication with

Definition at line 130 of file gather_scatter.f90.

Here is the call graph for this function:

gs_init_mapping()

subroutine gather_scatter::gs_init_mapping ( type(gs_t), intent(inout), target  gs )

private

Note

this might be a bit overkill, but having many collisions makes the init take too long. This is really critical to performance of the init

Definition at line 474 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_fld()

subroutine gather_scatter::gs_op_fld ( class(gs_t), intent(inout)  gs, type(field_t), intent(inout)  u, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1542 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_r3()

subroutine gather_scatter::gs_op_r3 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u1, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u2, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u3, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1647 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_r3_device()

subroutine gather_scatter::gs_op_r3_device ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u1, real(kind=rp), dimension(n), intent(inout)  u2, real(kind=rp), dimension(n), intent(inout)  u3, integer, intent(in)  n, integer, intent(in)  op, integer, intent(in)  nc, integer, intent(in)  lo, integer, intent(in)  so, integer, intent(in)  m, integer, intent(in)  l, integer, intent(in)  tid, type(c_ptr), intent(inout)  scatter_event  )

private

Definition at line 1786 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_r4()

subroutine gather_scatter::gs_op_r4 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1558 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_vector()

subroutine gather_scatter::gs_op_vector ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1574 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_vector3()

subroutine gather_scatter::gs_op_vector3 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u1, real(kind=rp), dimension(n), intent(inout)  u2, real(kind=rp), dimension(n), intent(inout)  u3, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1671 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_schedule()

subroutine gather_scatter::gs_schedule ( type(gs_t), intent(inout), target  gs )

private

Discovers, for every shared dof, the set of ranks that also hold it, using a canonical-owner rendezvous routed through the crystal router instead of the previous O(P) shifted neighbour exchange. Each dof global id is hashed to an owner rank, mod(gid, P); the owner gathers all holders and reflects, to each holder, the other holders. Each holder then registers the dof for both send and receive with every peer. The per-peer dof lists are ordered by dof global id, which gives the same ordering on both ranks of a pair (the matching invariant the gather-scatter exchange relies on) without ever transmitting a single sequenced key list. Scales to >1e5 ranks without O(P) buffers or unexpected-message buffer exhaustion.

Definition at line 1349 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_sort_i8()

subroutine gather_scatter::gs_sort_i8 ( integer(i8), dimension(n), intent(inout)  a, integer, dimension(n), intent(out)  ind, integer, intent(in)  n  )

private

Definition at line 1489 of file gather_scatter.f90.

Here is the caller graph for this function: