gather_scatter Module Reference

Gather-scatter.

Data Types
type	gs_t
	Gather-scatter kernel. More...

Functions/Subroutines
subroutine	gs_init (gs, dofmap, bcknd, comm_bcknd)
	The runtime autotuning of the comm. backend, implemented in the gs_tune submodule: everything the selection needs (the candidate list, the NEKO_GS_TUNE parsing, switching a live gs over to another backend) is private to it, and only what gs_init drives is declared here.
subroutine, public	gs_comm_alloc (comm, comm_bcknd)
	Allocate a gather-scatter comm. backend of type comm_bcknd.
character(len=12) function, public	gs_comm_name (comm_bcknd)
	Name of the gather-scatter comm. backend comm_bcknd, right-adjusted for the log.
logical function, public	gs_comm_on_device (comm_bcknd)
	Whether the comm. backend comm_bcknd exchanges the shared dofs straight out of device memory rather than out of the host mirror of the shared buffer, which is what decides where the gather-scatter backend leaves them (gs_bcknd_tshared_on_host). A host backend on a device build is not device-resident: it drives the exchange from the mirror, at the price of a copy in each direction.
subroutine	gs_free (gs)
	Deallocate a gather-scatter kernel.
subroutine	gs_init_mapping (gs)
	Setup mapping of dofs to gather-scatter operations.
subroutine	gs_schedule (gs)
	Schedule shared gather-scatter operations.
subroutine	gs_sort_i8 (a, ind, n)
	Heap sort for 64-bit integer arrays, returning the permutation ind. Local helper for gs_schedule (the generic math sort has no i8 variant).
subroutine	gs_op_fld (gs, u, op, event)
	Gather-scatter operation on a field u with op op.
subroutine	gs_op_r4 (gs, u, n, op, event)
	Gather-scatter operation on a rank 4 array.
subroutine	gs_op_vector (gs, u, n, op, event)
	Gather-scatter operation on a vector u with op op.
subroutine	gs_op_r3 (gs, u1, u2, u3, n, op, event)
	Gather-scatter operation on a 3-component vector of rank-4 arrays (u1, u2, u3) with op op; see gs_op_vector3.
subroutine	gs_op_vector3 (gs, u1, u2, u3, n, op, event)
	Gather-scatter operation on a 3-component vector (u1, u2, u3) with op op. When the comm backend supports a fused vector halo exchange, the three components are communicated in a single round; otherwise this falls back to three independent scalar gs_op_vector calls (identical result, 3 rounds). The on-node gather/scatter is always done per component (scalar), so only the communication cost is reduced. On device backends the caller's event is recorded once, after the last component's scatter (or superseded by hard synchronisation on host-mirrored comms), so a single event sync covers all components.
subroutine	gs_op_r3_device (gs, u1, u2, u3, n, op, nc, lo, so, m, l, tid, scatter_event)
	Device path for the fused 3-component gs. The device backend's shared gather/scatter always operate on its internal shared buffer, so each component is staged between that buffer and its column of the compact vector buffer gsshared_gs_v: through the host mirror when the comm backend is a host backend (shared_on_host, e.g. OpenCL/Metal or a device build with host MPI), or with device-to-device copies when the comm is device-resident (device MPI/NCCL/NVSHMEM). Apart from the column staging the two modes are identical; the comm backends transparently pick the host array or its device pointer, as in gs_op_vector.

Variables
integer, parameter	gs_tune_ntrials = 100
	Number of timed (and untimed, warm-up) gather-scatter operations per candidate in the runtime autotuning of comm. backends and device synchronisation strategies.
integer, parameter	gs_tune_nwarmup = 2

Function/Subroutine Documentation

gs_comm_alloc()

subroutine, public gather_scatter::gs_comm_alloc	(	class(gs_comm_t), intent(out), allocatable	comm,
		integer, intent(in)	comm_bcknd
	)

Parameters

comm	the allocated backend. Must not hold a live backend on entry: intent(out) releases it without running its free, which would leak whatever resources it holds (symmetric memory, coarrays, uTofu VCQs). Both callers free the previous backend themselves, see gs_free and gs_comm_switch (in the gs_tune submodule).
comm_bcknd	comm. backend to allocate, one of the GS_COMM_* constants

Definition at line 412 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_comm_name()

character(len=12) function, public gather_scatter::gs_comm_name

(

integer, intent(in)

comm_bcknd

)

Parameters

comm_bcknd

comm. backend to name, one of the GS_COMM_* constants

Definition at line 448 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_comm_on_device()

logical function, public gather_scatter::gs_comm_on_device

(

integer, intent(in)

comm_bcknd

)

Parameters

comm_bcknd

comm. backend to check, one of the GS_COMM_* constants

Returns

whether the backend is device-resident

Definition at line 490 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_free()

subroutine gather_scatter::gs_free ( class(gs_t), intent(inout)  gs )

private

Definition at line 504 of file gather_scatter.f90.

gs_init()

subroutine gather_scatter::gs_init ( class(gs_t), intent(inout)  gs, type(dofmap_t), intent(inout), target  dofmap, integer, optional  bcknd, integer, optional  comm_bcknd  )

private

Time ntrials gather-scatter operations on u, returning the average wall time (s) per operation on this rank Benchmark the candidate comm. backends and leave gs holding the fastest one Bind the non-blocking synchronisation strategy of the device MPI comm. backend held by gs, benchmarking the strategies unless NEKO_GS_STRTGY names one Initialize a gather-scatter kernel

Parameters

dofmap,global	numbering of points and connectivity to base gs on
bcknd,backend	for executing the gs_ops
comm_bcknd,backend	for excuting the communication with

Definition at line 187 of file gather_scatter.f90.

gs_init_mapping()

subroutine gather_scatter::gs_init_mapping ( type(gs_t), intent(inout), target  gs )

private

Note

this might be a bit overkill, but having many collisions makes the init take too long. This is really critical to performance of the init

Definition at line 609 of file gather_scatter.f90.

Here is the call graph for this function:

gs_op_fld()

subroutine gather_scatter::gs_op_fld ( class(gs_t), intent(inout)  gs, type(field_t), intent(inout)  u, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1677 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_op_r3()

subroutine gather_scatter::gs_op_r3 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u1, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u2, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u3, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1782 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_op_r3_device()

subroutine gather_scatter::gs_op_r3_device ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u1, real(kind=rp), dimension(n), intent(inout)  u2, real(kind=rp), dimension(n), intent(inout)  u3, integer, intent(in)  n, integer, intent(in)  op, integer, intent(in)  nc, integer, intent(in)  lo, integer, intent(in)  so, integer, intent(in)  m, integer, intent(in)  l, integer, intent(in)  tid, type(c_ptr), intent(inout)  scatter_event  )

private

Definition at line 1921 of file gather_scatter.f90.

Here is the call graph for this function:

gs_op_r4()

subroutine gather_scatter::gs_op_r4 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(:,:,:,:), intent(inout), contiguous  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1693 of file gather_scatter.f90.

Here is the caller graph for this function:

gs_op_vector()

subroutine gather_scatter::gs_op_vector ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1709 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_op_vector3()

subroutine gather_scatter::gs_op_vector3 ( class(gs_t), intent(inout)  gs, real(kind=rp), dimension(n), intent(inout)  u1, real(kind=rp), dimension(n), intent(inout)  u2, real(kind=rp), dimension(n), intent(inout)  u3, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout), optional  event  )

private

Definition at line 1806 of file gather_scatter.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_schedule()

subroutine gather_scatter::gs_schedule ( type(gs_t), intent(inout), target  gs )

private

Discovers, for every shared dof, the set of ranks that also hold it, using a canonical-owner rendezvous routed through the crystal router instead of the previous O(P) shifted neighbour exchange. Each dof global id is hashed to an owner rank, mod(gid, P); the owner gathers all holders and reflects, to each holder, the other holders. Each holder then registers the dof for both send and receive with every peer. The per-peer dof lists are ordered by dof global id, which gives the same ordering on both ranks of a pair (the matching invariant the gather-scatter exchange relies on) without ever transmitting a single sequenced key list. Scales to >1e5 ranks without O(P) buffers or unexpected-message buffer exhaustion.

Definition at line 1484 of file gather_scatter.f90.

Here is the call graph for this function:

gs_sort_i8()

subroutine gather_scatter::gs_sort_i8 ( integer(i8), dimension(n), intent(inout)  a, integer, dimension(n), intent(out)  ind, integer, intent(in)  n  )

private

Definition at line 1624 of file gather_scatter.f90.

Variable Documentation

gs_tune_ntrials

integer, parameter gather_scatter::gs_tune_ntrials = 100

Definition at line 146 of file gather_scatter.f90.

gs_tune_nwarmup

integer, parameter gather_scatter::gs_tune_nwarmup = 2

private

Definition at line 147 of file gather_scatter.f90.