Gather-scatter communication using device SHMEM. The arrays are indexed per PE like send_pe and @ recv_pe. More...

Inheritance diagram for gs_device_shmem::gs_device_shmem_t:

Collaboration diagram for gs_device_shmem::gs_device_shmem_t:

[legend]

Public Member Functions
procedure, pass(this)	init (this, send_pe, recv_pe)
	Initialise MPI based communication method.

procedure, pass(this)	free (this)
	Deallocate MPI based communication method.

procedure, pass(this)	nbsend (this, u, n, tag, deps, strm)
	Post non-blocking send operations.

procedure, pass(this)	nbrecv (this, tag)
	Post non-blocking receive operations.

procedure, pass(this)	nbwait (this, u, n, op, strm)
	Wait for non-blocking operations.

procedure, pass(this)	nbsend_vec (this, u, n, nc, tag, deps, strm)
	Fused nc-component send. u is the compact shared device buffer (component-outer, per-component stride n = nshared). Bulk-packs the parity slab of the fused vector send buffer; see gs_device_shmem_nbsend for the ordering and parity-slab rationale.

procedure, pass(this)	nbrecv_vec (this, tag, nc)
	No-op: everything happens in nbwait_vec.

procedure, pass(this)	nbwait_vec (this, u, n, nc, op, strm)
	Fused nc-component push + unpack (the pack happens in nbsend_vec).

procedure(gs_comm_init), deferred, pass	init gs_comm_init

procedure(gs_comm_free), deferred, pass	free gs_comm_free

procedure(gs_nbsend), deferred, pass	nbsend gs_nbsend

procedure(gs_nbrecv), deferred, pass	nbrecv gs_nbrecv

procedure(gs_nbwait), deferred, pass	nbwait gs_nbwait

procedure, pass(this)	init_dofs (this)

procedure, pass(this)	free_dofs (this)

procedure, pass(this)	init_order (this, send_pe, recv_pe)
	Obtains which ranks to send and receive data from.

procedure, pass(this)	free_order (this)

procedure, pass(this)	take_schedule (this, src)
	Take over the gather-scatter schedule (dof lists and peer order) of src, avoiding a second (expensive) pass over the connectivity. The data is moved rather than copied, so src is left without a schedule and must not be used for communication afterwards (it can still be freed). No communication resources are set up here; complete the handover with init_schedule once src has been freed, so that the two backends never hold their resources at the same time.

procedure, pass(this)	init_schedule (this)
	Set up this communication method for the schedule taken over by take_schedule. Collective, as init is.

procedure, pass(this)	init_vec (this)
	Fused vector halo exchange. Default implementations abort; backends that set vec_supported = .true. override them.

procedure, pass(this)	nbsend_vec (this, u, n, nc, tag, deps, strm)
	Default fused vector send. Abort unless a backend overrides it.

procedure, pass(this)	nbrecv_vec (this, tag, nc)
	Default fused vector receive. Abort unless a backend overrides it.

procedure, pass(this)	nbwait_vec (this, u, n, nc, op, strm)
	Default fused vector wait/reduce. Abort unless a backend overrides it.

Public Attributes
type(gs_device_shmem_buf_t)	send_buf

type(gs_device_shmem_buf_t)	recv_buf

type(c_ptr), dimension(:), allocatable	stream

type(c_ptr), dimension(:), allocatable	event

integer	iter = 0
	Round counter for the rank-indexed signal protocol. Advances once per gs op (lockstep across ranks, SPMD); all waits use CMP_GE, so no cross-rank counter matching is required and peer counts may differ between ranks.

type(c_ptr)	done_sig_d = C_NULL_PTR
	Symmetric rank-indexed signal arrays, pe_size slots each (a single collective allocation per array, so nvshmem_malloc collectivity is independent of the local peer count). done_sig(r) on our PE is set to iter by rank r's put_signal when its slab has landed in our recv buffer; ready_sig(r) on our PE is set to iter by rank r once it has consumed our round-iter slab (so we may put into its recv slab again).

type(c_ptr)	ready_sig_d = C_NULL_PTR

type(c_ptr)	pack_event = C_NULL_PTR
	Records the bulk pack on the main stream in nbsend; every per-peer stream waits on it, ordering all reads of the shared buffer u before any unpack write to u.

type(stack_i4_t), dimension(:), allocatable	send_dof
	A list of stacks of dof indices local to this process to send to rank_i.

type(stack_i4_t), dimension(:), allocatable	recv_dof
	recv_dof(rank_i) is a stack of dof indices local to this process to receive from rank_i. size(recv_dof) == pe_size

integer, dimension(:), allocatable	send_pe
	Array of ranks that this process should send to.

integer, dimension(:), allocatable	recv_pe
	array of ranks that this process will receive messages from

logical	vec_supported = .false.
	Whether this backend implements the fused vector (multi-component) halo exchange (nbsend_vec/nbrecv_vec/nbwait_vec). When .false., the gs_op_r3 caller falls back to nc independent scalar exchanges.

logical	vec_ready = .false.
	Whether the buffers the fused vector exchange needs are in place. They are sized GS_VEC_NC times the halo, quadrupling what the backend holds, and only gs_op_r3 ever touches them, so a backend that can allocate them on its own defers that to the first fused exchange (see init_vec) rather than paying for it in every run. A backend whose vector buffers are part of an allocation the whole run has to agree on – symmetric memory, coarrays, registered memory, an RMA window – cannot defer, since a rank with no shared dofs never reaches the first fused exchange; those allocate in init and set this there.

Detailed Description

Definition at line 80 of file gs_device_shmem.F90.

Member Function/Subroutine Documentation

◆ free() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::free ( class(gs_device_shmem_t), intent(inout) this )

Definition at line 104 of file gs_device_shmem.F90.

◆ free() [2/2]

procedure(gs_comm_free), deferred, pass gs_comm::gs_comm_t::free

pure virtualinherited

Definition at line 82 of file gs_comm.f90.

◆ free_dofs()

procedure, pass(this) gs_comm::gs_comm_t::free_dofs ( class(gs_comm_t), intent(inout) this )

inherited

Definition at line 87 of file gs_comm.f90.

◆ free_order()

procedure, pass(this) gs_comm::gs_comm_t::free_order ( class(gs_comm_t), intent(inout) this )

inherited

Definition at line 89 of file gs_comm.f90.

◆ init() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::init	(	class(gs_device_shmem_t), intent(inout)	this,
		type(stack_i4_t), intent(inout)	send_pe,
		type(stack_i4_t), intent(inout)	recv_pe
	)

Definition at line 103 of file gs_device_shmem.F90.

◆ init() [2/2]

procedure(gs_comm_init), deferred, pass gs_comm::gs_comm_t::init

pure virtualinherited

Definition at line 81 of file gs_comm.f90.

◆ init_dofs()

procedure, pass(this) gs_comm::gs_comm_t::init_dofs ( class(gs_comm_t), intent(inout) this )

inherited

Definition at line 86 of file gs_comm.f90.

◆ init_order()

procedure, pass(this) gs_comm::gs_comm_t::init_order	(	class(gs_comm_t), intent(inout)	this,
		type(stack_i4_t), intent(inout)	send_pe,
		type(stack_i4_t), intent(inout)	recv_pe
	)

inherited

Parameters

send_pe,only	contains rank ids this porcesss should send to
recv_pe,only	the ranks this process should receive from

Definition at line 88 of file gs_comm.f90.

◆ init_schedule()

procedure, pass(this) gs_comm::gs_comm_t::init_schedule ( class(gs_comm_t), intent(inout) this )

inherited

Definition at line 91 of file gs_comm.f90.

◆ init_vec()

procedure, pass(this) gs_comm::gs_comm_t::init_vec ( class(gs_comm_t), intent(inout) this )

inherited

Note: Rank local by contract: a rank with no shared dofs skips the fused exchange entirely, so an override must not communicate.

Definition at line 94 of file gs_comm.f90.

◆ nbrecv() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::nbrecv	(	class(gs_device_shmem_t), intent(inout)	this,
		integer, intent(in)	tag
	)

Definition at line 106 of file gs_device_shmem.F90.

◆ nbrecv() [2/2]

procedure(gs_nbrecv), deferred, pass gs_comm::gs_comm_t::nbrecv

pure virtualinherited

Definition at line 84 of file gs_comm.f90.

◆ nbrecv_vec() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::nbrecv_vec	(	class(gs_device_shmem_t), intent(inout)	this,
		integer, intent(in)	tag,
		integer, intent(in)	nc
	)

Definition at line 109 of file gs_device_shmem.F90.

◆ nbrecv_vec() [2/2]

procedure, pass(this) gs_comm::gs_comm_t::nbrecv_vec	(	class(gs_comm_t), intent(inout)	this,
		integer, intent(in)	tag,
		integer, intent(in)	nc
	)

inherited

Definition at line 96 of file gs_comm.f90.

◆ nbsend() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::nbsend	(	class(gs_device_shmem_t), intent(inout)	this,
		real(kind=rp), dimension(n), intent(inout)	u,
		integer, intent(in)	n,
		integer, intent(in)	tag,
		type(c_ptr), intent(inout)	deps,
		type(c_ptr), intent(inout)	strm
	)

Definition at line 105 of file gs_device_shmem.F90.

◆ nbsend() [2/2]

procedure(gs_nbsend), deferred, pass gs_comm::gs_comm_t::nbsend

pure virtualinherited

Definition at line 83 of file gs_comm.f90.

◆ nbsend_vec() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::nbsend_vec	(	class(gs_device_shmem_t), intent(inout)	this,
		real(kind=rp), dimension(nc*n), intent(inout)	u,
		integer, intent(in)	n,
		integer, intent(in)	nc,
		integer, intent(in)	tag,
		type(c_ptr), intent(inout)	deps,
		type(c_ptr), intent(inout)	strm
	)

Definition at line 108 of file gs_device_shmem.F90.

◆ nbsend_vec() [2/2]

procedure, pass(this) gs_comm::gs_comm_t::nbsend_vec	(	class(gs_comm_t), intent(inout)	this,
		real(kind=rp), dimension(nc*n), intent(inout)	u,
		integer, intent(in)	n,
		integer, intent(in)	nc,
		integer, intent(in)	tag,
		type(c_ptr), intent(inout)	deps,
		type(c_ptr), intent(inout)	strm
	)

inherited

Parameters

u	compact shared buffer, component-outer: u((c-1)*n + idx)
n	number of shared dofs (per component)
nc	number of components

Definition at line 95 of file gs_comm.f90.

◆ nbwait() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::nbwait	(	class(gs_device_shmem_t), intent(inout)	this,
		real(kind=rp), dimension(n), intent(inout)	u,
		integer, intent(in)	n,
		integer	op,
		type(c_ptr), intent(inout)	strm
	)

Definition at line 107 of file gs_device_shmem.F90.

◆ nbwait() [2/2]

procedure(gs_nbwait), deferred, pass gs_comm::gs_comm_t::nbwait

pure virtualinherited

Definition at line 85 of file gs_comm.f90.

◆ nbwait_vec() [1/2]

procedure, pass(this) gs_device_shmem::gs_device_shmem_t::nbwait_vec	(	class(gs_device_shmem_t), intent(inout)	this,
		real(kind=rp), dimension(nc*n), intent(inout)	u,
		integer, intent(in)	n,
		integer, intent(in)	nc,
		integer	op,
		type(c_ptr), intent(inout)	strm
	)

Definition at line 110 of file gs_device_shmem.F90.

◆ nbwait_vec() [2/2]

procedure, pass(this) gs_comm::gs_comm_t::nbwait_vec	(	class(gs_comm_t), intent(inout)	this,
		real(kind=rp), dimension(nc*n), intent(inout)	u,
		integer, intent(in)	n,
		integer, intent(in)	nc,
		integer	op,
		type(c_ptr), intent(inout)	strm
	)

inherited

Definition at line 97 of file gs_comm.f90.

◆ take_schedule()

procedure, pass(this) gs_comm::gs_comm_t::take_schedule	(	class(gs_comm_t), intent(inout)	this,
		class(gs_comm_t), intent(inout)	src
	)

inherited

Parameters

src	the backend to take the schedule from, left without one

Definition at line 90 of file gs_comm.f90.

Member Data Documentation

◆ done_sig_d

type(c_ptr) gs_device_shmem::gs_device_shmem_t::done_sig_d = C_NULL_PTR

Definition at line 96 of file gs_device_shmem.F90.

◆ event

type(c_ptr), dimension(:), allocatable gs_device_shmem::gs_device_shmem_t::event

Definition at line 84 of file gs_device_shmem.F90.

◆ iter

integer gs_device_shmem::gs_device_shmem_t::iter = 0

Definition at line 89 of file gs_device_shmem.F90.

◆ pack_event

type(c_ptr) gs_device_shmem::gs_device_shmem_t::pack_event = C_NULL_PTR

Definition at line 101 of file gs_device_shmem.F90.

◆ ready_sig_d

type(c_ptr) gs_device_shmem::gs_device_shmem_t::ready_sig_d = C_NULL_PTR

Definition at line 97 of file gs_device_shmem.F90.

◆ recv_buf

type(gs_device_shmem_buf_t) gs_device_shmem::gs_device_shmem_t::recv_buf

Definition at line 82 of file gs_device_shmem.F90.

◆ recv_dof

type(stack_i4_t), dimension(:), allocatable gs_comm::gs_comm_t::recv_dof

inherited

Definition at line 58 of file gs_comm.f90.

◆ recv_pe

integer, dimension(:), allocatable gs_comm::gs_comm_t::recv_pe

inherited

Definition at line 64 of file gs_comm.f90.

◆ send_buf

type(gs_device_shmem_buf_t) gs_device_shmem::gs_device_shmem_t::send_buf

Definition at line 81 of file gs_device_shmem.F90.

◆ send_dof

type(stack_i4_t), dimension(:), allocatable gs_comm::gs_comm_t::send_dof

inherited

Definition at line 55 of file gs_comm.f90.

◆ send_pe

integer, dimension(:), allocatable gs_comm::gs_comm_t::send_pe

inherited

Note: : this will usually be fewer than the total number of ranks size(send_pe) <= pe_size

Definition at line 62 of file gs_comm.f90.

◆ stream

type(c_ptr), dimension(:), allocatable gs_device_shmem::gs_device_shmem_t::stream

Definition at line 83 of file gs_device_shmem.F90.

◆ vec_ready

logical gs_comm::gs_comm_t::vec_ready = .false.

inherited

Definition at line 79 of file gs_comm.f90.

◆ vec_supported

logical gs_comm::gs_comm_t::vec_supported = .false.

inherited

Definition at line 68 of file gs_comm.f90.

The documentation for this type was generated from the following file:

/home/runner/work/neko/neko/src/gs/bcknd/device/gs_device_shmem.F90

Public Member Functions

Public Attributes

Detailed Description

Member Function/Subroutine Documentation

◆ free() [1/2]

◆ free() [2/2]

◆ free_dofs()

◆ free_order()

◆ init() [1/2]

◆ init() [2/2]

◆ init_dofs()

◆ init_order()

◆ init_schedule()

◆ init_vec()

◆ nbrecv() [1/2]

◆ nbrecv() [2/2]

◆ nbrecv_vec() [1/2]

◆ nbrecv_vec() [2/2]

◆ nbsend() [1/2]

◆ nbsend() [2/2]

◆ nbsend_vec() [1/2]

◆ nbsend_vec() [2/2]

◆ nbwait() [1/2]

◆ nbwait() [2/2]

◆ nbwait_vec() [1/2]

◆ nbwait_vec() [2/2]

◆ take_schedule()

Member Data Documentation

◆ done_sig_d

◆ event

◆ iter

◆ pack_event

◆ ready_sig_d

◆ recv_buf

◆ recv_dof

◆ recv_pe

◆ send_buf

◆ send_dof

◆ send_pe

◆ stream

◆ vec_ready

◆ vec_supported