gs_device_crystal Module Reference

Defines GPU aware crystal router gather-scatter communication. More...

Data Types
interface	device_mpi_free_reqs
interface	device_mpi_init_reqs
interface	device_mpi_irecv
interface	device_mpi_isend
interface	device_mpi_waitall
type	gs_device_crystal_t
	Gather-scatter communication using a crystal router on the device. More...
interface	hip_gs_pack
interface	hip_gs_pack_vec
interface	hip_gs_unpack
interface	hip_gs_unpack_vec

Functions/Subroutines
subroutine	gs_device_crystal_init (this, send_pe, recv_pe)
	Initialise crystal router based device communication See gs_comm.f90 for details.
subroutine	gs_device_crystal_init_vec (this)
	Allocate the fused vector working and send buffers in device memory, sized for GS_VEC_NC components. Deferred to the first fused exchange, see gs_comm_t. The nc-scaled stage index lists are uploaded separately, on the first exchange of a given nc (see vec_nc), and the routing plan is shared with the scalar exchange, so this stays rank local.
subroutine	gs_device_crystal_free (this)
	Deallocate crystal router based device communication.
subroutine	gs_device_crystal_nbrecv (this, tag)
	Post the receives of the first routing stage.
subroutine	gs_device_crystal_nbsend (this, u, n, tag, deps, strm)
	Pack the shared vector and post the send of the first routing stage.
subroutine	gs_device_crystal_nbwait (this, u, n, op, strm)
	Drive the remaining routing stages and reduce what is delivered into the shared vector.
subroutine	gs_device_crystal_nbrecv_vec (this, tag, nc)
	Post the receives of the first routing stage, fused nc-component.
subroutine	gs_device_crystal_nbsend_vec (this, u, n, nc, tag, deps, strm)
	Pack the shared vector and post the send of the first routing stage, fused nc-component.
subroutine	gs_device_crystal_nbwait_vec (this, u, n, nc, op, strm)
	Drive the remaining routing stages and reduce what is delivered into the shared vector, fused nc-component.
subroutine	cr_gather (src_d, dst_d, idx_d, n, strm)
	Gather n words of src into dst through the 1-based index list idx_d, i.e. dst(j) = src(idx(j))
subroutine	cr_gather_vec (src_d, dst_d, idx_d, n, nc, ns, strm)
	Gather n positions of the component-outer src (stride ns) into the interleaved dst.
subroutine	cr_scatter (u_d, op, buf_d, idx_d, n, strm)
	Reduce n delivered words into the shared vector under op.
subroutine	cr_scatter_vec (u_d, op, buf_d, idx_d, n, nc, ns, strm)
	Reduce n delivered positions into the component-outer shared vector.
subroutine	cr_vec_index (this, nc)
	Build the per-stage index lists of the fused vector exchange, which address the same words as the scalar ones with the components spelled out: position p of a stage becomes nc*(p-1) + 1 .. nc*p.
subroutine	cr_expand (idx, n, nc, out)
	Spell out the components of the 1-based index list idx over nc.
subroutine	cr_upload (ptr, idx, n)
	Copy the first n entries of an index list to the device, leaving ptr null when there are none.
subroutine	cr_free_ptrs (ptrs)
	Release a list of device pointers, leaving them null.
subroutine	cr_mark_dupes (dof, out, n)
	Copy dof into out, negating every index that appears more than once so that the unpack kernel reduces those atomically.

Detailed Description

The device counterpart of gs_crystal: the same routing plan, with the halo never leaving the device. Every word movement the routing asks for is an indexed gather, which is what the existing gs pack kernel already does (buf[j] = u[dof[j]-1]), so forwarding a stage costs one kernel launch and no new device code – the working buffer simply takes the place of the shared vector as the kernel's source.

The two working buffer columns are separate allocations, so each stage addresses its source and destination from offset zero and the received words land straight behind the ones that stay. As on the host, only the first stage overlaps the local gather-scatter.

Function/Subroutine Documentation

cr_expand()

subroutine gs_device_crystal::cr_expand ( integer, dimension(:), intent(in)  idx, integer, intent(in)  n, integer, intent(in)  nc, integer, dimension(:), intent(out), allocatable  out  )

private

Definition at line 719 of file gs_device_crystal.F90.

Here is the caller graph for this function:

cr_free_ptrs()

subroutine gs_device_crystal::cr_free_ptrs ( type(c_ptr), dimension(:), intent(inout), allocatable  ptrs )

private

Definition at line 755 of file gs_device_crystal.F90.

Here is the caller graph for this function:

cr_gather()

subroutine gs_device_crystal::cr_gather ( type(c_ptr), intent(in)  src_d, type(c_ptr), intent(in)  dst_d, type(c_ptr), intent(in)  idx_d, integer, intent(in)  n, type(c_ptr), intent(inout)  strm  )

private

Definition at line 611 of file gs_device_crystal.F90.

Here is the call graph for this function:

Here is the caller graph for this function:

cr_gather_vec()

subroutine gs_device_crystal::cr_gather_vec ( type(c_ptr), intent(in)  src_d, type(c_ptr), intent(in)  dst_d, type(c_ptr), intent(in)  idx_d, integer, intent(in)  n, integer, intent(in)  nc, integer, intent(in)  ns, type(c_ptr), intent(inout)  strm  )

private

Definition at line 630 of file gs_device_crystal.F90.

Here is the call graph for this function:

Here is the caller graph for this function:

cr_mark_dupes()

subroutine gs_device_crystal::cr_mark_dupes ( integer, dimension(:), intent(in)  dof, integer, dimension(:), intent(out)  out, integer, intent(in)  n  )

private

Definition at line 770 of file gs_device_crystal.F90.

Here is the caller graph for this function:

cr_scatter()

subroutine gs_device_crystal::cr_scatter ( type(c_ptr), intent(in)  u_d, integer, intent(in)  op, type(c_ptr), intent(in)  buf_d, type(c_ptr), intent(in)  idx_d, integer, intent(in)  n, type(c_ptr), intent(inout)  strm  )

private

Definition at line 648 of file gs_device_crystal.F90.

Here is the call graph for this function:

Here is the caller graph for this function:

cr_scatter_vec()

subroutine gs_device_crystal::cr_scatter_vec ( type(c_ptr), intent(in)  u_d, integer, intent(in)  op, type(c_ptr), intent(in)  buf_d, type(c_ptr), intent(in)  idx_d, integer, intent(in)  n, integer, intent(in)  nc, integer, intent(in)  ns, type(c_ptr), intent(inout)  strm  )

private

Definition at line 666 of file gs_device_crystal.F90.

Here is the call graph for this function:

Here is the caller graph for this function:

cr_upload()

subroutine gs_device_crystal::cr_upload ( type(c_ptr), intent(inout)  ptr, integer, dimension(:), intent(inout)  idx, integer, intent(in)  n  )

private

Definition at line 736 of file gs_device_crystal.F90.

Here is the caller graph for this function:

cr_vec_index()

subroutine gs_device_crystal::cr_vec_index ( class(gs_device_crystal_t), intent(inout)  this, integer, intent(in)  nc  )

private

Parameters

nc	number of components to build the lists for. Rebuilt if a later exchange asks for a different number, which nothing does today

Definition at line 688 of file gs_device_crystal.F90.

Here is the call graph for this function:

Here is the caller graph for this function:

gs_device_crystal_free()

subroutine gs_device_crystal::gs_device_crystal_free ( class(gs_device_crystal_t), intent(inout)  this )

private

Definition at line 321 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_init()

subroutine gs_device_crystal::gs_device_crystal_init ( class(gs_device_crystal_t), intent(inout)  this, type(stack_i4_t), intent(inout)  send_pe, type(stack_i4_t), intent(inout)  recv_pe  )

private

Definition at line 239 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_init_vec()

subroutine gs_device_crystal::gs_device_crystal_init_vec ( class(gs_device_crystal_t), intent(inout)  this )

private

Definition at line 307 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_nbrecv()

subroutine gs_device_crystal::gs_device_crystal_nbrecv ( class(gs_device_crystal_t), intent(inout)  this, integer, intent(in)  tag  )

private

Definition at line 360 of file gs_device_crystal.F90.

gs_device_crystal_nbrecv_vec()

subroutine gs_device_crystal::gs_device_crystal_nbrecv_vec ( class(gs_device_crystal_t), intent(inout)  this, integer, intent(in)  tag, integer, intent(in)  nc  )

private

Definition at line 480 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_nbsend()

subroutine gs_device_crystal::gs_device_crystal_nbsend ( class(gs_device_crystal_t), intent(inout)  this, real(kind=rp), dimension(n), intent(inout)  u, integer, intent(in)  n, integer, intent(in)  tag, type(c_ptr), intent(inout)  deps, type(c_ptr), intent(inout)  strm  )

private

Definition at line 386 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_nbsend_vec()

subroutine gs_device_crystal::gs_device_crystal_nbsend_vec ( class(gs_device_crystal_t), intent(inout)  this, real(kind=rp), dimension(nc*n), intent(inout)  u, integer, intent(in)  n, integer, intent(in)  nc, integer, intent(in)  tag, type(c_ptr), intent(inout)  deps, type(c_ptr), intent(inout)  strm  )

private

Parameters

u	compact shared device buffer, component-outer with stride n

Definition at line 515 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_nbwait()

subroutine gs_device_crystal::gs_device_crystal_nbwait ( class(gs_device_crystal_t), intent(inout)  this, real(kind=rp), dimension(n), intent(inout)  u, integer, intent(in)  n, integer  op, type(c_ptr), intent(inout)  strm  )

private

Definition at line 418 of file gs_device_crystal.F90.

Here is the call graph for this function:

gs_device_crystal_nbwait_vec()

subroutine gs_device_crystal::gs_device_crystal_nbwait_vec ( class(gs_device_crystal_t), intent(inout)  this, real(kind=rp), dimension(nc*n), intent(inout)  u, integer, intent(in)  n, integer, intent(in)  nc, integer  op, type(c_ptr), intent(inout)  strm  )

private

Definition at line 546 of file gs_device_crystal.F90.

Here is the call graph for this function: