- Generated on Tue Feb 18 2025 03:36:48 for Neko by
1.9.8
|
Neko 0.9.99
A portable framework for high-order spectral element flow simulations
|
The user file is a fortran file where the user can implement their own functions to extend the capabilities of the default Neko executable. The user file can be used for setting advanced initial/boundary conditions, source terms, I/O operations, and interactions with the Neko framework.
The user file is a regular Fortran .f90 file that needs to be compiled with makeneko, located in the bin folder of your neko installation. To compile a user file user.f90, run:
If everything goes well, you should observe the following output:
Compiling your user file with makeneko will create a neko executable, which you will need to execute with your case file as an argument. For example, if your case file is called user.case:
Or in parallel using MPI:
The current high-level structure of the user file is shown below.
The user file implements the user module. The user modules contains a subroutine named user_setup, which we use to interface the internal procedures defined in src/common/user_intf.f90 with the subroutines that you will implement in your user file. Each user subroutine should be implemented under the contains statement, below user_setup.
bin/neko in your local neko installation folder.The following user functions, if defined in the user file, will always be executed, regardless of what is set in the case file:
rho, mu, cp and lambda.d=1. For more information on the scalar, see the relevant section of the case file.The two subroutines user_init_modules and user_finalize_modules may be used to initialize/finalize any user defined variables, external objects, or processes. They are respectively executed right before/after the simulation time loop.
In the example above, the subroutines initialize and finalize contain the actual implementations. They must also be interfaced to the internal procedures user_init_modules and user_finalize_modules in user_setup:
user_init_modules and user_finalize_modules are independent of each other. Using one does not require the use of the other.The subroutine user_check is executed at the end of every time step. It can be used for computing and/or outputting your own variables/quantities at every time step.
In the example above, the subroutine usercheck contains the actual implementation, and needs to be registered by adding:
to our user_setup.
material_properties allows for more complex computations and setting of various material properties, such as rho, mu for the fluid and cp, lambda for the scalar. The example below is taken from the rayleigh_benard_cylinder example.
And of course not forgetting to register our function in user_setup by adding the following line:
This user function allows for the modification of the mesh at runtime, by acting on the element nodes of the mesh specified in the case file. This function is only called once before the simulation time loop. The example below is taken from the tgv example.
The registering of the above function in user_setup should then be done as follows:
This user function can be used to specify the scalar boundary values, on all zones that are not already set to uniform Dirichlet or Neumann values e.g. d=1 or n=0. For more information on the scalar, see the relevant section of the case file. The example below sets the scalar boundary condition values to be a linear function of the z coordinate (taken from the rayleigh_benard example).
This function will be called on all the points on the relevant boundaries. The registering of the above function in user_setup should be done as follows:
In addition to the case-specific user functions, the user can also define their own simulation components. This can be done by writing a new type which extends the simulation_component_t type, and implementing the necessary functions for the new type. The user can then specify the component in the list of simulation components in the case file. The setting is_user should be set to true in the JSON object for the new simulation component. The typename is used to extract the settings for the simulation component from the JSON file.
In the example above, the subroutine user_simcomp contains the actual implementation, and needs to be registered by adding:
A full example of a user-defined simulation component can be found in the examples.
As explained in the case file page, certain components of the simulation can be set to be user defined. These components and their associated user functions are:
| Description | User function | JSON Object in the case file |
|---|---|---|
| Fluid initial condition | fluid_user_ic | case.fluid.initial_condition |
| Scalar initial condition | scalar_user_ic | case.scalar.initial_condition |
| Fluid inflow boundary condition | fluid_user_if | case.fluid.inflow_condition |
| Scalar boundary conditions | scalar_user_bc | (user function is always called) |
| Fluid source term | fluid_user_f_vector or fluid_user_f | case.fluid.source_terms |
| Scalar source term | scalar_user_f_vector or scalar_user_f | case.scalar.source_terms |
| Fluid and Scalar boundary conditions | field_dirichlet_update | case.fluid.boundary_types and/or case.scalar.boundary_types |
Note that scalar_user_bc is included for completeness but is technically not case-specific.
Enabling user defined initial conditions for the fluid and/or scalar is done by setting the initial_condition.type to "user" in the relevant sections of the case file, case.fluid and/or case.scalar.
See the relevant sections on the fluid and scalar initial conditions in the case file page for more details.
The associated user functions for the fluid and/or scalar initial conditions can then be added to the user file. An example for the fluid taken from the advecting cone example, is shown below.
NEKO_BCKND_DEVICE flag, which will be set to 1 if running on GPUs, and the calls to device_memcpy to transfer data between the host and the device. See Running on GPUs for more information on how this works.The same can be done for the scalar, with the example below also inspired from the advecting cone example:
NEKO_BCKND_DEVICE flag, which will be set to 1 if running on GPUs, and the calls to device_memcpy to transfer data between the host and the device. See Running on GPUs for more information on how this works.We should also add of the following lines in user_setup, registering our user functions set_velocity and set_s_ic to be used as the fluid and scalar initial conditions:
Enabling user defined inflow condition for the fluid is done by setting the case.fluid.inflow_condition.type to "user":
See the the relevant section in the case file page for more details. The associated user function for the fluid inflow condition can then be added to the user file. An example inspired from the lid-driven cavity example is shown below.
We should also add of the following line in user_setup, registering our user function user_bc to be used as the fluid inflow conditions:
Enabling user defined source terms for the fluid and/or scalar is done by adding JSON Objects to the case.fluid.source_terms and/or case.scalar.source_terms lists.
See the relevant sections on the fluid and scalar source terms in the case file page for more details.
_user_f and _user_f_vector. The former is called when setting "user_pointwise" as the source term type, while the latter requires the use of the "user_vector" keyword in the case file. The pointwise variant, fluid_user_f is not supported on GPUs. In general, fluid_user_f_vector is the prefered variant, and is the one which will be use in our examples below. The same applies for the scalar source term user functions.The associated user functions for the fluid and/or scalar source terms can then be added to the user file. An example for the fluid, taken from the rayleigh_benard_cylinder example, is shown below.
neko_field_registry to retrieve the velocity and scalar fields. See Registries for more information about registries in neko. NEKO_BCKND_DEVICE flag, which will be set to 1 if running on GPUs, and the use of device_ functions. See Running on GPUs for more information on how this works.The same can be done for the scalar, with the example below also taken from the scalar_mms example:
NEKO_BCKND_DEVICE flag, which will be set to 1 if running on GPUs, and the call to device_memcpy to transfer data between the host and the device. See Running on GPUs for more information on how this works.We should also add of the following lines in user_setup, registering our user functions set_boussinesq_forcing_term and set_source to be used as the fluid and scalar source terms:
This user function can be used to specify Dirichlet boundary values for velocity components u,v,w, the pressure p, and/or the scalar s. This type of boundary condition allows for time-dependent velocity profiles (currently not possible with the user_pointiwse boundary condition) or non-uniform pressure profiles to e.g. impose an outlet pressure computed from another simulation.
The user routine is called by the user_velocity and user_pressure boundary conditions for the fluid, and the user boundary condtition for the scalar. Once the appropriate boundaries have been identified, the user function field_dirichlet_update should be used to compute and apply the desired values to our velocity/pressure/scalar field(s). The prefix "field" in field_dirichlet_update refers to the fact that a list of entire fields is passed down for the user to edit.
The fields that are passed down depends on the underlying boundary conditions. For user_velocity, it will be a list of 3 fields with names u, v, w; For user_pressure, a list with a single field p; For the scalar, also a single field with the same name as the solution field for the scalar (s by default).
It is crucial to understand that all three boundary condition will call the same routine! So, if one has, for example, both user_velocity for the fluid and user for the scalar, it is necessary to have an if statement in the user routine to distinguish between the two cases. The convenient way to do that is to check the size of the passed field list and the names of the fields inside. For example, if there is one field and it is called s, one executes the code setting the boundary values for the scalar s.
Note that the fields that one manipulates in the user routine are not the actual velocity fields, etc. Instead, the code does a masked copy from the dummy fields manipulated in the user routine to the actual fields of unknowns, with the mask corresponding to the boundary faces. So, even if you somehow manipulate the fields elsewhere in the domain inside the user routine, that will not affect the solution.
In the following example, we indicate in case.fluid.boundary_conditions that we would like to apply a velocity profile on the boundary number 1 (in this case, the inlet boundary). On boundary number 2 (the outlet boundary), we change the pressure. In case.scalar.boundary_conditions, we indicate the same for the scalar on boundaries 1 and 2 (inlet and outlet).
The header of the user function is given in the code snippet below.
The arguments and their purpose are as follows:
field_bc_list is the list of the field that can be edited. It is a list of field_t objects.i contained in field_bc_list is accessed using field_bc_list%items(i)%ptr and will refer to a field_t object.bc contains a field_dirichlet_t object to help access the boundary indices through the boundary mask, msk.bcobject is accessed via bc%msk. It contains the linear indices of each GLL point on the boundary facets. msk(0) contains the size of the array. The first boundary index is msk(1).coef is a coef_t object containing various numerical parameters and variables, such as the polynomial order lx, derivatives, facet normals...t, tstep are self-explanatory.Links to the documentation to learn more about what the types mentioned above contain and how to use them: src/field/field.f90, src/bc/bc.f90, src/sem/coef.f90.
The user function should be registered in user_setup with the following line:
A very simple example illustrating the above is shown below, which is taken from the cyl_boundary_layer example
This example is applying constant dirichlet values at the selected boundaries for the velocity components and presure. The scalar is applied a function s(y,z) = sin(y)*sin(z) to demonstrate the usage of boundary masks.
u = 1.0_rp is only possible because of the overloading of the assignement operator = in field_t. In general, a field's array should be accessed and modified with u%x.Note that we are only applying our boundary values at the first timestep, which is done simply with the line if (tstep .ne. 1) return. This is a trick that can be used for time-independent boundary profiles that require some kind of time consuming operation like interpolation or reading from a file, which would add overhead if executed at every time step.
device_memcpy to make sure the device arrays are also updated.When running on GPUs, special care must be taken when using certain user functions. The short explanation is that the device (GPU) has its own memory and cannot directly access the memory on the host (CPU). This means that data and more specifically arrays must be copied manually from the host to the device (see device::device_memcpy).
device_memcpy is avoidable. Neko has some device math functions implemented that operate directly on device arrays. If you can decompose whatever operations you are performing in a user function into a set of instructions from the math module (e.g. cadd, cfill, sub2, ...), you may use the corresponding device_math functions to offload work to the GPU. See the fluid forcing code snippet for a simple example. For more advanced examples, see the rayleigh_benard example or the tgv example.To illustrate this, let us have a look at the fluid initial condition code snippet:
The code above is used to set the fluid initial condition, by specifying the values of fields u,v,w (and p) at all points in the domain. Notice that we have divided the above code into two parts.
In the first part, we set the velocity components u=-y*pi*, v=x*pi*, and w=0, which updates the velocity field arrays u%x, v%x, w%x allocated on the host (CPU). If we were to run on GPUs, these lines of code would only act on the velocity arrays on the host (CPU), leaving the device (GPU) arrays untouched.
We take care of this in the second part, for all three velocity arrays. To update the device (GPU) arrays, we use device_memcpy to copy the data contained in a host (CPU) array to a device (GPU) array. Looking at the details of the device_memcpy calls, we note the following:
_d to the host array variable name (e.g. u%x and u%x_d). This is the standard in Neko.HOST_TO_DEVICE. Other flags can also be used to move data from device to host (DEVICE_TO_HOST) or device to device (DEVICE_TO_DEVICE). See the accelerators page for more details on this.sync argument is a non-optional argument which dictates wether or not to perform the data transfer synchronously.sync = .true. as a starting point.Finally, observe that we use the flag NEKO_BCKND_DEVICE to check if we are indeed running on GPUs. In that case, NEKO_BCKND_DEVICE would be equal to 1.
Neko uses the concept of registry as a practical way to retrieve fields and point zones anywhere in the user file.
The field registry neko_field_registry is often used in user functions where certain fields are not directly accessible as arguments. One can retrieve any field in the registry by its name with neko_field_registry%get_field(name). Default fields that are added to the registry are u,v,w,p and s if running with the scalar enabled. For a practical example of usage, see the rayleigh benard example
Other fields may be added to the registry by various simulation components. For example:
simulation_components.vorticity enabled, the fields omega_x, omega_y, omega_z will be accessible in the registry.simulation_components.lambda2 enabled, the field lambda2 will be accessible in the registry.neko_field_registry%add_field (see field_registry::field_add).The point zone registry, neko_point_zone_registry, can be used to retrieve pointers to point_zone_t objects defined in the case file. See using point zones for detailed instructions.
1.9.8