|
Neko 1.99.9
A portable framework for high-order spectral element flow simulations
|
The case file defines all the parameters of a simulation. The format of the file is JSON, making it easy to read and write case files using the majority of the popular programming languages. JSON is hierarchical and consists of parameter blocks enclosed in curly braces. These blocks are referred to as objects. The case file makes use of objects to separate the configuration of different parts of the solver. We refer the reader to the examples shipped with the code to get a good idea of how a case file looks. The table below provides a complete reference for all possible configuration choices.
/, ! or # to make line comments and tolerates trailing commas before a closing } or ].A JSON Schema for the case file is available in doc/schemas/. The helper contrib/validate_case_schema.py can be used to validate case files against that schema. This is a convenient way to quickly check that your case file is valid without having to run a simulation.
10 from JSON reals such as 10.0. Make sure that you use the correct number type for the parameter you are setting.Keys that the schemas do not define are rejected in the objects of built-in components, so that a misspelt key, which Neko would silently ignore, does not go unnoticed. Other keys are accepted at the top level of the file, next to case, for instance for the settings of other tools, in case, case.fluid and case.scalar, and in user-defined components: initial conditions of type user, boundary conditions of the user_* types and source terms of type user, whose parameters the user file defines.
A case file can be checked without running Neko with checkneko, which is installed with the other contrib tools and needs neither MPI nor the Neko library:
Run it in the directory Neko will run in, or give that directory with -d dir, since file names in the case file are relative to it. The case file is parsed the way Neko parses it and validated against the schemas in doc/schemas, which are installed with the tool in share/neko/schemas. It then checks what the schemas cannot:
case.constants, of the kind Neko reads there..nmsh mesh, the zone_indices of the boundary conditions of the fluid and the scalars must match the labeled zones of the mesh, with one boundary condition for each zone, as Neko requires.case.point_zones.A table lists the labeled zones with their number of facets and the boundary conditions of the fluid and of each scalar on them, for example
where - means that a field has no boundary condition on a zone, and the name of an entry is given in parentheses. Each problem is then reported on one line with its line number and its location in the case file, for example, with the list of types shortened here,
The exit status is 1 if there are errors. Warnings, for instance about a key that appears twice or a misspelled key in an object that allows any key, do not change the exit status. The tool can also be compiled on its own, for instance when Neko is cross-compiled for another architecture, see contrib/checkneko/README.md.
The current high-level structure of the case file is shown below.
The version keyword is reserved to track changes in the format of the file. The subsections below list all the configuration options for each of the high-level objects. Some parameters will have default values, and are therefore optional.
scalars object is used to set up scalar advection-diffusion-reaction equations, as discussed below. The object scalar can alternatively be used to set up a single equation, simplifying the layout.This object is mostly used as a high-level container for all the other objects, but also defines several parameters that pertain to the simulation as a whole.
| Name | Description | Admissible values | Default value |
|---|---|---|---|
no_defaults | Prevents filling in default values to case file entries. | true or false | false |
mesh_file | The name of the mesh file. | Strings ending with .nmsh | - |
output_boundary | Whether to write a bdry0.f0000 file with boundary labels. Can be used to check boundary conditions. | true or false | false |
output_directory | Folder for redirecting solver output. Note that the folder has to exist! | Path to an existing directory | . |
output_precision | Whether to output snapshots in single or double precision | single or double | single |
output_layout | Data layout for adios2 files. (Choose 2 or 3 for ADIOS2 supported compressors BigWhoop or ZFP.) | Positive integer 1, 2, 3 | 1 |
load_balancing | Whether to apply load balancing. | true or false | false |
output_partitions | Whether to write a partitions.vtk file with domain partitioning. | true or false | false |
output_checkpoints | Whether to output checkpoints, i.e. restart files. | true or false | - |
checkpoint_control | Defines the interpretation of checkpoint_value to define the frequency of writing checkpoint files. | nsamples, simulationtime, tsteps, never | - |
checkpoint_value | The frequency of sampling in terms of checkpoint_control. | Positive real or integer | - |
checkpoint_filename | The filename of written checkpoint. | Strings such as my_name | fluid |
checkpoint_format | The file format of checkpoints | chkp or hdf5 | chkp |
restart_file | Checkpoint to use for a restart from previous data | Strings ending with .chkp, .h5, or .hdf5 | - |
restart_mesh_file | If the restart file is on a different mesh, specify the .nmsh file used to generate it here | Strings ending with .nmsh | - |
mesh2mesh_tolerance | Tolerance for the restart when restarting from another mesh | Positive reals | 1e-6 |
job_timelimit | The maximum wall clock duration of the simulation. | String formatted as [[[DD-]HH:]MM:]SS | No limit |
output_at_end | Whether to always write all enabled output at the end of the run. | true or false | true |
output_at_start | Whether to write the initial state of the simulation. Checkpoints and statistics never do. | true or false | true |
Some additional practical comments are provided regarding the output triggered by job_timelimit and output_at_end keywords.
If output_at_end is set to true, a write is performed after the execution of the simulation time-loop is finished. This triggers most outputs, like the fluid solvers, the checkpoint, etc. An output that has already been written at the last time step of the run is not written a second time, so case settings under which an output lands on the last step anyway (e.g. end_time: 5, checkpoint_value: 5, checkpoint_control: simulationtime) produce one file at the end of the run and not two. output_at_end does override a never control, which is the way to ask for an output that is written once, at the end of the run, and never in between.
The file numbering follows the schedule: the k-th file is the k-th scheduled write, and a restart resumes where the schedule says the run has got to. A run that repeats an interval it has already covered, or that changes the output frequency at the restart, therefore writes over the files of the earlier run, and Neko warns when the first file it is about to write already exists.
The purpose of job_timelimit is to gracefully stop the simulation in a typical supercomputer environment, where your runtime is limited. When Neko detects that the time of the run exceeds the job_timelimit, it exits the time-loop. At this point, if one sets output_at_end to true, this will trigger a write as per usual. However, if output_at_end is false, Neko will still write a special checkpoint file, with the filename called joblimit#####.chkp. This is done so that the user is at least provided a restart file, and none of the computer time spent on the simulation is wasted. Generally, however, it is recommended to have output_at_end set to true in tandem with job_timelimit, so that what exactly gets written is controlled by the case file settings. Note that the time format is flexible. For example, 1-01:00:00 and 25:00:00 are both valid ways to specify a 25-hour time limit.
The constants array allows the user to define parameters that are global to the case file, and can be referred to when setting the values of other parameters. Two types of parameters can be defined: scalars and arrays. Each is represented as a subobject inside the constants object and should containt two entries: name and value. Here is an example:
Other parameters in the case file that require a scalar or array entry, can instead be defined as a string, pointing to the name of the corresponding parameter in the constants object. As an example, recall that output frequency is controlled by the keyword output_value. It is a plausible scenario that the frequency is the same for multiple solvers, simulation components, etc. Assuming a simulation with both fluid and [scalar](Scalar) solvers active, the following could be used.
The advantage is that this guarantees that the fluid and scalar output will be in sync, and if one wants to change the frequency only does that in one place in the case file. Another use case is demonstrated in the hemi example, where the freestream velocity is defined under constants and then used to setup both initial and boundary conditions.
Under the hood, Neko stores the constants in an object called neko_const_registry, which is of the type registry_t (same as neko_registry). The object is accessible in the user file.
Some parameters accept a mathematical expression, written as a string, instead of a number. The expression is evaluated in every point where the parameter is needed, which makes it possible to prescribe a spatially varying profile without writing a user file.
The following can be used in an expression.
| Kind | Available |
|---|---|
| Coordinates | x, y, z |
| Time | t, dt, where the parameter is evaluated during the simulation |
| Constants | pi, and any scalar declared under case.constants |
| Operators | +, -, *, /, ^ (also written **), parentheses |
| Functions | sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, exp, log, log10, sqrt, abs, erf, erfc, step |
atan2(y, x), min(a, b), max(a, b), mod(a, b) |
step(a) is 0 for a < 0 and 1 otherwise. Precedence and associativity follow Fortran, so -x^2 is -(x^2) and 2^3^2 is 2^(3^2).
Constants are read when the expression is compiled, at setup, and multiplication must always be written out, so 2*x is valid whereas 2x is not. An expression that refers to an unknown name is reported as an error at setup, rather than during the simulation. An expression that does not evaluate to a finite value everywhere, typically because of a division by zero or the square root of a negative number, is also reported as an error, at setup if it does not depend on time and otherwise every time it is evaluated.
The time object is used to define the time-stepping of the simulation, including the time-step size, the start and end time, and the variables related to the variable time-stepping algorithm. For the variable timestep, one can specify a timestep, such that the first timestep will be assigned to the smallest of timestep and the value calculated from the target CFL number.
| Name | Description | Admissible values | Default value |
|---|---|---|---|
start_time | Start time at which the simulation is initiated. | Non-negative reals | 0.0 |
end_time | Final time after which the simulation is stopped. | Positive reals | - |
timestep | Time-step size | Positive reals | - |
variable_timestep | Whether to use variable dt | true or false | false |
max_timestep | Maximum time-step size when variable time step is activated | Positive reals | huge |
min_timestep | Minimum time-step size when variable time step is activated | Non-negative reals | 0.0 |
target_cfl | The desired CFL number | Positive real | 0.4 |
max_update_frequency | Minimum number of time steps between two time-step updates triggered by CFL deviation | Non-negative integer | 0 |
min_update_frequency | Maximum number of time steps before forcing a time-step update | Non-negative integer | huge |
max_dt_increase_factor | The maximum scaling factor to increase time step | Positive real greater than 1 | 1.2 |
min_dt_decrease_factor | The minimum scaling factor to decrease time step | Positive real less than 1 | 0.5 |
cfl_deviation_tolerance | The tolerance of the deviation from the target CFL number | Positive real less than 1 | 0.2 |
cfl_running_avg_coeff | The running average coefficient a where cfl_avg_new = a * cfl_new + (1-a) * cfl_avg_old | Positive real between 0 and 1 | 0.5 |
exact_output_time | Whether to fit the variable dt so that the sampling and output times are reached exactly | true or false | false |
Restarts will restart the simulation from the exact state at a given time that the checkpoint was written. This means that the flow field and potential scalars will be at the exact same values before as after restarts. However, derived quantities from the flow field and any observables are not guaranteed to be restarted. In addition, Neko does not guarantee that any files are not overwritten. As such, it is recommended to run in different directories if doing large scale simulations that require many restarts. Unless output_at_end is disabled Neko will also ensure that all output is written to file when reaching the end_time or the job_timelimit. In particular, unless output_checkpoints and output_at_end are set to false a checkpoint at the final time will be written as to avoid losing progress as far as possible.
When the output_boundary setting is set to true, and additional .fld file will be stored in the beginning of the simulation, where the recognized boundary conditions for the fluid will be marked with an integer number. This is a good way to debug the simulation setup. The value of the number depends on the type of the boundary as follows.
| Boundary Condition | Key |
|---|---|
| no_slip (stationary wall) | 1 |
| velocity_value | 2 |
| outflow, normal_outflow (+dong) | 3 |
| symmetry | 4 |
| periodic | 6 |
| user_velocity | 7 |
| user_pressure | 8 |
| shear_stress | 9 |
| wall_model | 10 |
| blasius_profile | 11 |
| no_slip (moving wall) | 12 |
For a description of the boundary conditions themselves, see below.
Used to define the properties of the numerical discretization.
| Name | Description | Admissible values | Default value |
|---|---|---|---|
polynomial_order | The order of the polynomial basis. | Integers >= 3 for pnpn; integers >= 1 for compressible (typically 5 to 9) | - |
time_order | The order of the time integration scheme. Refer to the time_scheme_controller type documentation for details. | 1, 2, 3 | - |
dealias | Whether to apply dealiasing to advection terms. | true or false | false |
dealiased_polynomial order | The polynomial order in the higher-order space used in the dealising. | Integer | 3/2(polynomial_order + 1) - 1 |
oifs | Whether to apply the Operator-Integration-Factor-Splitting (OIFS). | true or false | false |
oifs_target_cfl | The desired OIFS-CFL number. Requires variable_timestep = true in the time control object. | Positive real | 1.9 |
compress_geometric_factors | Whether the standard Helmholtz operators of the fluid, the scalars and the multigrid levels read the geometric factors from one copy per distinct element shape, which cuts their memory traffic on meshes of repeated shapes. The copies are held on top of the full factors. CPU, SX and device backends, not XSMM. | true or false | false |
compress_geometric_factors_tolerance | Tolerance on the absolute difference of the geometric factors, summed over the quadrature points of an element with the off-diagonal factors counted twice, under which two elements share them. As it is absolute, it depends on the size of the elements. | Non-negative real | 1e-7 |
The configuration of the fluid and scalar solvers is described on separate pages:
Simulation components enable the user to perform various additional operations, which are not strictly necessary to run the solver. An example could be computing and output of additional fields, e.g. vorticity.
A more detailed description as well as a full list of available components and their setup is provided in a separate page of the manual.
Point zones enable the user to select GLL points in the computational domain according to some geometric criterion. Three predefined geometric shapes are selectable from the case file: boxes, spheres, and cylinders.
A point zone object defined in the case file can be retrieved from the point zone registry, neko_point_zone_registry, and can be used to perform any zone-specific operations (e.g. localized source term, probing...). User-specific point zones can also be added manually to the point zone registry from the user file.
A more detailed description as well as a full list of available components and their setup is provided in a separate page of the manual.
This object adds the collection of runtime statistics (timings) for identified profiling regions. A region is defined as all functions between a call to profiler_start_region(name, id) and profiler_end_region(name, id), where the id is optional. Regions without an id are added as they are first started, so their number is not limited. The ids 1..25 are reserved for regions started with an explicit id. A region is identified by the first 25 characters of its name, so regions whose names share those are timed as one region.
| Name | Description | Admissible values | Default value |
|---|---|---|---|
enabled | Whether to enable gathering of runtime statistics | true or false | false |
output_profile | Whether to output all gathered profiling data as a CSV file | true or false | false |
A common scheme for controlling the output frequency is applied for various outputs. It is described already now in order to clarify the meaning of several parameters found in the tables below.
The frequency is controlled by two parameters, ending with _control and _value, respectively. The latter name is perhaps not ideal, but it is somewhat difficult to come up with a good one, suggestions are welcome.
The _value parameter defines the output frequency, but both its JSON type and interpretation depend on the choice of _control. Neko requires an integer literal for tsteps and nsamples, and a real literal for simulationtime. For example, use 10 for ten time steps but 10.0 for ten simulation-time units. The following options are possible.
simulationtime, then _value is the time interval between the outputs.tsteps, then _value is the number of time steps between the outputs.nsamples, then _value is the total number of outputs that will be performed in the course of the simulation.never, then _value is ignored and output is never performed.The times at which an output is written are a property of the case, not of the run. With simulationtime they are the whole multiples of \( \Delta t_{out} \), so a case asking for an output every 1.0 writes at 11.0, 12.0, 13.0 whether it started from 0, from 10.5, or from a checkpoint in between. With nsamples the samples divide the interval over which the output is active instead, \( t_k = t_{start} + k (t_{end} - t_{start}) / n
\), where \( t_{start} \) is the start of the simulation or, for the statistics, the start of the averaging, and the last sample falls on end_time. Only the times inside the interval where the output is active are written, and since the schedule does not depend on when the run was started, restarting neither skips nor repeats an output and the file numbering carries on.
Each of those times is written at the first step to reach it, at most one file per output per step, so a time step longer than the interval writes once and moves on rather than building up a backlog. The write scheduled for end_time still happens when the last step overshoots it.
The initial state is written in addition to the scheduled times, unless output_at_start is false. Checkpoints and statistics never write it: a checkpoint holds nothing the initial condition does not, and an average over an interval of zero length is empty.