inciter::OversetFE class

OversetFE Charm++ chare array used to advance PDEs in time with OversetFE+RK.

Contents

Public static functions

static void registerReducers()
Configure Charm++ custom reduction types initiated from this chare array.

Constructors, destructors, conversion operators

OversetFE(CkMigrateMessage* msg) explicit
Migrate constructor.

Public functions

auto OversetFE(const CProxy_Discretization& disc, const CProxy_Ghosts& ghostsproxy, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>& bnode, const std::vector<std::size_t>& triinpoel) -> OversetFE_SDAG_CODE explicit
Constructor.
void ResumeFromSync() override
Return from migration.
void setup()
Start setup for solution.
void box(tk::real v, const std::vector<tk::real>& blkvols)
Receive total box IC volume and set conditions in box.
void transferSol()
Transfer solution to other solver and mesh if coupled.
void start()
[Merge normals and continue]
void advance(tk::real newdt, tk::real nmovedmesh)
Advance equations to next time step.
void lhs()
Compute left-hand side of transport equations.
void comdfnorm(const std::unordered_map<tk::UnsMesh::Edge, std::array<tk::real, 3>, tk::UnsMesh::Hash<2>, tk::UnsMesh::Eq<2>>& dfnorm)
Receive contributions to duual-face normals on chare boundaries.
void comnorm(const std::unordered_map<int, std::unordered_map<std::size_t, std::array<tk::real, 4>>>& innorm)
Receive boundary point normals on chare-boundaries.
void comblk(const std::vector<std::size_t>& gid, const std::vector<std::set<std::size_t>>& mb)
Receive mesh block information for nodes on chare-boundaries.
void comChBndGrad(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& G)
Receive contributions to gradients on chare-boundaries.
void comrhs(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& R)
Receive contributions to right-hand side vector on chare-boundaries.
void update(const tk::Fields& a)
Update solution at the end of time step.
void refine(const std::vector<tk::real>& l2res)
Optionally refine/derefine mesh.
void extractFieldOutput(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk&, const tk::UnsMesh::Coords&, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>&, const tk::NodeCommMap&, const std::map<int, std::vector<std::size_t>>&, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>&, CkCallback)
Extract field output to file.
auto solution() const -> const tk::Fields&
void step()
Evaluate whether to continue with next time step.
void evalLB(int nrestart)
void stage()
Evaluate whether to continue with next time step stage.
void next()
Continue to next time step.
void resizeComm()
Size communication buffers (no-op)
void nodeNeighSetup()
Setup node-neighborhood (no-op)
void resizePostAMR(const std::vector<std::size_t>&, const tk::UnsMesh::Chunk&, const tk::UnsMesh::Coords&, const std::unordered_map<std::size_t, tk::UnsMesh::Edge>&, const std::unordered_map<std::size_t, std::size_t>&, const std::set<std::size_t>&, const std::unordered_map<std::size_t, std::size_t>&, const tk::NodeCommMap&, const std::map<int, std::vector<std::size_t>>&, const std::map<int, std::vector<std::size_t>>&, const std::vector<std::size_t>&, const std::unordered_map<std::size_t, std::set<std::size_t>>&)
Receive new mesh from Refiner (no-op)
void resized()
Resizing data structures after mesh refinement has completed (no-op)

Charm++ pack/unpack serializer member functions

CProxy_Discretization m_disc
Discretization proxy.
std::size_t m_nsol
Counter for high order solution vector nodes updated.
std::size_t m_ngrad
Counter for nodal gradients updated.
std::size_t m_nrhs
Counter for right-hand side vector nodes updated.
std::size_t m_nbnorm
Counter for receiving boundary point normals.
std::size_t m_ndfnorm
Counter for receiving dual-face normals on chare-boundary edges.
std::size_t m_nmblk
Counter for mesh block nodes updated.
std::map<int, std::vector<std::size_t>> m_bnode
Boundary node lists mapped to side set ids used in the input file.
std::map<int, std::vector<std::size_t>> m_bface
Boundary face lists mapped to side set ids used in the input file.
std::vector<std::size_t> m_triinpoel
Boundary triangle face connecitivity where BCs are set by user.
std::vector<std::size_t> m_bndel
Elements along mesh boundary.
std::unordered_map<tk::UnsMesh::Edge, std::array<tk::real, 3>, tk::UnsMesh::Hash<2>, tk::UnsMesh::Eq<2>> m_dfnorm
Dual-face normals along edges.
std::unordered_map<tk::UnsMesh::Edge, std::array<tk::real, 3>, tk::UnsMesh::Hash<2>, tk::UnsMesh::Eq<2>> m_dfnormc
Receive buffer for dual-face normals along chare-boundary edges.
std::vector<tk::real> m_dfn
Streamable dual-face normals.
std::pair<std::vector<std::size_t>, std::vector<std::size_t>> m_esup
El;ements surrounding points.
std::pair<std::vector<std::size_t>, std::vector<std::size_t>> m_psup
Points surrounding points.
tk::Fields m_u
Unknown/solution vector at mesh nodes.
tk::Fields m_uc
Copy of unknown/solution vector at mesh nodes for m2m transfer, appended with a solution-transfer-flag. This flag indicates appropriate solution transfers for the overset procedure. Value 0: Do not transfer solution, 1: transfer sol, 2: blank nodes TODO: avoid creating this copy.
tk::Fields m_un
Unknown/solution vector at mesh nodes at previous time.
tk::Fields m_rhs
Right-hand side vector (for the high order system)
std::unordered_map<std::size_t, std::vector<tk::real>> m_rhsc
tk::Fields m_chBndGrad
Nodal gradients at chare-boundary nodes.
std::unordered_map<std::size_t, std::vector<std::pair<bool, tk::real>>> m_dirbc
std::unordered_map<std::size_t, std::vector<tk::real>> m_chBndGradc
std::vector<tk::real> m_blank
Blanking coefficient for overset, indicating hole in the background mesh.
NodeDiagnostics m_diag
Diagnostics object.
std::unordered_map<int, std::unordered_map<std::size_t, std::array<tk::real, 4>>> m_bnorm
std::unordered_map<int, std::unordered_map<std::size_t, std::array<tk::real, 4>>> m_bnormc
Receive buffer for communication of the boundary point normals associated to side sets.
std::unordered_set<std::size_t> m_symbcnodes
Unique set of nodes at which symmetry BCs are set.
std::unordered_set<std::size_t> m_farfieldbcnodes
Unique set of nodes at which farfield BCs are set.
std::vector<int> m_symbctri
Vector with 1 at symmetry BC boundary triangles.
std::vector<std::unordered_set<std::size_t>> m_timedepbcnodes
Unique set of nodes at which time dependent BCs are set.
std::vector<tk::Table<5>> m_timedepbcFn
User defined discrete function of time used in the time dependent.
std::size_t m_stage
Runge-Kutta stage counter.
std::vector<std::unordered_set<std::size_t>> m_boxnodes
Mesh node ids at which user-defined box ICs are defined (multiple boxes)
std::vector<std::size_t> m_edgenode
Local node IDs of edges.
std::vector<std::size_t> m_edgeid
Edge ids in the order of access.
std::vector<tk::real> m_dtp
Time step size for each mesh node.
std::vector<tk::real> m_tp
Physical time for each mesh node.
int m_finished
True in the last time step.
int m_movedmesh
True if overset mesh moved.
std::size_t m_nusermeshblk
Number of mesh-blocks with user-defined ICs.
std::unordered_map<std::size_t, std::set<std::size_t>> m_nodeblockid
Local node ids associated with mesh block ids.
std::unordered_map<std::size_t, std::set<std::size_t>> m_nodeblockidc
std::size_t m_ixfer
Counter for two-way transfer.
std::array<tk::real, 3> m_uservel
User-specified mesh velocity.
void pup(PUP::er& p) override
Pack/Unpack serialize member function.
void operator|(PUP::er& p, OversetFE& i)
Pack/Unpack serialize operator|.

Function documentation

static void inciter::OversetFE::registerReducers()

Configure Charm++ custom reduction types initiated from this chare array.

Since this is a [initnode] routine, the runtime system executes the routine exactly once on every logical node early on in the Charm++ init sequence. Must be static as it is called without an object. See also: Section "Initializations at Program Startup" at in the Charm++ manual http://charm.cs.illinois.edu/manuals/html/charm++/manual.html.

OversetFE_SDAG_CODE inciter::OversetFE::OversetFE(const CProxy_Discretization& disc, const CProxy_Ghosts& ghostsproxy, const std::map<int, std::vector<std::size_t>>& bface, const std::map<int, std::vector<std::size_t>>& bnode, const std::vector<std::size_t>& triinpoel) explicit

Constructor.

Parameters
disc in Discretization proxy
ghostsproxy
bface in Boundary-faces mapped to side sets used in the input file
bnode in Boundary-node lists mapped to side sets used in input file
triinpoel in Boundary-face connectivity where BCs set (global ids)

void inciter::OversetFE::ResumeFromSync() override

Return from migration.

This is called when load balancing (LB) completes. The presence of this function does not affect whether or not we block on LB.

void inciter::OversetFE::setup()

Start setup for solution.

[setup]

void inciter::OversetFE::box(tk::real v, const std::vector<tk::real>& blkvols)

Receive total box IC volume and set conditions in box.

Parameters
v in Total volume within user-specified box
blkvols in Vector of mesh block discrete volumes with user ICs

[setup]

void inciter::OversetFE::start()

[Merge normals and continue]

[start]

void inciter::OversetFE::advance(tk::real newdt, tk::real nmovedmesh)

Advance equations to next time step.

Parameters
newdt in The smallest dt across the whole problem
nmovedmesh in (negative of) if any chunk of this mesh moved

void inciter::OversetFE::lhs()

Compute left-hand side of transport equations.

[Compute lhs]

Also (re-)compute all data structures if the mesh changed.

void inciter::OversetFE::comdfnorm(const std::unordered_map<tk::UnsMesh::Edge, std::array<tk::real, 3>, tk::UnsMesh::Hash<2>, tk::UnsMesh::Eq<2>>& dfnorm)

Receive contributions to duual-face normals on chare boundaries.

Parameters
dfnorm in Incoming partial sums of dual-face normals associated to chare-boundary edges

void inciter::OversetFE::comnorm(const std::unordered_map<int, std::unordered_map<std::size_t, std::array<tk::real, 4>>>& innorm)

Receive boundary point normals on chare-boundaries.

Parameters
innorm in Incoming partial sums of boundary point normal contributions to normals (first 3 components), inverse distance squared (4th component), associated to side set ids

void inciter::OversetFE::comblk(const std::vector<std::size_t>& gid, const std::vector<std::set<std::size_t>>& mb)

Receive mesh block information for nodes on chare-boundaries.

Parameters
gid in Global mesh node IDs at which we receive RHS contributions
mb in Block ids for each node on chare-boundaries

This function receives mesh block information for nodes on chare boundaries. While m_nodeblockid stores block information for own nodes, m_nodeblockidc collects the neighbor chare information during communication. This way work on m_nodeblockid and m_nodeblockidc is overlapped. The two are combined in continueSetup().

void inciter::OversetFE::comChBndGrad(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& G)

Receive contributions to gradients on chare-boundaries.

Parameters
gid in Global mesh node IDs at which we receive grad contributions
G in Partial contributions of gradients to chare-boundary nodes

This function receives contributions to m_chBndGrad, which stores nodal gradients at mesh chare-boundary nodes. While m_chBndGrad stores own contributions, m_chBndGradc collects the neighbor chare contributions during communication. This way work on m_chBndGrad and m_chBndGradc is overlapped. The two are combined in rhs().

void inciter::OversetFE::comrhs(const std::vector<std::size_t>& gid, const std::vector<std::vector<tk::real>>& R)

Receive contributions to right-hand side vector on chare-boundaries.

Parameters
gid in Global mesh node IDs at which we receive RHS contributions
R in Partial contributions of RHS to chare-boundary nodes

This function receives contributions to m_rhs, which stores the right hand side vector at mesh nodes. While m_rhs stores own contributions, m_rhsc collects the neighbor chare contributions during communication. This way work on m_rhs and m_rhsc is overlapped. The two are combined in solve().

void inciter::OversetFE::refine(const std::vector<tk::real>& l2res)

Optionally refine/derefine mesh.

Parameters
l2res in L2-norms of the residual for each scalar component computed across the whole problem

[Refine]

const tk::Fields& inciter::OversetFE::solution() const

Returns Const-ref to current solution

Const-ref access to current solution

void inciter::OversetFE::evalLB(int nrestart)

Parameters
nrestart in Number of times restarted

void inciter::OversetFE::stage()

Evaluate whether to continue with next time step stage.

[Refine]

[stage]

void inciter::OversetFE::pup(PUP::er& p) override

Pack/Unpack serialize member function.

Parameters
p in/out Charm++'s PUP::er serializer object reference

void inciter::OversetFE::operator|(PUP::er& p, OversetFE& i)

Pack/Unpack serialize operator|.

Parameters
p in/out Charm++'s PUP::er serializer object reference
i in/out OversetFE object reference

Variable documentation

std::unordered_map<std::size_t, std::vector<tk::real>> inciter::OversetFE::m_rhsc

Receive buffer for communication of the right hand side

Key: global node id, value: rhs for all scalar components per node.

std::unordered_map<std::size_t, std::vector<std::pair<bool, tk::real>>> inciter::OversetFE::m_dirbc

Boundary conditions evaluated and assigned to local mesh node IDs

Vector of pairs of bool and boundary condition value associated to local mesh node IDs at which the user has set Dirichlet boundary conditions for all PDEs integrated. The bool indicates whether the BC is set at the node for that component the if true, the real value is the increment (from t to dt) in the BC specified for a component.

std::unordered_map<std::size_t, std::vector<tk::real>> inciter::OversetFE::m_chBndGradc

Receive buffer for communication of the nodal gradients

Key: chare id, value: gradients for all scalar components per node

std::unordered_map<int, std::unordered_map<std::size_t, std::array<tk::real, 4>>> inciter::OversetFE::m_bnorm

Face normals in boundary points associated to side sets

Key: local node id, value: unit normal and inverse distance square between face centroids and points, outer key: side set id

std::unordered_map<int, std::unordered_map<std::size_t, std::array<tk::real, 4>>> inciter::OversetFE::m_bnormc

Receive buffer for communication of the boundary point normals associated to side sets.

Key: global node id, value: normals (first 3 components), inverse distance squared (4th component), outer key, side set id

std::unordered_map<std::size_t, std::set<std::size_t>> inciter::OversetFE::m_nodeblockidc

Receive buffer for communication of the mesh block ids

Key: mesh block id, value: set of global node ids for nodes in this mesh block.