tk::Solver class

Linear system merger and solver Charm++ chare group class

Instantiations of Solver comprise a processor aware Charm++ chare group. When instantiated, a new object is created on each PE and not more (as opposed to individual chares or chare array object elements). The group's elements are used to collect information from all chare objects that happen to be on a given PE. See also the Charm++ interface file solver.ci.

Public static functions

static void registerReducers()
Configure Charm++ reduction types for concatenating BC nodelists.

Constructors, destructors, conversion operators

Solver(CProxy_SolverShadow sh, const SolverCallback& cb, std::size_t n)
Constructor.

Public functions

void chbounds(std::size_t lower, std::size_t upper)
void pebounds(int p, std::size_t lower, std::size_t upper)
Compute lower and upper bounds across all PEs.
void computeBounds(int c, std::size_t lower, std::size_t upper)
Compute lower and upper bounds across PEs.
void next()
void nchare(int n)
Set number of worker chares expected to contribute on my PE.
void charecom(const inciter::CProxy_MatCG& worker, int fromch, const std::vector<std::size_t>& row)
Chares contribute their global row ids for establishing communications.
void created()
Signal the runtime system that the workers have been created.
void addrow(int fromch, int frompe, const std::set<std::size_t>& row)
Receive global row ids from fellow group branches.
void recrow()
Acknowledge received row ids.
void charesol(int fromch, const std::vector<std::size_t>& gid, const Fields& solution)
Chares contribute their solution nonzero values.
void addsol(int fromch, const std::map<std::size_t, std::vector<tk::real>>& solution)
Receive solution vector nonzeros from fellow group branches.
void charelhs(int fromch, const std::vector<std::size_t>& gid, const std::pair<std::vector<std::size_t>, std::vector<std::size_t>>& psup, const tk::Fields& lhsd, const tk::Fields& lhso)
Chares contribute their matrix nonzero values.
void addlhs(int fromch, const std::map<std::size_t, std::map<std::size_t, std::vector<tk::real>>>& l)
Receive matrix nonzeros from fellow group branches.
void charerhs(int fromch, const std::vector<std::size_t>& gid, const Fields& r)
Chares contribute their rhs nonzero values.
void addrhs(int fromch, const std::map<std::size_t, std::vector<tk::real>>& r)
Receive+add right-hand side vector nonzeros from fellow group branches.
void charelowrhs(int fromch, const std::vector<std::size_t>& gid, const Fields& lowrhs)
Chares contribute to the rhs of the low-order linear system.
void addlowrhs(int fromch, const std::map<std::size_t, std::vector<tk::real>>& lowrhs)
Receive+add low-order rhs vector nonzeros from fellow group branches.
void charelowlhs(int fromch, const std::vector<std::size_t>& gid, const Fields& lowlhs)
Chares contribute to the lhs of the low-order linear system.
void addlowlhs(int fromch, const std::map<std::size_t, std::vector<tk::real>>& lowlhs)
Receive and add lhs vector to the low-order system from fellow group branches.
void comfinal()
All communications have been establised among PEs.
auto dirbc() -> const std::unordered_map<std::size_t, std::vector<std::pair<bool, tk::real>>>&
void charebc(const std::unordered_map<std::size_t, std::vector<std::pair<bool, tk::real>>>& bc)
Chares contribute their global row ids and associated Dirichlet boundary condition values at which they set BCs.
void addbc(CkReductionMsg* msg)
Reduction target collecting the final aggregated BC node list map.
void charediag(int fromch, uint64_t it, tk::real t, tk::real dt, const std::vector<std::size_t>& gid, const Fields& u, const Fields& a, const std::vector<tk::real>& v)
Chares contribute their numerical and analytical solutions nonzero values for computing diagnostics.
void adddiag(int fromch, std::map<std::size_t, std::vector<std::vector<tk::real>>>& solution)
Receive numerical and analytical solution vector nonzeros from fellow group branches for computing diagnostics.
auto pe(std::size_t gid) -> int
Return processing element for global mesh row id.

Function documentation

static void tk::Solver::registerReducers()

Configure Charm++ reduction types for concatenating BC nodelists.

Since this is a [nodeinit] routine, the runtime system executes the routine exactly once on every logical node early on in the Charm++ init sequence. Must be static as it is called without an object. See also: Section "Initializations at Program Startup" at in the Charm++ manual http://charm.cs.illinois.edu/manuals/html/charm++/manual.html.

tk::Solver::Solver(CProxy_SolverShadow sh, const SolverCallback& cb, std::size_t n)

Constructor.

Parameters
sh in Solver "shadow" Charm++ chare group proxy for starting reductions at the same time as other reductions from Charm++ chare Solver
cb in Charm++ callbacks for Solver
in Total number of scalar components in the linear system

void tk::Solver::chbounds(std::size_t lower, std::size_t upper)

Parameters
lower in Lower index of the global rows of sending chare
upper in Upper index of the global rows of sending chare

Receive lower and upper global node IDs from chares

void tk::Solver::pebounds(int p, std::size_t lower, std::size_t upper)

Compute lower and upper bounds across all PEs.

Parameters
in PE whose bounds being received
lower in Lower index of the global rows of sending PE
upper in Upper index of the global rows of sending PE

void tk::Solver::next()

Prepare for next step Prepare for next step

Re-enable SDAG waits for rebuilding the right-hand side vector only

void tk::Solver::nchare(int n)

Set number of worker chares expected to contribute on my PE.

Parameters
in Total number of chares (work units) across all PEs

void tk::Solver::charecom(const inciter::CProxy_MatCG& worker, int fromch, const std::vector<std::size_t>& row)

Chares contribute their global row ids for establishing communications.

Parameters
worker in Charm chare array proxy contribution coming from
fromch in Charm chare array index contribution coming from
row in Global mesh point (row) indices contributed

void tk::Solver::addrow(int fromch, int frompe, const std::set<std::size_t>& row)

Receive global row ids from fellow group branches.

Parameters
fromch in Charm chare array index contribution coming from
frompe in PE contribution coming from
row in Global mesh point (row) indices received

void tk::Solver::charesol(int fromch, const std::vector<std::size_t>& gid, const Fields& solution)

Chares contribute their solution nonzero values.

Parameters
fromch in Charm chare array index contribution coming from
gid in Global row indices of the vector contributed
solution in Portion of the unknown/solution vector contributed

void tk::Solver::addsol(int fromch, const std::map<std::size_t, std::vector<tk::real>>& solution)

Receive solution vector nonzeros from fellow group branches.

Parameters
fromch in Charm chare array index contribution coming from
solution in Portion of the unknown/solution vector contributed, containing global row indices and values for all components

void tk::Solver::charelhs(int fromch, const std::vector<std::size_t>& gid, const std::pair<std::vector<std::size_t>, std::vector<std::size_t>>& psup, const tk::Fields& lhsd, const tk::Fields& lhso)

Chares contribute their matrix nonzero values.

Parameters
fromch in Charm chare array index contribution coming from
gid in Global row indices of the matrix chunk contributed
psup in Points surrounding points using local indices. See also tk::genPsup().
lhsd in Portion of the left-hand side matrix contributed, containing non-zero values (for all scalar components of the equations solved) as a sparse matrix diagonal
lhso in Portion of the left-hand side matrix contributed, containing non-zero values (for all scalar components of the equations solved) as a sparse matrix off-diagonal entries in compressed row storage format

void tk::Solver::addlhs(int fromch, const std::map<std::size_t, std::map<std::size_t, std::vector<tk::real>>>& l)

Receive matrix nonzeros from fellow group branches.

Parameters
fromch in Charm chare array index contribution coming from
in Portion of the left-hand side matrix contributed, containing global row and column indices and non-zero values

void tk::Solver::charerhs(int fromch, const std::vector<std::size_t>& gid, const Fields& r)

Chares contribute their rhs nonzero values.

Parameters
fromch in Charm chare array index contribution coming from
gid in Global row indices of the vector contributed
in Portion of the right-hand side vector contributed

void tk::Solver::addrhs(int fromch, const std::map<std::size_t, std::vector<tk::real>>& r)

Receive+add right-hand side vector nonzeros from fellow group branches.

Parameters
fromch in Charm chare array index contribution coming from
in Portion of the right-hand side vector contributed, containing global row indices and values

void tk::Solver::charelowrhs(int fromch, const std::vector<std::size_t>& gid, const Fields& lowrhs)

Chares contribute to the rhs of the low-order linear system.

Parameters
fromch in Charm chare array index contribution coming from
gid in Global row indices of the vector contributed
lowrhs in Portion of the low-order rhs vector contributed

void tk::Solver::addlowrhs(int fromch, const std::map<std::size_t, std::vector<tk::real>>& lowrhs)

Receive+add low-order rhs vector nonzeros from fellow group branches.

Parameters
fromch in Charm chare array index contribution coming from
lowrhs in Portion of the low-order rhs vector contributed, containing global row indices and values

void tk::Solver::charelowlhs(int fromch, const std::vector<std::size_t>& gid, const Fields& lowlhs)

Chares contribute to the lhs of the low-order linear system.

Parameters
fromch in Charm chare array the contribution coming from
gid in Global row indices of the vector contributed
lowlhs in Portion of the low-order lhs vector contributed

void tk::Solver::addlowlhs(int fromch, const std::map<std::size_t, std::vector<tk::real>>& lowlhs)

Receive and add lhs vector to the low-order system from fellow group branches.

Parameters
fromch in Charm chare array index contribution coming from
lowlhs in Portion of the lhs vector contributed to the low-order linear system, containing global row indices and values

void tk::Solver::comfinal()

All communications have been establised among PEs.

At this point all solver objects on all PEs must have received their global row ids which means that the communications have been established among all PEs and this the communications (maps) are final on all PEs.

const std::unordered_map<std::size_t, std::vector<std::pair<bool, tk::real>>>& tk::Solver::dirbc()

Chares query Dirichlet boundary conditions

void tk::Solver::charebc(const std::unordered_map<std::size_t, std::vector<std::pair<bool, tk::real>>>& bc)

Chares contribute their global row ids and associated Dirichlet boundary condition values at which they set BCs.

Parameters
bc in Vector of pairs of bool and BC value (BC vector) associated to global node IDs at which the boundary condition is set. Here the bool indicates whether the BC value is set at the given node by the user. The size of the vectors is the number of PDEs integrated times the number of scalar components in all PDEs.

int tk::Solver::pe(std::size_t gid)

Return processing element for global mesh row id.

Parameters
gid in Global mesh point (matrix or vector row) id
Returns PE that owns global row id

First we attempt to the point index in the cache. If that fails, we resort to a linear search across the division map. Once the PE is found, we store it in the cache, so next time the search is quicker. This procedure must find the PE for the id.