1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310 | // *****************************************************************************
/*!
\file src/Inciter/Scheme.hpp
\copyright 2012-2015 J. Bakosi,
2016-2018 Los Alamos National Security, LLC.,
2019-2021 Triad National Security, LLC.
All rights reserved. See the LICENSE file for details.
\brief Polymorphic glue for calling Charm++ entry methods to base class
Discretization, its children implementing specific discretization schemes,
and helper classes
\details
The purpose of this class is to hide, behind a single type, different
Charm++ proxy types that model a single concept, i.e., define some common
functions as Charm++ entry methods that can be used in either a broadcast
and/or in a way of addressing a single array element. As a result, member
functions can be invoked by client code without knowing the underlying type
or any specifics to the underlying differences of the classes that model the
same concept, i.e., expose the same member functions. The idea is very
similar to inheritance and runtime polymorphism with base classes and
virtual functions: some member functions and data are common to all types
modeled (and thus are not repeated and/or copied), while some are specific.
A difference is that the "base" and "child" classes are Charm++ proxies.
Note that while Charm++ does support inheritance and runtime polymorphism
with chare arrays, we still prefer the implementation below because it uses
entirely value semantics (inside and in client code) and thus it keeps the
complexity of the dispatch behind this class and does not expose it to
client code.
The advantages of this class over traditional runtime polymorphism are (1)
value semantics (both internally and to client code), (2) not templated,
and (3) PUPable, i.e., an instance of Scheme can be sent across the network
using Charm++'s pup framework. Also, since the class only holds a couple of
chare proxies, it is lightweight.
Example usage from client code:
\code{.cpp}
// Instantiate a Scheme object
Scheme s( ctr::SchemeType::DG ); // see Control/Inciter/Options/Scheme.h
// Issue broadcast to child scheme entry method
s.bcast< Scheme::setup >(...);
// Issue broadcast to base (Discretization) entry method
s.disc().totalvol();
\endcode
Organization, implementation details, end extension of the class:
Scheme contains (at least) two Charm++ proxies: discproxy and proxy. The
former contains data and functionality common to all discretizations, and
this can be considered as an equivalent to a base class in the OOP sense.
The latter, proxy, contains data and functionality specific to a particular
discretization. When instantiated, Scheme is configured for a single
specific discretization which must be selected from the list of types in
SchemeBase::Proxy.
The underlying type of proxy is a variant, which allows storing exactly one
object. A variant is a type-safe union. An instance of a variant at any
given time either holds a value of one of its alternative types. Read more
on std::variant on how they work.
Adding a new child scheme is done by
(1) Adding a new type of Charm++ chare array proxy to Scheme::Proxy,
(2) Adding a new type of Charm++ chare array element proxy to
Scheme::ProxyElem, and
(3) Adding a new branch to the if test in Scheme's constructor.
\see A talk on "Concept-based runtime polymorphism with Charm++ chare arrays
using value semantics given by J. Bakosi at the 16th Annual Workshop on
Charm++ and its Applications, April 2018, discussing an earlier, more
verbose implementation of the idea, using C++11.
*/
// *****************************************************************************
#ifndef Scheme_h
#define Scheme_h
#include "Exception.hpp"
#include "PUPUtil.hpp"
#include "Inciter/Options/Scheme.hpp"
#include "NoWarning/discretization.decl.h"
#include "NoWarning/diagcg.decl.h"
#include "NoWarning/alecg.decl.h"
#include "NoWarning/distfct.decl.h"
#include "NoWarning/dg.decl.h"
#include "NoWarning/ale.decl.h"
#include "NoWarning/conjugategradients.decl.h"
namespace inciter {
//! Base class for generic forwarding interface to discretization proxies
class Scheme {
private:
//! Variant type listing all chare proxy types modeling the same concept
using Proxy = std::variant< CProxy_DiagCG
, CProxy_DG
, CProxy_ALECG >;
public:
//! Variant type listing all chare element proxy types
using ProxyElem = std::variant< CProxy_DiagCG::element_t
, CProxy_DG::element_t
, CProxy_ALECG::element_t >;
//! Empty constructor for Charm++
explicit Scheme() {}
//! Constructor
//! \param[in] scheme Discretization scheme
//! \param[in] ale True if enable ALE
//! \param[in] linearsolver True if enable a linear solver
//! \details Based on the input enum we create at least two empty chare
//! arrays: (1) discproxy which contains common functionality and data for
//! all discretizations, and (2) proxy, which have functionality and data
//! specific to a given discretization. Note that proxy is bound (in
//! migration behavior and properties) to discproxy.
//! \note There may be other bound proxy arrays created depending on the
//! specific discretization configured by the enum.
explicit Scheme( ctr::SchemeType scheme,
bool ale = false,
bool linearsolver = false ) :
discproxy( CProxy_Discretization::ckNew() )
{
bound.bindTo( discproxy );
if (scheme == ctr::SchemeType::DiagCG) {
proxy = static_cast< CProxy_DiagCG >( CProxy_DiagCG::ckNew(bound) );
fctproxy = CProxy_DistFCT::ckNew(bound);
} else if (scheme == ctr::SchemeType::DG ||
scheme == ctr::SchemeType::P0P1 ||
scheme == ctr::SchemeType::DGP1 ||
scheme == ctr::SchemeType::DGP2 ||
scheme == ctr::SchemeType::PDG)
{
proxy = static_cast< CProxy_DG >( CProxy_DG::ckNew(bound) );
} else if (scheme == ctr::SchemeType::ALECG) {
proxy = static_cast< CProxy_ALECG >( CProxy_ALECG::ckNew(bound) );
} else Throw( "Unknown discretization scheme" );
if (ale) aleproxy = CProxy_ALE::ckNew(bound);
if (linearsolver)
conjugategradientsproxy = tk::CProxy_ConjugateGradients::ckNew(bound);
}
//! Entry method tags for specific Scheme classes to use with bcast()
struct setup {};
struct box {};
struct advance {};
struct resized {};
struct resizeComm {};
struct refine {};
struct lhs {};
struct nodeNeighSetup {};
struct diag {};
struct evalLB {};
struct doneInserting {};
//! Issue broadcast to Scheme entry method
//! \tparam Fn Function tag identifying the entry method to call
//! \tparam Args Types of arguments to pass to entry method
//! \param[in] args Arguments to member function entry method to be called
//! \details This function issues a broadcast to a member function entry
//! method of the Scheme chare array (the child of Discretization) and is
//! thus equivalent to proxy.Fn(...).
template< typename Fn, typename... Args >
void bcast( Args&&... args ) {
std::visit( [&]( auto& p ){<--- Parameter 'p' can be declared with const
if constexpr( std::is_same_v< Fn, setup > )
p.setup( std::forward< Args >( args )... );
if constexpr( std::is_same_v< Fn, box > )
p.box( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, advance > )
p.advance( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, resized > )
p.resized( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, resizeComm > )
p.resizeComm( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, refine > )
p.refine( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, lhs > )
p.lhs( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, nodeNeighSetup > )
p.nodeNeighSetup( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, diag > )
p.diag( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, evalLB > )
p.evalLB( std::forward< Args >( args )... );
else if constexpr( std::is_same_v< Fn, doneInserting > )
p.doneInserting( std::forward< Args >( args )... );
}, proxy );
}
//! Function tags for specific Scheme classes to use with ckLocal()
struct resizePostAMR {};
struct extractFieldOutput {};
struct solution {};
//! Call Scheme function via Charm++ chare array element's ckLocal()
//! \tparam Fn Function tag identifying the function to call
//! \tparam Args Types of arguments to pass to function
//! \param[in] x Chare array element index
//! \param[in] args Arguments to member function function to be called
//! \details This function calls a member function via Charm++'s ckLocal()
//! behind the element proxy configured, indexed by the array index x.
//! Since the call is behind ckLocal(), the member function does not have
//! to be a Charm++ entry method.
template< typename Fn, typename... Args >
auto ckLocal( const CkArrayIndex1D& x, Args&&... args ) const {
auto e = element( x );
return std::visit( [&]( auto& p ){
if constexpr( std::is_same_v< Fn, resizePostAMR > )
return p.ckLocal()->resizePostAMR( std::forward<Args>(args)... );
else if constexpr( std::is_same_v< Fn, extractFieldOutput > )
return p.ckLocal()->extractFieldOutput(std::forward<Args>(args)...);
else if constexpr( std::is_same_v< Fn, solution > )
return p.ckLocal()->solution( std::forward<Args>(args)... );
}, e );
}
//! Function to call the insert entry method of an element proxy
//! \param[in] x Chare array element index
//! \param[in] args Arguments to member function (entry method) to be called
//! \details This function calls the insert member function of a chare array
//! element proxy and thus equivalent to proxy[x].insert(...), using the
//! last argument as default.
template< typename... Args >
void insert( const CkArrayIndex1D& x, Args&&... args ) {
auto e = element( x );
std::visit( [&]( auto& p ){ p.insert(std::forward<Args>(args)...); }, e );<--- Parameter 'p' can be declared with const
}
//! Get reference to discretization proxy
//! \return Discretization Charm++ chare array proxy
CProxy_Discretization& disc() noexcept { return discproxy; }
//! Get reference to DistFCT proxy
//! \return DistFCT Charm++ chare array proxy
CProxy_DistFCT& fct() noexcept { return fctproxy; }
//! Get reference to ALE proxy
//! \return ALE Charm++ chare array proxy
CProxy_ALE& ale() noexcept { return aleproxy; }
//! Get reference to ConjugateGradients proxy
//! \return ConjugateGradients Charm++ chare array proxy
tk::CProxy_ConjugateGradients& conjugategradients() noexcept
{ return conjugategradientsproxy; }
//! Get reference to scheme proxy
//! Get reference to scheme proxy
//! \return Variant storing Charm++ chare array proxy configured
const Proxy& getProxy() noexcept { return proxy; }
//! Query underlying proxy type
//! \return Zero-based index into the set of types of Proxy
std::size_t index() const noexcept { return proxy.index(); }
//! Query underlying proxy element type
//! \return Zero-based index that can be used, e.g., indexing into the set
//! of types of ProxyElem
std::size_t index_element() const noexcept { return element(0).index(); }
//! Charm++ array options accessor for binding external proxies
//! \return Charm++ array options object reference
const CkArrayOptions& arrayoptions() { return bound; }
/** @name Charm++ pack/unpack serializer member functions */
///@{
//! \brief Pack/Unpack serialize member function
//! \param[in,out] p Charm++'s PUP::er serializer object reference
void pup( PUP::er &p ) {<--- Parameter 'p' can be declared with const
p | proxy;
p | discproxy;
p | fctproxy;
p | aleproxy;
p | conjugategradientsproxy;
p | bound;
}
//! \brief Pack/Unpack serialize operator|
//! \param[in,out] p Charm++'s PUP::er serializer object reference
//! \param[in,out] s Scheme object reference
friend void operator|( PUP::er& p, Scheme& s ) { s.pup(p); }
//@}
private:
//! Variant storing one proxy to which this class is configured for
Proxy proxy;
//! Charm++ proxy to data and code common to all discretizations
CProxy_Discretization discproxy;
//! Charm++ proxy to flux-corrected transport (FCT) driver class
CProxy_DistFCT fctproxy;
//! Charm++ proxy to ALE class
CProxy_ALE aleproxy;
//! Charm++ proxy to conjugate gradients linear solver class
tk::CProxy_ConjugateGradients conjugategradientsproxy;
//! Charm++ array options for binding chares
CkArrayOptions bound;
//! Function dereferencing operator[] of chare proxy inside variant
//! \param[in] x Chare array element index
//! \return Chare array element proxy as a variant, defined by ProxyElem
//! \details The returning element proxy is a variant, depending on the
//! input proxy.
ProxyElem element( const CkArrayIndex1D& x ) const {
return std::visit( [&]( const auto& p ){
return static_cast< ProxyElem >( p[x] ); }, proxy );
}
};
} // inciter::
#endif // Scheme_h
|