Advanced build options

Contents

This page discusses various advanced Quinoa build options, including requirements, optional libraries, defaults, and build system options. See Quick build for the default build.

Minimum requirements

  • Fortran compiler
  • C and C++ compilers, the latter implementing the C++17 standard
  • Message Passing Interface library, e.g., OpenMPI

Routinely tested compilers

  • GNU
  • Clang

Build using the defaults

Quinoa is built in two stages:

  1. Build third-party libraries (TPLs)
  2. Build Quinoa

Both stages consist of the following two steps

  • Use cmake to configure the build
  • Perform the build

The first stage may be optional if all TPLs are found on the system or are pre-installed and/or configured, e.g., via environment modules.

The easiest way to build Quinoa is to use the defaults.

Install the prerequisites (instructions for a Debian-based Linux):

   sudo apt-get install cmake gfortran gcc g++ openmpi-bin libopenmpi-dev

Clone the third-party libraries (TPLs):

   git clone --recurse-submodules https://github.com/quinoacomputing/quinoa-tpl.git

Build the TPLs in an empty directory:

   cd quinoa-tpl
   mkdir build
   cd build
   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 ..
   make -sj$(grep -c processor /proc/cpuinfo)

By default, this will install all TPLs in quinoa-tpl/install/<compiler>-<arch>/, e.g., quinoa-tpl/install/gnu-x86_64/. Next, clone Quinoa in a directory "adjacent" to quinoa-tpl/

   cd ../..
   git clone https://github.com/quinoacomputing/quinoa.git

Now, build the code, in a new empty build directory.:

   cd quinoa
   mkdir build
   cd build
   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc ../src
   make -sj$(grep -c processor /proc/cpuinfo)

If the TPLs have been built in a non-adjacent directory, we tell that location to cmake using the cmake variable TPL_DIR in the above cmake command (For example: -DTPL_DIR=<path-to-quinoa-tpl>/install/gnu-x86_64/). Once the code is built, run the unit-, and regression tests in the build directory:

   cd quinoa/build
   ./charmrun +p$(grep -c processor /proc/cpuinfo) Main/unittest -v -q
   ctest -j$(grep -c processor /proc/cpuinfo) -LE extreme

This will run all tests, except those that would take a long time. If all of the above steps finished without error, you have successfully built Quinoa. You can run the executables from build/Main or install them with

    make install

The default install directory is /usr/local, which usually needs root permissions. You can change that by configuring a different path in the CMAKE_INSTALL_PREFIX cmake variable.

Build specific executables only

The default build process, described in section Build using the defaults, does a recursive clone of most TPLs and builds all executables. If you only care about a particular tool, e.g., Inciter (Compressible flow solver), you can

  1. Clone only those TPLs that are required for the given tool (instead of cloning them all), e.g.,

     git clone https://github.com/quinoacomputing/quinoa-tpl.git && cd quinoa-tpl
 git submodule init
 git -c submodule."src/tut".update=none -c submodule."src/backward-cpp".update=none -c submodule."src/doxygen".update=none -c submodule."src/m.css".update=none submodule update --init --recursive --progress

  2. Build only those TPLs that are required for the given executable by setting the <executable>_ONLY cmake variable to true, e.g.,

     cd quinoa-tpl && mkdir build && cd build
 cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DINCITER_ONLY=true ..
 make -sj$(grep -c processor /proc/cpuinfo)

    This will build only the TPLs required for Inciter (Compressible flow solver). The rest of the build process is the same as in Build using the defaults : running cmake configuring Quinoa will enable only those build targets for which it finds the required TPLs. Such *_ONLY cmake variables are defined for all executable targets in <quinoa-tpl>/CMakeLists.txt, in the section starting with the comment line

       # Set which libraries are built/searched by default

    Here are the git submodule update commands that pull in only the TPLs required for the given tool for each executable:

Inciter (Compressible flow solver)

   git -c submodule."src/tut".update=none -c submodule."src/backward-cpp".update=none -c submodule."src/doxygen".update=none -c submodule."src/m.css".update=none -c submodule."src/sol2".update=none submodule update --init --recursive

UnitTest

   git -c submodule."src/numdiff".update=none -c submodule."src/backward-cpp".update=none -c submodule."src/doxygen".update=none -c submodule."src/m.css".update=none -c submodule."src/sol2".update=none submodule update --init --recursive

MeshConv (Mesh format converter)

   git -c submodule."src/lapack".update=none -c submodule."src/h5part".update=none -c submodule."src/tut".update=none -c submodule."src/backward-cpp".update=none -c submodule."src/doxygen".update=none -c submodule."src/m.css".update=none -c submodule."src/sol2".update=none submodule update --init --recursive

Using system-wide packages

The default build process, described in section Build using the defaults, attempts to find (and if not found, build) all required and most optional packages that are enabled by default. See the section on Optional third-party libraries for what packages are enabled by default. On some platforms, it is advantageous (and could be faster) to use the packages provided by the system.

On a Debian-based Linux you can pre-install the packages given in

   quinoa/tools/docker/Dockerfile.quinoa-build-debian

On Alpine Linux you can pre-install the packages given in

   quinoa/tools/docker/Dockerfile.quinoa-build-alpine

Build on Mac OS

On Mac OS we do not recommend using the system-wide compilers. Instead, we use spack to install compilers and OpenMPI:

   spack install cmake libtool autoconf automake
   spack install environment-modules cmake gcc openmpi%gcc hdf5%gcc+hl

The above will use install gcc and OpenMPI on top of it. You will have to load the environment modules as, e.g.,

   module load gcc-9.2.0-clang-10.0.0-apple-vcpkolh openmpi-3.1.4-gcc-9.2.0-my2rkcv hdf5-1.10.5-gcc-9.2.0-prdkirn

Then follow Build using the defaults.

Build using Clang or Intel compilers

The default build process on Linux, described in section Build using the defaults, uses the system-wide gnu compiler suite. You can also use the clang or the Intel compilers. We do this by either building OpenMPI using clang or Intel or by using the machine-specific MPI wrappers pointing to the compiler and then use the build process given in section Build using the defaults.

Build using Charm++'s randomized message queues

For serious development work, especially involving asynchronous parallel programming, we also build and test with Charm++ using randomized message queues. This enables more thorough testing of the available space for scheduling messages and thus more heavily testing the asynchronous logic. We do this by configuring the TPL build as

   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCHARM_EXTRA_ARGS="--enable-error-checking;--with-prio-type=int;--enable-randomized-msgq;--suffix;randq-debug" <path-to-TPL-CMakeLists.txt>

Build using Charm++'s SMP mode

For better performance computing large problems using many threads, Quinoa can also be used using Charm++'s symmetric multi-processing (SMP) mode. This can be done by configuring the TPL build as

   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCHARM_EXTRA_ARGS="smp" <path-to-TPL-CMakeLists.txt>

You can run the unit-, and regression tests in SMP mode as, e.g.,

   ./charmrun +p 34 --bind-to none Main/unittest -v -q +ppn 17
   ctest -j 36 --output-on-failure -LE extreme

The above will use 36 PEs, distributed into two logical (compute) nodes. See also the Charm++ manual on how to run in SMP mode.

Build with Charm++'s Projections enabled

To use Charm++'s performance analysis tool, Projections, you can build Charm++ with

   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCHARM_EXTRA_ARGS="--enable-tracing;--enable-commthread-tracing" <path-to-TPL-CMakeLists.txt>

Then configure the Quinoa build with the extra link arguments:

   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DEXTRA_LINK_ARGS="-tracemode projections" <path-to-Quinoa-src-CMakeLists.txt>

See also the Charm++ Projections manual.

Using ninja instead of make

Similar to other CMake/C++ projects, Quinoa can also be built with ninja instead of make, by configuring the Quinoa build with

   cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -GNinja <path-to-Quinoa-src-CMakeLists.txt>

Optional third-party libraries

The default build process, described in section Build using the defaults, will attempt to build a set of default TPLs. Not all TPLs are required: some are optional, some must be specifically enabled if needed. Then Quinoa can be configured to build whatever it can based on the libraries it can find. See also the section on Build specific executables only.

All libraries, required or optional, are listed on page Libraries.

Build in Debug mode

Similar to other cmake projects, Quinoa can be configured for Debug, Release, or other types of builds cmake supports. As usual, this is done by setting the CMAKE_BUILD_TYPE variable, e.g., -DCMAKE_BUILD_TYPE=Debug. The default is Release.

Disabling packages

Sometimes it may be necessary to disable searching for packages, e.g., due to broken or unsuitable system-wide installs, etc. This can be done by configuring the TPL build by passing to cmake, e.g.,

   -DCMAKE_DISABLE_FIND_PACKAGE_Boost=true

Configuring the C++ standard library

libc++ or libstdc++ can be configured by the cmake variable STDLIBCPP, as

   -DSTDLIBCPP=libc++

or

   -DSTDLIBCPP=libstdc++

Build system features, requirements, and defaults

General requirements for configuring the TPL as well as Quinoa builds:

  • In-source builds are not allowed.
  • The default build type for both TPLs and Quinoa is Release.
  • Shared cmake code is in https://github.com/quinoacomputing/cmake-modules.git, pulled in as a git submodule into the TPL build and as a git subtree into the Quinoa build. See also the page on Modules.
  • The TPL build requires a Fortran, C, and a C++ compiler.
  • The Quinoa build requires only a C and a C++ compiler.
  • If unspecified, the default install directory, configured by the cmake variable CMAKE_INSTALL_PREFIX, for the TPL build is <root>/install/<compiler>-<architecture>/, where <root> is the root of the TPL repository clone.
  • If unspecified, the default TPL directory, configured by the cmake variable TPL_DIR, for the Quinoa build is <root>/../quinoa-tpl/install/<compiler>-architecture>/, where <root> is the root of the Quinoa repository clone; i.e. a directory "adjacent" to the Quinoa clone.

  • For the list of TPLs searched (and if not found, built) by default, see <quinoa-tpl>/CMakeLists.txt, section starting with the comment line

     # Set which libraries are built/searched by default
  • MPI is required and Charm++ is built on top of MPI.
  • The default C++ standard library is libstdc++ for gnu compilers, and libc++ for clang and Intel.

Helpful docker image configurations

You can get more examples on ways we build Quinoa by browsing the following docker images:

Main cmake files

The most important (top-level) files responsible for configuring the build system are

If you have questions not answered here, consult the above.

Feedback

If you would like to improve this page, contact us using the links in the footer below or submit a pull request on github.