Spack

Spack is a package manager which is very useful to install scientific software.

Preparing a Linux system

We will install the gcc and gfortran compilers from the system package manager. Dont install anything else since they will be installed using Spack. There are some instructions for OpenSUSE and Debian. Since Spack installs its own MPI library, it is best to uninstall any MPI you may have installed from your system package manager, since having multiple MPI around can cause problems.

Preparing a Mac system

Install Spack

We will install Spack into SPACK_ROOT directory. Add following lines to your .bashrc or .profile file 

export SPACK_ROOT=/path/to/spack
. ${SPACK_ROOT}/share/spack/setup-env.sh

Now get Spack 

git clone https://github.com/LLNL/spack $SPACK_ROOT

You can add following line to .bashrc/.profile to have full spack functionality available 

. ${SPACK_ROOT}/share/spack/setup-env.sh

Here are some useful Spack commands 

spack info gcc  -- show information for package gcc
spack find      -- shows installed packages
spack compilers -- show available compilers

Setup compilers by running

spack compiler find

which creates the file compilers.yaml inside $HOME/.spack directory. Edit it if needed.

On Mac, use clang from Xcode and gfortran from Homebrew. My file $HOME/.spack/darwin/compilers.yaml looks like this 

compilers:
- compiler:
    environment: {}
    extra_rpaths: []
    flags: {}
    modules: []
    operating_system: highsierra
    paths:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: /usr/local/bin/gfortran
      fc: /usr/local/bin/gfortran
    spec: clang@9.0.0-apple
    target: x86_64

On Linux, use the gcc and gfortran from your system.

Set some package variants in packages.yaml file like this

packages:
   hdf5:
      variants: +fortran
   petsc:
      variants: +mumps
   dealii:
      variants: ~python
   all:
      providers:
         mpi: [mpich]

By default, Spack uses openmpi but you can select mpich as above. If you want to use openmpi, then remove the last few lines in packages.yaml file.

Install softwares

I install deal.II and its dependencies which provides all the softwares that I need for my work. On Mac, install the modules program 

spack install -j4 environment-modules

On Linux you can install this from your system package manager or using Spack as above. Now install deal.II; since I like to compile deal.II myself, I will install only dependencies. But first check that all variants you need are correct by running 

spack spec dealii@develop

If this is ok, then we install 

spack install -j4 --only dependencies dealii@develop

The flag -j4 means that four processes will be used while compiling; increase this if you have more cores on your computer. If you primarily use Petsc and Metis, and dont need other stuff, then do 

spack install -j4 petsc

Now create a Spack view by running the script spack_view.sh which will install symlinks into SPACK_VIEW directory. 

export SPACK_VIEW=/path/to/spack/view # Add to .bashrc or .profile file
sh /path/to/spack_view.sh

If the SPACK_VIEW directory is in not in your home directory, then set ownership 

sudo mkdir $SPACK_VIEW
sudo chown praveen:users $SPACK_VIEW

and then run the spack_view.sh script. This is useful on a Linux computer where I want to install the software for all users.

Here is a sample spack_view.sh script which you can edit for your needs. If you use mpich, then change openmpi to mpich. 

COMMAND="spack view -v -d no"
ACTION=symlink

if [ ! -d "$SPACK_VIEW" ]; then
   echo "Directory $SPACK_VIEW does not exist"
   exit
fi

# Delete all existing stuff
rm -rf $SPACK_VIEW/*
rm -rf $SPACK_VIEW/.spack

$COMMAND $ACTION $SPACK_VIEW arpack-ng
$COMMAND $ACTION $SPACK_VIEW boost
$COMMAND $ACTION $SPACK_VIEW bzip2
$COMMAND $ACTION $SPACK_VIEW cmake
$COMMAND $ACTION $SPACK_VIEW gsl
$COMMAND $ACTION $SPACK_VIEW hdf5
$COMMAND $ACTION $SPACK_VIEW metis
$COMMAND $ACTION $SPACK_VIEW muparser
$COMMAND $ACTION $SPACK_VIEW mumps
$COMMAND $ACTION $SPACK_VIEW netcdf
$COMMAND $ACTION $SPACK_VIEW netcdf-cxx
$COMMAND $ACTION $SPACK_VIEW oce
$COMMAND $ACTION $SPACK_VIEW openblas
$COMMAND $ACTION $SPACK_VIEW openmpi
$COMMAND $ACTION $SPACK_VIEW p4est
$COMMAND $ACTION $SPACK_VIEW petsc
rm $SPACK_VIEW/include/index.html
$COMMAND $ACTION $SPACK_VIEW slepc
$COMMAND $ACTION $SPACK_VIEW suite-sparse
$COMMAND $ACTION $SPACK_VIEW tbb
$COMMAND $ACTION $SPACK_VIEW trilinos
$COMMAND $ACTION $SPACK_VIEW zlib

You can set some variables in your .bashrc or .profile file 

export PETSC_DIR=$SPACK_VIEW
export SLEPC_DIR=$SPACK_VIEW
export METIS_DIR=$SPACK_VIEW
export HDF5_DIR=$SPACK_VIEW

export PATH=$SPACK_VIEW/bin:$PATH

Upgrading

When I want to upgrade, I usually reinstall everything from scratch. 

cd $SPACK_ROOT
git pull

Update your compilers.yaml and packages.yaml files if needed. Verify that you get all package variants that you need. Then continue 

spack uninstall --all
spack install -j4 --only dependencies dealii@develop

Now we create a new spack view 

sh /path/to/spack_view.sh