Spack

Spack is a package manager which is very useful to install scientific software.

Preparing a Linux system

We will install the gcc and gfortran compilers from the system package manager. Dont install anything else since they will be installed using Spack. There are some instructions for OpenSUSE and Debian. Since Spack installs its own MPI library, it is best to uninstall any MPI you may have installed from your system package manager, since having multiple MPI around can cause problems.

Preparing a Mac system

Install Spack

We will install Spack into SPACK_ROOT directory. Add following lines to your .bashrc or .profile file 

export SPACK_ROOT=/path/to/spack
. ${SPACK_ROOT}/share/spack/setup-env.sh

Now get Spack 

git clone https://github.com/LLNL/spack $SPACK_ROOT

Here are some useful Spack commands 

spack info gcc  -- show information for package gcc
spack find      -- shows installed packages
spack compilers -- show available compilers

Setup compilers by running

spack compiler find

which creates the file compilers.yaml inside $HOME/.spack/darwin directory. Edit it if needed.

On Mac, use clang from Xcode and gfortran from Homebrew. My file $HOME/.spack/darwin/compilers.yaml looks like this 

compilers:
- compiler:
    environment: {}
    extra_rpaths: []
    flags: {}
    modules: []
    operating_system: highsierra
    paths:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: /usr/local/bin/gfortran
      fc: /usr/local/bin/gfortran
    spec: clang@9.0.0-apple
    target: x86_64

On Linux, use the gcc and gfortran from your system.

Set some package variants in $HOME/.spack/darwin/packages.yaml file like this

packages:
   hdf5:
      variants: +fortran +cxx +hl
   petsc:
      variants: +mumps
   dealii:
      variants: ~python
   trilinos:
      variants: +rol
   all:
      providers:
         mpi: [mpich]

By default, Spack uses openmpi but you can select mpich as above. If you want to use openmpi, then remove the last few lines in packages.yaml file.

Install softwares

I install deal.II and its dependencies which provides all the softwares that I need for my work. Since I like to compile deal.II myself, I will install only dependencies. But first check that all variants you need are correct by running 

spack spec dealii@develop

If this is ok, then we install 

spack install -j4 --only dependencies dealii@develop

The flag -j4 means that four processes will be used while compiling; increase this if you have more cores on your computer. If you primarily use Petsc and Metis, and dont need other stuff, then do 

spack install -j4 petsc

Now create a Spack view by running the script spack_view.sh which will install symlinks into SPACK_VIEW directory. 

export SPACK_VIEW=/path/to/spack/view # Add to .bashrc or .profile file
sh /path/to/spack_view.sh

If the SPACK_VIEW directory is in not in your home directory, then set ownership 

sudo mkdir $SPACK_VIEW
sudo chown praveen:users $SPACK_VIEW

and then run the spack_view.sh script. This is useful on a Linux computer where I want to install the software for all users.

Here is a sample spack_view.sh script which you can edit for your needs. If you use mpich, then change openmpi to mpich. 

#!/bin/bash

# Set this to your actual spack view dir
SPACK_VIEW=/opt/spack

COMMAND="spack view -v -d no"
ACTION=symlink

dolink()
{
   $COMMAND $ACTION $SPACK_VIEW $1
   # Delete some useless stuff
   rm -f $SPACK_VIEW/include/index.html
   rm -f $SPACK_VIEW/share/info/dir
}

if [ ! -d "$SPACK_VIEW" ]; then
   echo "Directory $SPACK_VIEW does not exist"
   exit
fi

# Delete all existing stuff
rm -rf $SPACK_VIEW/*
rm -rf $SPACK_VIEW/.spack
rm -rf $SPACK_VIEW/.spack-empty
rm -rf $SPACK_VIEW/.nagged

dolink adol-c
dolink arpack-ng
dolink assimp
dolink boost
dolink bzip2
dolink cmake
dolink gmsh
dolink gsl
dolink hdf5
dolink metis
dolink muparser
dolink mumps
dolink nanoflann
dolink netcdf
dolink netcdf-cxx
dolink netlib-scalapack
dolink oce
dolink openblas
dolink openmpi
dolink p4est
dolink petsc
dolink slepc
dolink suite-sparse
dolink sundials
dolink tbb
dolink trilinos
dolink zlib

You can set some variables in your .bashrc or .profile file 

export PETSC_DIR=$SPACK_VIEW
export SLEPC_DIR=$SPACK_VIEW
export METIS_DIR=$SPACK_VIEW
export HDF5_DIR=$SPACK_VIEW
export PATH=$SPACK_VIEW/bin:$PATH

Upgrading

When I want to upgrade, I usually reinstall everything from scratch. 

cd $SPACK_ROOT
git pull

Update your compilers.yaml and packages.yaml files if needed. Verify that you get all package variants that you need. Then continue 

spack uninstall --all
spack install -j4 --only dependencies dealii@develop

Now we create a new spack view 

sh /path/to/spack_view.sh