Les Modules PlaFRIM

Table of Contents

Ce fichier est disponible à https://www.labri.fr/perso/furmento/bidouille/modules/

\newpage

1 Introduction

  • Dynamic modification of a user's environment via modulefiles.
  • Each modulefile contains the information needed to use the specific library/compiler/… by altering or setting shell environment variables such as \texttt{PATH}, \texttt{MANPATH}, etc.
  • Different versions of applications.
  • Metamodules can be used to load an entire suite of different applications.

\newpage

\newpage

2 Users modules

  • All users can install modules, one only needs to be added to the Unix group \texttt{plafrim-dev} (ticket to \texttt{plafrim-support@inria.fr}) …
  • … and install modules in \texttt{/cm/shared/dev/modules}
  • But there is some rules to follow …

\newpage

3 Architecture

  • For modules which can run on all nodes, the directory \texttt{generic} should be used, with the sub-directories \texttt{apps} and \texttt{modulesfiles}.
  • For modules aimed to specific architectures, the following directories should be used
    • \texttt{intel/haswell} for the nodes \texttt{miriel} and \texttt{sirocco[01-05]}
    • \texttt{intel/broadwell} for the nodes \texttt{sirocco[07-13]}
    • \texttt{intel/skylake} for the nodes \texttt{bora}
    • \texttt{intel/knightslanding} for the nodes \texttt{kona}
  • When connecting to a node, the environnment variable \texttt{MODULEPATH} contains the directory \texttt{generic} and the node-specific directory \texttt{manufacturer}/\texttt{chip}

\newpage

4 Categories

  • In order to increase and to ease the use of the modules on the platform, modules are grouped within categories. Each module belongs to a specific category, which can be for example \texttt{trace} or \texttt{statistics}.
  • The module naming policy is as follows:

    \textit{category}/\textit{module}/\textit{option}/\textit{version}

  • the number of options being between 0 and as many as you want.

    \texttt{partitioning/scotch/int32/6.0.4}

    \texttt{partitioning/scotch/int64/6.0.4}

\newpage

5 File System

  • Modules files go in \texttt{/cm/shared/dev/modules/XXX/modulefiles} by following the architecture and the naming policies.
    Generic module \texttt{fxt} in the category \texttt{trace} with version \texttt{0.3.9} and without any specific option
    \(\rightarrow\) \texttt{/cm/shared/dev/modules/generic/modulefiles/trace/fxt/0.3.9}
  • Installation module files go in \texttt{/cm/shared/dev/modules/XXX/apps} with the same architecture and naming policies.
    \(\rightarrow\) \texttt{/cm/shared/dev/modules/generic/apps/trace/fxt/0.3.9/}
  • Make sure all the files can be read by anyone on the platform.
  • Write a real description in whatis, the command \texttt{module show/whatis xxx} should say something more meaningful than \texttt{loads the xxx environment}

\newpage

6 How to create a module

  • Let's assume we install a \texttt{generic} module
  • Install your software in the appropriate sub-directory in \texttt{/cm/shared/dev/modules/generic/apps}
  • For example in the directory \texttt{/cm/shared/dev/modules/generic/apps/tools/boost/1.71.0/}
  • The module file is \texttt{/cm/shared/dev/modules/generic/modulefiles/tools/boost/1.71.0}
proc ModulesHelp { } {
  puts stderr "\tAdds boost 1.71.0 to your environment variables,"
}

module-whatis "adds boost 1.71.0 debug tool to your environment variables"

set             version         1.71.0
set             prefix          /cm/shared/dev/modules/generic/apps/tools/boost/
set             root            $prefix/$version

#%Module

#path added in the beginning
prepend-path   CPATH  $root/include
prepend-path   LIBRARY_PATH  $root/lib
prepend-path   LD_LIBRARY_PATH  $root/lib
setenv   BOOST_ROOT  $root
  • Set the correct permissions
$ module load tools/module_cat
$ module_perm my_directory

\newpage

  • If your module depends on other modules \texttt{/cm/shared/dev/modules/generic/modulefiles/perftools/simgrid/3.24}
    • specify the dependencies

      prereq compiler/gcc
      prereq tools/boost/1.71.0
      
    • and use their informations

      set     prefix          /cm/shared/dev/modules/generic/apps/perftools/simgrid/$version/install/gcc_$env(GCC_VER)
      
    • The module can have different versions
      • \texttt{/cm/shared/dev/modules/generic/apps/perftools/simgrid/3.24/install/gcc\_8.2.0/}
      • \texttt{/cm/shared/dev/modules/generic/apps/perftools/simgrid/3.24/install/gcc\_9.2.0/}
  • To avoid loading 2 versions of the same module

    conflict runtime/starpu
    
  • When a module depends on the versions of other modules

    set     prefix          /cm/shared/dev/modules/generic/apps/perftools/simgrid/$version/install/gcc_$env(GCC_VER)
    if {![file exists $prefix]} {
        puts stderr "\t[module-info name] Load Error: $prefix does not exist"
        break
        exit 1
    }
    

\newpage

  • What are the useful variables?

    • Path to development headers (for compiling)
    CPATH
    FPATH
    INCLUDE
    C_INCLUDE_PATH
    CPLUS_INCLUDE_PATH
    OBJC_INCLUDE_PATH
    
    • Path to libraries (for linking)
    LIBRARY_PATH
    
    • Path to tools and libraries (for running)
    PATH
    LD_LIBRARY_PATH
    
    • pkg-config to simplifying build systems (point to directories with .pc files)
    prepend-path    PKG_CONFIG_PATH     $prefix/lib/pkgconfig
    
    • Manpages
    append-path     MANPATH         $man_path
    append-path     MANPATH         $man_path/man1
    append-path     MANPATH         $man_path/man3
    append-path     MANPATH         $man_path/man7
    
    • Some modules define specific variables, likely because one random project ever decided to use that
    setenv            CUDA_INSTALL_PATH   $root
    setenv            CUDA_PATH           $root
    setenv            CUDA_SDK            $root
    prepend-path      CUDA_INC_PATH       $root/include
    
    setenv          HWLOC_HOME          $prefix
    
    setenv         MPI_HOME          $prefix
    setenv         MPI_RUN           $prefix/bin/mpirun
    setenv         MPI_NAME          $name
    setenv         MPI_VER           $version
    

\newpage

  • Which version gets selected by default during load?
    • The default is in (reverse) alphabetical order

      $ module av formal/sage
      
      ------------------ /cm/shared/dev/modules/generic/modulefiles ------------------
      formal/sage/7.0 formal/sage/8.9 formal/sage/9.0
      $ module av 2>&1 | grep sage
      formal/sage/7.0
      formal/sage/8.9
      formal/sage/9.0
      $ module load formal/sage
      $ module list
      Currently Loaded Modulefiles:
        1) formal/sage/9.0
      
    • Another default version may be enforced. Useful if your last beta release isn't stable or backward compatible but still needed for some hardcode users.

      $ cat /cm/shared/dev/modules/generic/modulefiles/hardware/hwloc/.version
      #%Module1.0#
      set ModulesVersion "2.1.0"
      

\newpage

7 More on environment dependent modules

$ module_grep starpu
runtime/starpu/1.3.2/mpi
runtime/starpu/1.3.2/mpi-fxt
runtime/starpu/1.3.3/mpi
runtime/starpu/1.3.3/mpi-cuda
runtime/starpu/1.3.3/mpi-cuda-fxt
runtime/starpu/1.3.3/mpi-fxt
  • StarPU has different module files which differ in the \texttt{prereq} commands and the prefix setting.

    > prereq compiler/cuda/10.1
    > prereq trace/fxt/0.3.9
    < set     prefix          /cm/shared/dev/modules/generic/apps/runtime/starpu/1.3.3/gcc@9.2.0-hwloc@2.1.0-openmpi@4.0.2
    > set     prefix          /cm/shared/dev/modules/generic/apps/runtime/starpu/1.3.3/gcc@8.2.0-hwloc@2.1.0-openmpi@4.0.1-cuda@10.1
    > set     prefix          /cm/shared/dev/modules/generic/apps/runtime/starpu/1.3.3/gcc@8.2.0-hwloc@2.1.0-openmpi@4.0.1-cuda@10.1-fxt@0.3.9
    

\newpage

  • Create a module file with all the cases
if {![ is-loaded compiler/gcc ]} {
   module load compiler/gcc
}
if {![ is-loaded hardware/hwloc ]} {
   module load hardware/hwloc/2.1.0
}

conflict runtime/starpu

set cuda ""
set fxt ""
if {[ is-loaded compiler/cuda/10.1 ]} {
  set cuda -cuda@10.1
}
if {[ is-loaded trace/fxt/0.3.9 ]} {
  set fxt -fxt@0.3.9
}

# for Tcl script use only
set     name            starpu
set     version         1.3.3

set     prefix          /cm/shared/dev/modules/generic/apps/runtime/$name/$version/gcc@$env(GCC_VER)-hwloc@2.1.0-openmpi@4.0.1${cuda}${fxt}

\newpage

  • Different environments lead to a different version of StarPU to be used:
    • 1st case

      $ module purge
      $ module load runtime/starpu/42
      $ module list
      Currently Loaded Modulefiles:
        1) compiler/gcc/9.2.0     2) hardware/hwloc/2.1.0   3) runtime/starpu/42
      $ module show runtime/starpu/42
      ...
      setenv		 STARPU_DIR /cm/shared/dev/modules/generic/apps/runtime/starpu/1.3.3/gcc@9.2.0-hwloc@2.1.0-openmpi@4.0.1 
      
    • 2nd case

      $ module purge && module load compiler/gcc/8.2.0 compiler/cuda/10.1 runtime/starpu/42
      $ module show runtime/starpu/42
      ...
      setenv		 STARPU_DIR /cm/shared/dev/modules/generic/apps/runtime/starpu/1.3.3/gcc@8.2.0-hwloc@2.1.0-openmpi@4.0.1-cuda@10.1 
      
    • 3rd case

      $ module purge && module load compiler/gcc/8.2.0 compiler/cuda/10.1 trace/fxt runtime/starpu/42
      $ module show runtime/starpu/42
      ...
      setenv		 STARPU_DIR /cm/shared/dev/modules/generic/apps/runtime/starpu/1.3.3/gcc@8.2.0-hwloc@2.1.0-openmpi@4.0.1-cuda@10.1-fxt@0.3.9 
      

8 More modules commands

9 \texttt{tools/module\_cat}

  • \texttt{module load tools/module\_cat}
  • You can then list all the modules whose name contains a specific string.
$ module_grep scotch
partitioning/scotch/int32/5.1.12
partitioning/scotch/int32/6.0.9
partitioning/scotch/int64/5.1.12
partitioning/scotch/int64/6.0.9
partitioning/scotch_with_esmumps/6.0.7
  • List all the categories
$ module_list
--------------------------------------------------------------------------------------------- /cm/shared/modules/generic/modulefiles ---------------------------------------------------------------------------------------------
build		compiler	language	mpi		tools
--------------------------------------------------------------------------------------------- /cm/shared/modules/intel/haswell/modulefiles ---------------------------------------------------------------------------------------------
compiler	linalg
--------------------------------------------------------------------------------------------- /cm/shared/dev/modules/generic/modulefiles ---------------------------------------------------------------------------------------------
benchmark	dnn		gis		partitioning	tools
bigdata		editor		hardware	perftools	trace
build		fft		io		physics		visu
compiler	formal		language	runtime
dataencoding	genome		linalg		scm
  • List the content of a given category
$ module_list hardware
--------------------------------------------------------------------------------------------- /cm/shared/dev/modules/generic/modulefiles ---------------------------------------------------------------------------------------------
hardware/hwloc		hardware/libpciaccess
  • Restrict MODULEPATH to a specific set of categories
$ module_restrict mpi compiler && echo $MODULEPATH
:/cm/shared/modules/generic/modulefiles/compiler:/cm/shared/modules/generic/modulefiles/mpi:/cm/shared/modules/intel/haswell/modulefiles/compiler:/cm/shared/dev/modules/generic/modulefiles/compiler
  • Add some categories
$ module_add tools && echo $MODULEPATH
[module_add] adding /cm/shared/modules/generic/modulefiles/tools
[module_add] adding /cm/shared/dev/modules/generic/modulefiles/tools
/cm/shared/dev/modules/generic/modulefiles/tools:/cm/shared/modules/generic/modulefiles/tools:::/cm/shared/modules/generic/modulefiles/compiler:/cm/shared/modules/generic/modulefiles/mpi:/cm/shared/modules/intel/haswell/modulefiles/compiler:/cm/shared/dev/modules/generic/modulefiles/compiler
  • Remove some categories
$ module_rm mpi compiler && echo $MODULEPATH
/cm/shared/dev/modules/generic/modulefiles/tools:/cm/shared/modules/generic/modulefiles/tools:
  • Help for the different options are displayed by calling
$ module_help
module_<command> with <command> being:
      help                                 - display help
      list [<category>]                    - list categories (from the top level by default, or the given category)
      init                                 - list categories from the initial setup
      reset                                - reset initial MODULEPATH
      add <category1> ... <categoryn>      - add given category/ies to MODULEPATH
      rm <category1> ... <categoryn>       - remove given category/ies from MODULEPATH
      restrict <category1> ... <categoryn> - only keep given category/ies in MODULEPATH

Date: 14 février 2020

Author: Nathalie Furmento et Brice Goglin

Created: 2020-02-21 ven. 09:42

Validate