Specification and software

Hardware specification

Compute nodes

The cluster consists of the following hardware:

  • 90 standard compute nodes, each containing 168 cores (two 84-core AMD EPYC 9634 CPUs), 384GB of memory and a local 1.7TB scratch disk.
    SLURM partitions: small-short, small-long, large-short and large-long.
  • 3 GPU nodes each containing eight Nvidia L40S 48GB GPUs. Otherwise similar to standard compute nodes.
    SLURM partition: gpu.L40S
  • 3 GPU nodes containing four Nvidia A100 80GB GPUs (and 128 CPU cores and 1TB memory).
  • SLURM partition: gpu.A100
  • 1 GPU node containing two A30 GPUs, 32 CPU cores and 256GB memory.
    SLURM partition: gpu.A30
  • 2 high-memory nodes, each containing 2TB memory. Otherwise similar to standard compute node.
    Slurm partition: bigmem
  • 1 very-high-memory node, containing 4TB memory. Otherwise similar to standard compute node.
    Slurm partition: vbigmem

Storage

2PB of shared disk are available as home and scratch space via the parallel GPFS file system. In addition, there is 100TB fast flash storage for I/O dependent applications (also GPFS). The latter is used by default for newly created files in the /sharedscratch filesystem. If not used for a year, it will be moved to disk. This process is transparent: the data can still be accessed in the same way. Data in /home is always on disk.

Network

All components are linked via HDR100 Infiniband (HDR200 for the GPU nodes) and Gigabit Ethernet.

Available software

A variety of application software is installed on hypatia. The following is an incomplete list. If you require a package that is currently not installed, you can either request it to be installed, or install or compile your own software.

It is your responsibility to ensure that your use of hypatia and any software installed on it is legal and covered by the license of this software, in addition to being ethical and for the purposes of your research.

General Computing

MATLAB

MATLAB scripts can be run as batch jobs. Please do not start it interactively on the login node!

License: St Andrews site license
Installed version: 25.2
Example batch job: /software/examples/job-matlab

R

Install your own version in CONDA. Batch jobs can be run via Rscript (see example job).

License: open source
Example batch job: /software/examples/job-R

Materials Science

CASTEP

CASTEP is a plane-wave DFT code to calculate material properties, including NMR parameters.

License: Free for academic use, but requires license at research group level. Please "buy" a license from STFC.
Installed versions: 24.1 and 20.11
Example batch job: /usr/local/examples/job-castep

Quantum Espresso

Quantum Espresso is another plane-wave code for periodic materials.

License: open source
Installed version: 7.4
Example batch job: /software/examples/job-qe

CP2K

CP2K is a code for periodic calculations with DFT using atom-centred Gaussian-type basis sets, as well as force fields and QM/MM. It also implements a periodic version of Grimme's semi-empirical GFN-1 xtb method.

License: open source
Installed version: 2025.1
Example batch job: /software/examples/job-qe

CRYSTAL

CRYSTAL is a code for periodic calculations with DFT and MP2 using a atom-centred Gaussian-type basis sets.

License: free for UK researchers
Installed version: 23.1.0.1
Example batch job: /software/examples/job-crystal

Molecular Quantum Chemistry

Gaussian

Gaussian is a quantum chemistry code for molecular systems. It also includes molecular mechanics methods and some functionality for periodic systems.

License: St Andrews site license for academic use
Installed version: Gaussian 16, Revision C.01
Example batch job: /software/examples/job-g16

ORCA

ORCA is another quantum chemistry code for molecular systems. It offers a wide variety of methods using particularly efficient RI (resolution of the identity) approximations.

License: Free for academic use, but requires registration at the ORCA Forum.
Installed versions: 6.1.0 and 5.0.4
Example batch job: /software/examples/job-orca

PSI4

Besides the usual DFT and post-HF methods, people find it useful for symmetry-adapted perturbation theory. More details at the program's home page.

License: open source
Installed version: 1.9.1
Example batch job: /software/examples/job-psi4

xtb

xtb is a program implementing the semi-empirical extended tight binding methods GFN-1 and GFN-2, as well as the GFN-FF force field. It can be used directly or via ORCA (see above).

License: open source
Installed version: 6.7.1
Example batch job: /software/examples/job-xtb (or job-orca if used as a method in ORCA)

Classical Molecular Dynamics

LAMMPS

LAMMPS is a classical molecular dynamics code with a focus on materials modeling.

License: open source
Installed version: 4 Feb 2025
Example batch job: /software/examples/job-lammps

GULP

GULP is a lattice dynamics and MD code in 0, 1, 2 and 3 dimensions.

License: open source
Installed version: 6.3.1
Example batch job: /software/examples/job-gulp