Memory issues in GPAW

Try to use default parameters for the calculator. Simple and often useful.

Below you find a list of suggestions that should be considered when encountering a memory problem – when a calculation does not fit an allocated memory limit.

Note1: You can use –dry-run to make memory estimation and check for parallelization over kpts, domain, and bands as well as use of symmetry.

gpaw python --dry-run=N script.py

Mind that the memory estimation with –dry-run is underestimated. https://gitlab.com/gpaw/gpaw/-/issues/614

Note2: You can use def monkey_patch_timer() to write information about memory usage into mem.* files. Call the function before the actual work is started.

from gpaw.utilities.memory import monkey_patch_timer

monkey_patch_timer()

SUBMISSION OPTIONS

  1. Try increasing the total memory or memory per tasks in the submission script, if you are sure that everything else (see below) is correct.
  2. Try increasing number of tasks (CPUs×threading) and nodes, if only you are sure that everything else (see below) is correct. Note that your calculation accesses all the nodes’ memory independent on the number of allocated tasks, but not not all memory is actually available because some is used by the OS and other running jobs. Also, increasing the number of tasks decreases parallelization efficiency and might decrease the queue priority (depending on the queuing system).

GEOMETRY

  1. Check the model geometry. Perhaps, you can make a more compact model. For example, with orthorhombic=False option.
  2. In slab calculations, use just enough vacuum. Mind that PW mode is egg-box effect free, so, with the dipole-layer correction, you can reduce the vacuum layer significantly. Just check for the energy convergence.
    https://wiki.fysik.dtu.dk/gpaw/tutorialsexercises/electrostatics/dipole_correction/dipole.htm
  3. Ensure that symmetry is used. Sometimes, the calculator uses less symmetry than there is. In that case, recheck the geometry. Remember that you can preserve symmetry during optimization. https://wiki.fysik.dtu.dk/ase/ase/constraints.html#the-fixsymmetry-class

PARALLELIZATION

In general, parallelization over domains requires less memory than parallelization over bands and k-points, but the default order of parallelization is k-points, then domains, then bands. Remember the formula kpts×domain×bands = N, where N is the number of tasks (CPUs).

  1. In most cases, the default parallelization with symmetry is most efficient in terms of memory usage.
  2. Reprioritizing parallelization over domain can reduce memory consumption, but also slow down the calculation as parallelization over k-points is usually more time-efficient.
  3. Parallelization over any type can be suppressed by setting, for example, for domains like parallel = {'domain':1}. In the LCAO mode, you should check whether parallelizing over bands, like parallel = {'bands':2}, helps with the memory issue.

CALCULATOR PARAMETERS

  1. Consider using a different mode. “With LCAO, you have fewer degrees of freedom, so memory usage is low. PW mode uses more memory and FD a lot more.” https://wiki.fysik.dtu.dk/gpaw/documentation/basic.html#manual-mode
  2. Change calculation parameters, like h, cutoff, setups (like setups={'Pt': '10'}), basis (like basis={'H': 'sz', 'C': 'dz'}), etc. Check for convergence of properties, like in this tutorial: wiki.fysik.dtu.dk/gpaw/summerschools/summerschool22/catalysis/catalysis.html#extra-material-convergence-test
  3. It is possible to reduce the memory by changing the mixer options.
    https://wiki.fysik.dtu.dk/gpaw/documentation/convergence.html