{"id":595,"date":"2022-06-23T12:10:50","date_gmt":"2022-06-23T12:10:50","guid":{"rendered":"https:\/\/doublelayer.eu\/vilab\/?p=595"},"modified":"2024-08-01T10:54:18","modified_gmt":"2024-08-01T10:54:18","slug":"installing-gpaw-with-conda","status":"publish","type":"post","link":"https:\/\/doublelayer.eu\/vilab\/2022\/06\/23\/installing-gpaw-with-conda\/","title":{"rendered":"Installing GPAW with conda"},"content":{"rendered":"\n<p>[Updated on 20.04.2022, 15.04.2023, 10.06.2023, 03.10.2023, 04.06.2024]<\/p>\n\n\n\n<p>In short, in a clean environment, everything should work with just five lines:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>wget https:\/\/repo.anaconda.com\/miniconda\/Miniconda3-latest-Linux-x86_64.sh<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>bash Miniconda3-latest-Linux-x86_64.sh<\/code><\/pre>\n\n\n\n<p>Initialize conda. If it is in the .bashch, source it. If not, source &#8220;PATHTOCONDA\/miniconda3\/etc\/profile.d\/conda.sh&#8221;.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda create --name gpaw -c conda-forge python=3.12<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>conda activate gpaw<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge openmpi ucx<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge gpaw=24.1.0=*openmpi*<\/code><\/pre>\n\n\n\n<p>For details, see the description below.<\/p>\n\n\n\n<p><strong>1.<\/strong><strong> Install conda \u2013 software and environment management system.<\/strong><\/p>\n\n\n\n<p>Here is the official instruction: <a href=\"https:\/\/docs.conda.io\/projects\/conda\/en\/latest\/user-guide\/install\/linux.html\">docs.conda.io\/projects\/conda\/en\/latest\/user-guide\/install\/linux.html<\/a><\/p>\n\n\n\n<p>On June 2024, run these:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>wget https:\/\/repo.anaconda.com\/miniconda\/Miniconda3-latest-Linux-x86_64.sh<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>bash Miniconda3-latest-Linux-x86_64.sh<\/code><\/pre>\n\n\n\n<p>If you wish to autostart conda, allow it to write to your .bashrc.<\/p>\n\n\n\n<p>P.S. Here are good intros to conda:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.earthdatascience.org\/courses\/intro-to-earth-data-science\/python-code-fundamentals\/use-python-packages\/introduction-to-python-conda-environments\/\">www.earthdatascience.org\/courses\/intro-to-earth-data-science\/python-code-fundamentals\/use-python-packages\/introduction-to-python-conda-environments\/<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/towardsdatascience.com\/introduction-to-conda-virtual-environments-eaea4ac84e28\">towardsdatascience.com\/introduction-to-conda-virtual-environments-eaea4ac84e28<\/a><\/li>\n<\/ul>\n\n\n\n<p>N.B! If the locale is not set, add it to your .bashrc export<\/p>\n\n\n\n<p><code>LC_ALL=en_US.UTF-8<\/code><\/p>\n\n\n\n<p>Without it python might give a segmentation fault (core dumped) error.<\/p>\n\n\n\n<p><strong>2. Create a conda virtual environment:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda create --name gpaw -c conda-forge python=3.12<\/code><\/pre>\n\n\n\n<p>If needed, remove the environment as:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda remove --name gpaw --all<\/code><\/pre>\n\n\n\n<p>You can check the available environments as:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda env list<\/code><\/pre>\n\n\n\n<p><strong>3. Activate the virtual environment.<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda activate gpaw<\/code><\/pre>\n\n\n\n<p><strong>4. Install gpaw:<\/strong><\/p>\n\n\n\n<p>Ensure that no interfering modules and environments are loaded.<\/p>\n\n\n\n<p>Purge modules by executing:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>module purge<\/code><\/pre>\n\n\n\n<p>To check whether some code (like mpirun) has an alternative path, try:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>which codename<\/code><\/pre>\n\n\n\n<p>or<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>codename --version<\/code><\/pre>\n\n\n\n<p>There should be no mpirun, ase, libxc, numpy, scipy, etc. Otherwise, the installation with conda will most probably fail due to conflicting paths.<\/p>\n\n\n\n<p><strong>4.1. <\/strong>It is safer to install using gpaw*.yml file from vliv\/conda directory on FEND:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda env create -f gpaw.yml<\/code><\/pre>\n\n\n\n<p>Note that there are many yml files with different versions of GPAW.<\/p>\n\n\n\n<p><strong>4.2.<\/strong> Pure installation is simple but might not work:<\/p>\n\n\n\n<p><code>conda install -c conda-forge openmpi<\/code><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge gpaw=*=*openmpi*<\/code><\/pre>\n\n\n\n<p>In 2022, there were problems with openmpi. Downgrading to version 4.1.2 helped:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge openmpi=4.1.2<\/code><\/pre>\n\n\n\n<p>You might wish to install ucx but be aware that there are many problems with it, e. g. <a href=\"https:\/\/github.com\/openucx\/ucx\/issues\/8100\">depending on mlx version<\/a>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge ucx<\/code><\/pre>\n\n\n\n<p>If you get an error about GLIBCXX, try upgrading gcc:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge gcc=12.1.0<\/code><\/pre>\n\n\n\n<p><strong>4.3.<\/strong> To quickly check the installation, run &#8220;gpaw -P 2 test&#8221; or &#8220;gpaw info&#8221;.<\/p>\n\n\n\n<p>The installation might fail. In case you succeed, save the yml file as:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda env export | grep -v \"^prefix: \" &gt; gpaw.yml<\/code><\/pre>\n\n\n\n<p>Now you can use it to install gpaw as:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda env create -f gpaw.yml<\/code><\/pre>\n\n\n\n<p>To properly test the installation install pytest and follow <a href=\"https:\/\/wiki.fysik.dtu.dk\/gpaw\/devel\/testing.html\">wiki.fysik.dtu.dk\/gpaw\/devel\/testing.html<\/a>. That might take hours.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge pytest pytest-xdist \n<\/code><\/pre>\n\n\n\n<p><strong>5. If needed, install extra packages within your specific conda environment (gpaw).<\/strong><\/p>\n\n\n\n<p>To apply D4 dispersion correction:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge dftd4 dftd4-python<\/code><\/pre>\n\n\n\n<p>To analyze trajectories:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>conda install -c conda-forge mdanalysis<\/code><code><code><\/code><\/code><\/pre>\n\n\n\n<p>To analyze electronic density (some might not work):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip install git+https:\/\/github.com\/funkymunkycool\/Cube-Toolz.git<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>pip install git+https:\/\/github.com\/theochem\/grid.git\npip install git+https:\/\/github.com\/theochem\/denspart.git<\/code><code><\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>pip install pybader<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>pip install cpmd-cube-tools<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge chargemol<\/code><\/pre>\n\n\n\n<p>To use catlearn:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip install catlearn<\/code><\/pre>\n\n\n\n<p>To work with crystal symmetries:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge spglib<\/code><\/pre>\n\n\n\n<p>Extra for visualization (matplotlib comes with ASE):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge pandas seaborn bokeh jmol<\/code><\/pre>\n\n\n\n<p>To use notebooks (you might need to install firefox as well):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>conda install -c conda-forge jupyterlab nodejs jupyter_contrib_nbextensions <\/code><\/pre>\n\n\n\n<p><strong>6. Run calculations by adding these lines to the submission script:<\/strong><\/p>\n\n\n\n<p>Note1: Check the path and change the USERNAME<\/p>\n\n\n\n<p>Note2: Turn off ucx.<\/p>\n\n\n\n<p>Note3: You may play with the number of openmp threads.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>module purge\nsource \"\/groups\/kemi\/USERNAME\/miniconda3\/etc\/profile.d\/conda.sh\"\nconda activate gpaw\nexport OMP_NUM_THREADS=1\nexport OMPI_MCA_pml=\"^ucx\"\nexport OMPI_MCA_osc=\"^ucx\"\nmpirun gpaw python script.py\n<\/code><\/pre>\n\n\n\n<p>Note4: Check an example in vliv\/conda\/sub directory.<\/p>\n\n\n\n<p><strong>7. Speeding-up calculations.<\/strong><\/p>\n\n\n\n<p>Add the &#8220;parallel&#8221; keyword to GPAW calculator:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code><\/code>parallel = {'augment_grids':True,'sl_auto':True},<\/code><\/pre>\n\n\n\n<p>For more options see <a href=\"https:\/\/wiki.fysik.dtu.dk\/gpaw\/documentation\/parallel_runs\/parallel_runs.html#manual-parallel\">wiki.fysik.dtu.dk\/gpaw\/documentation\/parallel_runs\/parallel_runs.html#manual-parallel<\/a>. For LCAO mode, try ELPA. See <a href=\"https:\/\/wiki.fysik.dtu.dk\/gpaw\/documentation\/lcao\/lcao.html#notes-on-performance\">wiki.fysik.dtu.dk\/gpaw\/documentation\/lcao\/lcao.html#notes-on-performance<\/a>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>parallel = {'augment_grids':True,'sl_auto':True,'use_elpa':True},<\/code><\/pre>\n\n\n\n<p>For calculations with vdW-functionals, use libvdwxc:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>xc = {'name':'BEEF-vdW', 'backend':'libvdwxc'},\n<\/code><\/pre>\n\n\n\n<p><strong>8. If needed, add fixes.<\/strong><\/p>\n\n\n\n<p>To do Bayesian error estimation (BEE) see <a href=\"https:\/\/doublelayer.eu\/vilab\/2022\/03\/30\/bayesian-error-estimation-for-rpbe\/\">doublelayer.eu\/vilab\/2022\/03\/30\/bayesian-error-estimation-for-rpbe\/<\/a>.<\/p>\n\n\n\n<p>To use MLMin\/NEB apply corrections from <a href=\"https:\/\/github.com\/SUNCAT-Center\/CatLearn\/pulls\">github.com\/SUNCAT-Center\/CatLearn\/pulls<\/a><\/p>\n\n\n\n<p><strong>9. Something worth trying:<\/strong><\/p>\n\n\n\n<p>Atomic Simulation Recipes:<\/p>\n\n\n\n<p><a href=\"https:\/\/asr.readthedocs.io\/en\/latest\/\">asr.readthedocs.io\/en\/latest\/<\/a><\/p>\n\n\n\n<p>gpaw-tools:<\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/lrgresearch\/gpaw-tools\/\">github.com\/lrgresearch\/gpaw-tools\/<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0927025622000155\">www.sciencedirect.com\/science\/article\/pii\/S0927025622000155<\/a><\/p>\n\n\n\n<p>ase-notebook (won&#8217;t install at FEND because of glibc 2.17):<\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/chrisjsewell\/ase-notebook\">github.com\/chrisjsewell\/ase-notebook<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/ase-notebook.readthedocs.io\/en\/latest\/\">ase-notebook.readthedocs.io\/en\/latest\/<\/a><\/p>\n\n\n\n<p>Optimizers:<\/p>\n\n\n\n<p><a href=\"https:\/\/gitlab.com\/gpatom\/ase-gpatom\">gitlab.com\/gpatom\/ase-gpatom<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/gitlab.com\/egarijo\/bondmin\/\">gitlab.com\/egarijo\/bondmin\/<\/a><\/p>\n\n\n\n<p>gpaw benchmarking:<\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/OleHolmNielsen\/GPAW-benchmark-2021\">github.com\/OleHolmNielsen\/GPAW-benchmark-2021<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/mlouhivu\/gpaw-benchmarks\">github.com\/mlouhivu\/gpaw-benchmarks<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/members.cecam.org\/storage\/presentation\/Ask_Hjorth_Larsen-1622631504.pdf\">members.cecam.org\/storage\/presentation\/Ask_Hjorth_Larsen-1622631504.pdf<\/a><\/p>\n\n\n\n<p>d4 parameters fitting:<\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/dftd4\/dftd4-fit\">github.com\/dftd4\/dftd4-fit<\/a><\/p>\n\n\n\n<p>k-point grid choosing:<\/p>\n\n\n\n<p><a href=\"https:\/\/gitlab.com\/muellergroup\/kplib\">gitlab.com\/muellergroup\/kplib<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>[Updated on 20.04.2022, 15.04.2023, 10.06.2023, 03.10.2023, 04.06.2024] In short, in a clean environment, everything should work with just five lines: Initialize conda. If it is in the .bashch, source it. If not, source &#8220;PATHTOCONDA\/miniconda3\/etc\/profile.d\/conda.sh&#8221;. For details, see the description below. 1. Install conda \u2013 software and environment management system. Here is the official instruction: docs.conda.io\/projects\/conda\/en\/latest\/user-guide\/install\/linux.html&hellip; <a class=\"read-more\" href=\"https:\/\/doublelayer.eu\/vilab\/2022\/06\/23\/installing-gpaw-with-conda\/\">Read More<\/a><\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[53,31,34],"class_list":["post-595","post","type-post","status-publish","format-standard","hentry","category-know-how","tag-gpaw","tag-hpc","tag-software"],"_links":{"self":[{"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/posts\/595","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/comments?post=595"}],"version-history":[{"count":52,"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/posts\/595\/revisions"}],"predecessor-version":[{"id":896,"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/posts\/595\/revisions\/896"}],"wp:attachment":[{"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/media?parent=595"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/categories?post=595"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/doublelayer.eu\/vilab\/wp-json\/wp\/v2\/tags?post=595"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}