Today I compiled NWChem on my IBM Intel InfiniBand cluster.\u00a0 I have no idea if my performance is optimal, but I do know that it works.<\/p>\n
Here is the secret to my success:<\/p>\n
First, you have to set some path variables to get things running.\u00a0 This is set in my home directory:<\/p>\n
cat ~\/.bashrc<\/p>\n
INTELCCROOT=\/home\/appls\/compilers\/intel\/11.0\/083<\/pre>\nINTELFCROOT=\/home\/appls\/compilers\/intel\/11.0\/081\r\n\r\nPGROOT=\/home\/appls\/compilers\/pgi\r\nPGCC=$PGROOT\/linux86-64\/8.0-5\/bin\r\nPGFLEXLM=$PGROOT\/linux86-64\/8.0\/bin\r\nLM_LICENSE_FILE=$LM_LICENSE_FILE:$PGROOT\/license.dat\r\n\r\n#MPI_HOME=\/home\/appls\/openmpi\/gcc\r\n#MPI_HOME=\/home\/appls\/openmpi\/pgi\r\nMPI_HOME=\/home\/appls\/openmpi\/intel\r\n\r\nNWCHEM=\/home\/appls\/QChem\/NWChem\/intel\/bin\r\n\r\nPATH=$PATH:$MPI_HOME\/bin:$HOME\/bin:$INTELCCROOT\/bin\/intel64:$INTELFCROOT\/bin\/intel64:$PGCC:$PGFLEXLM:$NWCHEM\r\n\r\nLD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MPI_HOME\/lib:$INTELCCROOT\/lib\/intel64:$INTELFCROOT\/lib\/intel64<\/pre>\nUnfortunately, I’m leaving out a lot of the details here, but I used the Intel compilers and had previously built openmpi.\u00a0 The way I built openmpi was:<\/p>\n
\r\n.\/configure --prefix \/home\/appls\/openmpi\/intel CC=icc CXX=icpc F77=ifort FC=ifort\r\n\r\nmake -j8\r\n\r\nmake install\r\n\r\nS<\/pre>\nhttp:\/\/wiki.cse.ucdavis.edu\/support:hpc:software:nwchem<\/pre>\nhttp:\/\/www.mcsr.olemiss.edu\/appssubpage.php?pagename=nwchem.inc<\/p>\n
So, after I got that set up, I did the normal:<\/p>\n
tar zxvf nwchem-5.1.tar.gz<\/pre>\nThen I made a script that pretty much did all the work.\u00a0 The script is called makeit.sh:<\/p>\n
export TCGRSH=\/usr\/bin\/ssh\r\nexport NWCHEM_TOP=\/home\/vallard\/qchem\/nwchem-5.1\/\r\nexport NWCHEM_TARGET=LINUX64\r\nexport USE_MPI=y\r\nexport USE_MPIF=y\r\nexport MPI_LOC=\/home\/appls\/openmpi\/intel\r\nexport MPI_LIB=$MPI_LOC\/lib\r\nexport LIBMPI=\"-L $MPI_LIB -lmpi -lopen-pal -lopen-rte -lmpi_f90 -lmpi_f77\"\r\nexport MPI_INCLUDE=$MPI_LOC\/include\r\nexport ARMCI_NETWORK=OPENIB\r\nexport LARGE_FILES\r\nexport NWCHEM_MODULES=all\r\nexport FC=ifort\r\nexport CC=icc\r\ncd $NWCHEM_TOP\/src\r\nmake CC=icc FC=ifort -j4<\/pre>\nAfter kicking that off, it ran for almost 20 minutes compiling!\u00a0 Forever!\u00a0 I saw a lot of vector loop messages, but gave them no head, and fearlessly pressed forward.<\/p>\n
After it was done compiling as root, I did the following:<\/p>\n
[root@mgt vallard]# export NWCHEM_TOP=\/home\/vallard\/qchem\/nwchem-5.1\/\r\n[root@mgt vallard]# mkdir $NWCHEM\/bin\r\n[root@mgt vallard]# mkdir $NWCHEM\/data\r\n[root@mgt vallard]# cp \/home\/vallard\/qchem\/nwchem-5.1\/bin\/LINUX64\/nwchem $NWCHEM\/bin\r\n^[[root@mgt vallard]# cp \/home\/vallard\/qchem\/nwchem-5.1\/bin\/LINUX64\/depend.x $NWEM\/bin\/\r\n[root@mgt vallard]# cd $NWCHEM_TOP\/src\/basis \r\n[root@mgt basis]# cp -r libraries $NWCHEM\/data\/\r\n[root@mgt basis]# cd $NWCHEM_TOP\/src\/\r\n[root@mgt src]# cp -r data $NWCHEM\r\n[root@mgt src]# cd $NWCHEM_TOP\/src\/nwpw\/libraryps\r\n[root@mgt libraryps]# cp -r pspw_default\u00a0 $NWCHEM\/data\/\r\n[root@mgt libraryps]# cp -r paw_default\/\u00a0 $NWCHEM\/data\/\r\n[root@mgt libraryps]# cp -r TM\u00a0 $NWCHEM\/data\/\r\n[root@mgt libraryps]# cp -r HGH_LDA\u00a0 $NWCHEM\/data\/<\/pre>\nThat got everything in place.\u00a0 When done, I went to my compute node and tried one of the examples:<\/p>\n
cd \/home\/vallard\/qchem\/nwchem-5.1\/examples\/dirdyvtst\/h3\r\nmpirun -np 32 -machinefile machinefile nwchem h3tr2.nw<\/pre>\nAfter that it all seemed to work.\u00a0 Any optimization info would be great!\u00a0 Thanks<\/p>\n","protected":false},"excerpt":{"rendered":"
Today I compiled NWChem on my IBM Intel InfiniBand cluster.\u00a0 I have no idea if my performance is optimal, but I do know that it works. Here is the secret to my success: First, you have to set some path variables to get things running.\u00a0 This is set in my home directory: cat ~\/.bashrc INTELCCROOT=\/home\/appls\/compilers\/intel\/11.0\/083…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[34,33,921],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/49"}],"collection":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=49"}],"version-history":[{"count":3,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/49\/revisions"}],"predecessor-version":[{"id":2848,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/49\/revisions\/2848"}],"wp:attachment":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=49"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=49"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}